-
Notifications
You must be signed in to change notification settings - Fork 24
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add a profiling tool in GnuCOBOL #110
Add a profiling tool in GnuCOBOL #110
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you for your work on this. Apart from formatting as centered/number, I fail to see a use in the xlsx - please note which benefits you see in that.
The created .csv will work with different table calculation programs (and on the command line and also as a log that's easy to query and/or use in log collection), so this should be definitely the default. I'm actually not sure if we should keep xlsx at all (note: it would be possible to provide a macro file for different programs that load a created perf.csv and nicely format it on the fly; and if we move it out, then you'd still be able to provide a file to be LD_PRELOADED [possibly COB_PRE_LOAD works as well?] for writing XSLT files).
From the data part: shouldn't CALLs be profiled separately, allowing to see (also for programs not compiled with profiling / non COBOL functions) how long the time is spent in there? This would also allow an output "self/inclusive".
Codecov ReportAttention: Patch coverage is
❗ Your organization needs to install the Codecov GitHub app to enable full functionality. Additional details and impacted files@@ Coverage Diff @@
## gcos4gnucobol-3.x #110 +/- ##
=====================================================
+ Coverage 65.87% 66.06% +0.19%
=====================================================
Files 32 33 +1
Lines 59483 59947 +464
Branches 15709 15778 +69
=====================================================
+ Hits 39182 39603 +421
- Misses 14282 14307 +25
- Partials 6019 6037 +18 ☔ View full report in Codecov by Sentry. |
Thank you for the review. |
We commonly don't agree with proprietary COBOL compilers on many points, and as there is no real feature in that and people can open the csv in Excel - let's drop that format and the added checks, code and docs for the dependencies. Note that csv and json/xml are also much more preferable if you want to use the data in your test or even CI environment. |
The choice is not between csv and xls, but to have both of them for different users: unix hackers will choose csv and use additional tools or scripts to extract data, but windows devs might prefer xls, because it is easier to view (csv always ask about the format to use to parse the file) and easier to graphically customize (adding colors to highlight the most important values, etc.). I think it would be better to have xls now, and then to drop it when powerful tools are available to post process csv files, rather than having only csv output now and never seeing such tools appear, or hard to find in the contribs... |
Excel is also able to open Open Document Spreadsheeds (ods files), I'd be ok with adding this, but again - to reduce the complexity of the changes I suggest to split this into a second PR, one "just the feature", then another "adding a format which can be chosen by the user". |
Ok I will make this PR "feature only" with csv file generation only, and make another one later with ODS file format. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should have an extra variable that needs to be set to turn profiling on - like COB_SET_TRACE
(this way you can compile it in but can selectively enable it, both for a specific process or even - with code adjustments of SET ENVIRONMENT...
for only part of run.
In this case the "counting" functions would check this flag.
This would also be different from tools like gprof where we can adjust the profiling only per source unit.
libcob/cobprof.c
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As we do have a single file in the runtime for this feature - please use it to give an outline how the profiling works (an overview and which function is called where/why, maybe similar to what strings.c has before cob_string_init
; each function should get a doc comment which may also provide more detailed information).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
pending TODO
And, as a "general" thought: instead of resolving the "textual" data in libcob and storing it there, this part should likely be postponed until there's an actual write to the profiling file (when all the section/paragraph/location info can be requested/read). Switching to this approach should both save memory during collection and reduce the profiling overhead. |
@lefessan If I understood this right, you're likely to take on this addition next, right? |
Yes, I have started fixing some of the issues you raised, but I still need some more time to get a better global picture of the patch before doing further changes. |
Any chance on getting this done in the next two weeks? |
Ok, I will do it or see if somebody else in the team can ! |
f09c48d
to
44c6e92
Compare
4b2ac13
to
42459b3
Compare
Hi @GitMensch , |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I consider this my final review of this PR - the overall state is "very good" already.
Feel free to directly commit this after taking the review comments into account.
Thank you all for your patience!
89a3a6b
to
fe48125
Compare
Initial work by Emilien Lemaire <emilien.lemaire@ocamlpro.com> Additional features: * Do not check for enter/exit section: instead, sum time from paragraphs * Support for modules, CALL and ENTRY points * Support for recursive calls * Allocate virtual stack on demand instead of statically * Correct handling of EXIT PARAGRAPH code with 'goto' * Prevent CANCEL from dlclose a module during profiling * Customize CSV result file with COB_PROF_FORMAT * Customize CSV filename using $b/$f/$d/$t * Add some tests for RECURSIVE on PROGRAM-ID
* fix compile under WIN32 and systems without CLOCK_MONOTONIC / CLOCK_MONOTONIC_RAW * fix use of defined but unavailable CLOCK_MONOTONIC_RAW * minor refactoring
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thank you all for the work on this. The PR is good to be merged uptream now (with the two commented changes either in or get in later).
I think that's it ? If you're okay with the way I handled the change about relative-to-absolute path conversion. |
That's fine, just add a changelog entry for the function extraction when doing the check-in "upstream". For the test - I'd tend to have two |
Well, the runtime outputs |
closed as it is merged upstream (it would be good to update the git mirror of course, but that's a different thing. |
checking common.h I've recognized that there is one change that should be applied here: the profiling types should be "anonymous" in common.h (then doesn't need more than @ddeclerck Sorry for not catching this beforehand - can you please adjust the code with a follow-up commit? |
Sadly #140 was not pulled in before - the VC CI builds are broken since 23 days -as there is a new file and an adjustment to libcob/Makefile.am - but no changes to build_windows//libcob - can you please fix these (similar to r5112 - 8bc9fed) and change to use the anonymous type? Oh - and while working on this - GCC says:
so please check the paths (most times but not always gcc is right) and only if it is a false-positive initialize the variable to zero. |
closed per follow-up PR "nearly checked in" |
This PR would add a simple profiling tool in GnuCOBOL which computes the time spent in each section and each paragraph of the COBOL program.