memory profiling feature request

[rdp]

I've noticed my app with crystal 0.32.1 seems to "monotonically increase" in the amount of RAM it uses over time. I can use valgrind to find "real C level memory leaks" but if the source is in crystal land (something hanging onto too many references) it's difficult to detect the culprit.

I wish there were a way to have an object graph showing relationships between objects and which ones are holding onto other objects. So similar to java jhat's "browse the memory space" functionality (which shows classes, references).

Or, barring that, a count of each class instantiated so I could narrow down the search. So similar to jhat's "histogram" of allocated classes, or ruby's ObjectSpace.count_objects.

I realize this might require a new GC or what not, but eventually it would be really useful for debugging, though possibly not the end all for my particular problem. Thanks.

[rdp]

Somewhat related: #5271

[jwoertink]

Yes, I'd love this so much right now. related. I saw Benchmark has a memory method, but it doesn't quite give enough information when your app starts to just run out of memory. And no one on my team knows C, so looking at something like valgrind becomes quite the task. I think starting with something super simple that just says "You instantiated X amount of objects for a total of 29203kb" would be a great start. Then we could throw that in to web frameworks for debugging to see if anything is leaking, or getting out of control with memory usage.

[rdp]

See also #3997 somewhat related

[rdp]

This will require either a new GC option (naive mark and sweep perhaps), or some significant improvements to bdw, I don't see much of an option there for it at present...

There is GC.stats => GC::Stats(@heap_size=262144, @free_bytes=204800, @unmapped_bytes=0, @bytes_since_gc=7296, @total_bytes=7296) at present, FWIW :)

[rdp]

I wonder if one low ball way to get this would be "in debug mode" count how many of each type of object are currently allocated (decrement some counter in a finalizer). Just a thought...

[jwoertink]

Yeah, I think we just need an initial anything. We don't need to start out with everything under the sun. Just something super basic to get people started, and build on top over time. That would be great for all these frameworks out there to see something like "oh, we're allocating 4 copies of this, when we should only be doing it once" type of stuff.

[rdp]

bdw has some options, for the adventurous: https://github.com/ivmai/bdwgc/blob/master/doc/debugging.md gc_dump, and maybe https://github.com/ivmai/bdwgc/blob/de8e2e986039495ed4416dcb61fe4fb34026a2dc/doc/porting.md gc_save_callers see also ivmai/bdwgc#360

[HertzDevil]

I wish there were a way to have an object graph showing relationships between objects and which ones are holding onto other objects. So similar to java jhat's "browse the memory space" functionality (which shows classes, references).

crystal-lang/perf-tools#5 will probably achieve this.

Or, barring that, a count of each class instantiated so I could narrow down the search. So similar to jhat's "histogram" of allocated classes, or ruby's ObjectSpace.count_objects.

This should now be possible with PerfTools::MemProf.log_object_counts from that repo.