Quoting bulia byak <buliabyak@...400...>:
According to Mental, this is how the garbage collector works - it does not release memory immediately.
This is true, but I wonder if I've mislead you a bit.
As you know, many automatic garbage collectors (like libgc) only free and recycle memory periodically. This means you may have some extra slush that could be freed, but hasn't yet.
There are other forces at work, though...
Pretty much all allocators, whether automatic or not, whether the system malloc() or some custom allocator like libgc's, work the same way: they request large blocks of memory from the operating system, then divvy those blocks into smaller ones internally to satisfy application allocation requests.
When an application frees memory, that memory is usually recycled internally rather than returned to the OS immediately. The reason for this is that the large memory blocks acquired from the OS must be completely unused before they can actually be freed.
Let's say for example that an allocator acquires 16 8MB blocks from the OS in response to 32768 4k application allocations...
In a worst-case scenario, it's possible that the application could free 32752 of those 4k blocks but the remaining 16 4k just happen to be distributed across the 16 8MB blocks requested from the OS.
If that happens, from the application's point of view it may only have 64k allocated, but as far as the OS is concerned, it's still using 128MB!
Note that this applies to nearly all allocators in common use.
While it's unusual for things to get quite that bad, memory fragmentation is common enough that many popular allocators (for example Perl's) simply don't bother trying to return memory to the OS at all (the memory will still get forcibly reclaimed by the OS when the process exits).
[ FWIW, libgc's allocator is one of the ones that _does_ make an effort to release memory to the OS, but it is limited by fragmentation like any other ]
Also note that for various reasons, the statistics you get from the OS aren't going to directly reflect the amount of heap-allocated memory. Be careful drawing conclusions from only looking at e.g. the output of top(1)...
(the worst thing is that due to the modern practice of overcommitting memory, the OS may literally lie to an application about the amount of memory it is being given, hoping the application won't really try to use it all)
The best approach to evaluating memory usage is if you can ask the allocator for information on memory usage directly, as that matches the world from the point of view of the application.
I think libgc provides means to get statistics; I don't know about the glibc allocator though. I am very tempted to add a memory usage dialog like Batik has...
-mental