I only briefly skimmed through the code in tbbmalloc/MemoryAllocator.cpp, I was not trying to model the situation, but it looks like thing which can cause unbounded memory consumption in scalable_malloc.
It's about returnEmptyBlock() function. It's called from scalable_free() when we are freeing own block, and it's called from mallocThreadShutdownNotification(). But it is NOT called on 'remote free' path (neither from freePublicObject() nor getPublicFreeListBlock()).
Assume following situation. Thread 1 allocated big amount of memory (large number of small objects), then thread 2 frees them. No free blocks will be returned to global pool, all memory will remain cached in thread 1. Then 2 scenarios are possible.
(1) Thread 1 doesn't make any further calls to scalable malloc (assume it is 'main' thread which has made 'initialization' and then blocks waiting for program completion). Memory still remains cached and unused.
(2) Thread 1 makes some further calls to scalable malloc. Memory can be reused. But if thread 1 again doesn't call scalable_free() itself, free blocks will not be returned to global pool.
And unbounded memory consumption can occur if such pattern repeats recurrently. Assume thread 1 allocates memory, and thread 2 frees it. Then thread 2 allocates memory, and thread 3 frees it. Then thread 3 allocates memory, and thread 4 frees it. And so on.
The simple solution is to provide function scalable_compact_thread_heap(), which must be called by user episodically or periodically (if he notices some problems). The hard solution is to make automatic reclamation of excessive unused memory.
Am I missing something? Or the problem doesn't worth solving? What do you think?