libumem should not use instant fit allocator by default
As part of our investigations with various customers, we have found that the instant fit allocator is severely broken. The instant fit allocator favors increasing the break as opposed to trying to search for memory that is in the heap, but not in use. Because of this, we have seen several riak processes that are limiting themselves to 9 GB of heap, actually have a head of 20-40+ GB. It's important to note that the heap in use has stayed constant; however, the top of the heap continues to increase.
To measure the impact of this to other applications, Brendan went and wrote several benchmarks to study the behavior and found that not only do we do better with fragmentation but we also had higher rates of allocation.