mountd and rpcbind should use libumem(3lib)
The malloc(3c) from libc does not scale well for massively multithreaded applications.
It was found that with the stock mountd(1m) we could easily get thousands of mountd threads due to libc malloc(3c) inefficiency. The mountd(1m) becomes very slow, and could exhaust the whole virtual memory.
When the libumem(3lib) was preloaded to mountd, the mountd was able to handle the load well with less than 20 threads.
We should link mountd with libumem(3lib) by default.
The rpcbind(1m) should be linked by default with libumem(3lib) as well for similar reasons.
Updated by Marcel Telka over 7 years ago
The issue (and the fix) was found when we tested thousands of NFS clients accessing the NFS server. Without the fix the clients were able to saturate mountd and mountd had to spawn thousands of threads to try to handle the workload. Even with the thousands of mountd threads spawned, we encountered the test failures, because mountd was slow. In addition, the default thread stack size in mountd is 1MB, so we had few cases when mountd (it is 32-bit app) exhausted its virtual memory (4k threads with 1MB stack = 4GB).
Once we preloaded libumem(3lib) to mountd we saw that with the same setup (and same load), the mountd threads started to be so fast that mountd had to spawn only less than 50 threads to handle the load.
We had similar problem with rpcbind and the libumem preload fixed the performance issue there as well.
Updated by Electric Monk over 7 years ago
- Status changed from Pending RTI to Closed
- % Done changed from 0 to 100
commit 961519c5bffd5ec670890fc3596d6c4ff1cefea0 Author: Marcel Telka <firstname.lastname@example.org> Date: 2014-07-11T04:08:47.000Z 4990 mountd and rpcbind should use libumem(3lib) Reviewed by: Rob Gittins <email@example.com> Reviewed by: Igor Kozhukhov <firstname.lastname@example.org> Reviewed by: Richard Lowe <email@example.com> Reviewed by: Jason King <firstname.lastname@example.org> Approved by: Dan McDonald <email@example.com>