Bug #11928
rpcmod's clnt_cots can do zero-length kmem allocations
100%
Description
While testing NFS service in a zone using a DEBUG SmartOS, I discovered rpcmod was calling kmem_alloc with zero-length. While examining the clnt_cots.c code, I also found some unnecessary zalloc(), when a plain alloc() would do.
This bug is only scoped on $UTS/common/rpc/clnt_cots.c. A fuller audit of rpcmod is in order, but should be filed separately, after all distros have the functionality in #11927.
Related issues
History
Updated by Dan McDonald 16 days ago
Updated by - Assignee set to Dan McDonald
NFSv4 hammering (callback or seems to trigger the zero-length allocations. Here's a list of stacks during one test:
T-0.000000000 addr=0 0
kmem_log_event+0x9b
kmem_alloc+0x17d
kmem_zalloc+0x10d
connmgr_get+0x49b
connmgr_wrapget+0x58
clnt_cots_kcallit+0x16b
rfs4_do_callback+0x97
rfs4_do_cb_recall+0xfc
do_recall+0x1af
thread_start+0xb
T-2967.420986776 addr=0 0
kmem_log_event+0x9b
kmem_alloc+0x17d
kmem_zalloc+0x10d
connmgr_get+0x49b
connmgr_wrapget+0x58
clnt_cots_kcallit+0x16b
rfs4_do_callback+0x97
rfs4_do_cb_recall+0xfc
do_recall+0x1af
thread_start+0xb
T-8424.755340132 addr=0 0
kmem_log_event+0x9b
kmem_alloc+0x17d
kmem_zalloc+0x10d
connmgr_get+0x49b
connmgr_wrapget+0x58
clnt_cots_kcallit+0x16b
rfs4_do_callback+0x97
rfs4_do_cb_recall+0xfc
do_recall+0x1af
thread_start+0xb
T-9263.085631357 addr=0 0
kmem_log_event+0x9b
kmem_alloc+0x17d
kmem_zalloc+0x10d
connmgr_get+0x49b
connmgr_wrapget+0x58
clnt_cots_kcallit+0x16b
rfs4_do_cb_null+0x11e
thread_start+0xb
T-16528.165042251 addr=0 0
kmem_log_event+0x9b
kmem_alloc+0x17d
kmem_zalloc+0x10d
connmgr_get+0x49b
connmgr_wrapget+0x58
clnt_cots_kcallit+0x16b
rfs4_do_cb_null+0x11e
thread_start+0xb
Updated by Dan McDonald 15 days ago
Tested by re-running a nightly build using an nfs-zone-served zone on a DEBUG kernel. Before this fix, there were results from mdb's `::kmalog zerosized` on SmartOS, after this fix, none. See #11927 for the SmartOS functionality in question.
Updated by
Updated by Marcel Telka 15 days ago
From the kmem_alloc(9f) man page:
NOTES
kmem_alloc(0, flag) always returns NULL. kmem_free(NULL, 0) is legal.
Updated by Marcel Telka 15 days ago
- Related to Bug #11927: Log, or optionally panic, on zero-length kmem allocations added
Updated by Dan McDonald 10 days ago
The output mentioned in comment 1 is from this mdb command from bug #11927:
::kmalog zerosized
Here are before (dump 3) and after (dump 4) tests:
root@larry (kebecloud) /var/crash/volatile]# echo '::kmalog zerosized' | mdb 3
mdb: warning: dump is from SunOS 5.11 joyent_20191022T221350Z; dcmds and macros may not match kernel implementation
T-0.000000000 addr=0 0
kmem_log_event+0x9b
kmem_alloc+0x17d
kmem_zalloc+0x10d
connmgr_get+0x49b
connmgr_wrapget+0x58
clnt_cots_kcallit+0x16b
rfs4_do_cb_null+0x11e
thread_start+0xb
[root@larry (kebecloud) /var/crash/volatile]# echo '::kmalog zerosized' | mdb 4
[root@larry (kebecloud) /var/crash/volatile]#
Updated by Electric Monk 7 days ago
Updated by - Status changed from New to Closed
- % Done changed from 0 to 100
git commit 4b6bffb4c4308c6219c095d4cf5bf96bb0970e04
commit 4b6bffb4c4308c6219c095d4cf5bf96bb0970e04
Author: Dan McDonald <danmcd@joyent.com>
Date: 2019-11-14T15:00:28.000Z
11928 rpcmod's clnt_cots can do zero-length kmem allocations
Reviewed by: Matthias Scheler <Matthias.Scheler@wdc.com>
Reviewed by: Toomas Soome <tsoome@me.com>
Approved by: Gordon Ross <gordon.w.ross@gmail.com>