mountd(1m) could run out of file descriptors
The main issue
mountd(1m) is running with limit of 256 file descriptors:
# plimit $(pgrep -x mountd) 13377: /usr/lib/nfs/mountd resource current maximum time(seconds) unlimited unlimited file(blocks) unlimited unlimited data(kbytes) unlimited unlimited stack(kbytes) 10240 unlimited coredump(blocks) unlimited unlimited nofiles(descriptors) 256 65536 vmemory(kbytes) unlimited unlimited #
In a case there are many NFS clients trying to mount simultaneously we could easily run out of the file descriptors (each TCP connection to mountd would take one file descriptor). If this is the case, then every attempt to open a file in mountd will fail. We encountered failures in get_seconfig() where nfssec.conf is fopen()ed. This lead to AUTH_TOOWEAK responses to clients.
To solve this issue we should increase the file descriptor limit.
Two other minor issues were found in get_seconfig() during the work on this:
Minor issue 1
In a case there are many NFS clients it can happen that a lot of them calls get_seconfig() in mountd concurrently. The get_seconfig() contains critical section; the main reason for the critical section seems to be that the current gettoken() implementation is not MT-safe.
So, we have a lot of threads in get_seconfig(). Every thread opens the /etc/nfssec.conf file and tries to lock the mutex. If we have a lot of such threads waiting to lock the matching_lock mutex, we are just uselessly wasting file descriptors during waiting for the lock.
We are fixing this by moving the fopen() call inside the critical section.
Minor issue 2
The other minor issue is that both nfs_get_qop_name() and get_seconfig() have their own private matching_lock mutex, but both functions calls the gettoken(). This will be fixed by making matching_lock global.
Updated by Marcel Telka over 8 years ago
- Status changed from Pending RTI to Resolved
commit 97adda444bedd8afa322c1d2233957d40bc8e35c Author: Marcel Telka <firstname.lastname@example.org> Date: Wed Oct 16 08:24:11 2013 +0200 4226 mountd(1m) could run out of file descriptors Reviewed by: Gordon Ross <email@example.com> Reviewed by: Jan Kryl <firstname.lastname@example.org> Approved by: Eric Schrock <email@example.com>