Bug #8558

lwp_create() returns EAGAIN on system with more than 80K ZFS filesystems

Added by Prakash Surya 3 months ago. Updated about 2 months ago.

Status:ClosedStart date:2017-08-03
Priority:NormalDue date:
Assignee:Prakash Surya% Done:

100%

Category:zfs - Zettabyte File System
Target version:-
Difficulty:Medium Tags:needs-triage

Description

On a system with more than 80K ZFS filesystems, we've seen cases where
lwp_create() will start to fail by returning EAGAIN. The problem being,
for each of those 80K ZFS filesystems, a taskq will be created for each
dataset as part of the ZIL for each dataset.

For each of these taskq's, a kernel thread will be created which results
in 24KB being allocated for each thread. With enough of these 24KB
allocations, we eventually exhaust the memory region set aside for these
allocations. Currently, segkpsize is set to a value of 2GB, which means
we can only support about 80K filesystems; 2GB / 24KB = ~80K.

The lwp_create() failure comes into play due to the fact that LWP
creation also allocates 24KB from this same region of memory. Thus, if
we've exhausted this region of memory due to the number of ZIL taskq's,
there won't be any memory avaible to allow the call to lwp_create() to
succeed.

History

#1 Updated by Prakash Surya 2 months ago

  • Subject changed from lwb_create() returns EAGAIN on system with more than 80K ZFS filesystems to lwp_create() returns EAGAIN on system with more than 80K ZFS filesystems

#2 Updated by Prakash Surya 2 months ago

  • Description updated (diff)

#3 Updated by Electric Monk about 2 months ago

  • % Done changed from 0 to 100
  • Status changed from New to Closed

git commit 216d7723a1a58124cf95c4950d51d5f99d3f4128

commit  216d7723a1a58124cf95c4950d51d5f99d3f4128
Author: Prakash Surya <prakash.surya@delphix.com>
Date:   2017-08-31T05:29:47.000Z

    8558 lwp_create() returns EAGAIN on system with more than 80K ZFS filesystems
    Reviewed by: George Wilson <george.wilson@delphix.com>
    Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
    Approved by: Robert Mustacchi <rm@joyent.com>

Also available in: Atom