illumos gate

From the analysis in the Joyent bug report

The threads that are deadlocked are:

                 swtch+0x141
                 turnstile_block+0x262
                 rw_enter_sleep+0x2b7
                 as_fault+0x2a9
                 pagefault+0x96
                 trap+0xd23
                 0xfffffffffb8001d6
                 0xfffffffffb853eaa
                 uiomove+0xf8
                 dmu_read_uio+0x144
                 zfs_read+0x19b
                 fop_read+0x8b
                 hyprlofs_read+0x34
                 fop_read+0x8b
                 read+0x2a7
                 read32+0x1e
                 _sys_sysenter_post_swapgs+0x149

And:

                 swtch+0x141
                 cv_wait+0x70
                 zfs_range_lock_writer+0x54
                 zfs_range_lock+0xbb
                 zfs_putpage+0x252
                 fop_putpage+0x4c
                 segvn_sync+0x549
                 as_ctl+0x232
                 memcntl+0x4b6
                 _sys_sysenter_post_swapgs+0x149

This is essentially the read-side variant of OS-1811. Deadlock is essentially the same as the write side – except the page fault is not happening as part of elective pre-faulting action by but rather as part of the uiomove().

Note that there isn't actually a deadlock in ZFS, but rather a synthetic deadlock induced by the strict priority that waiting writers have over new readers in rwlocks. In both , it was the address space lock that induced the deadlock – a lock which is nowhere near hot enough to merit concerns over starvation. My proposed fix is to add a new kind of rwlock acquisition – RW_READER_STARVEWRITER – that does not give a shit about blocked writers; if the lock is held as reader, it will increment the reader count and return. We want to use this somewhat sparingly, of course, but the a_lock seems an easy candidate. With this fix, both bugs are fixed (properly) – and at much lower risk that reworking ZFS/VM locking dances.