Project

General

Profile

Bug #9577

remove zfs_dbuf_evict_key tsd

Added by Brad Lewis over 1 year ago. Updated about 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
zfs - Zettabyte File System
Start date:
2018-06-02
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage

Description

The zfs_dbuf_evict_key TSD (thread-specific data) is not necessary - we can instead pass a flag down in a few places to prevent recursive dbuf eviction. Making this change has 3 benefits:

1. The code semantics are easier to understand.
2. On Linux, performance is improved, because creating/removing TSD values (by setting to NULL vs non-NULL) is expensive, and we do it very often.
3. According to Nexenta, the current semantics can cause a deadlock when concurrently calling dmu_objset_evict_dbufs() (which is rare today, but they are working on a "parallel unmount" change that triggers this more easily):

Thread A is evicting dbufs that are related to dnodeA
dnode_evict_dbufs(dnoneA) enters dn_dbufs_mtx after that walks the AVL of dbufs and calls:
dbuf_destroy()->...->dbuf_evict_notify()->dbuf_evict_one()->select a dbuf from cache->dbuf_destroy()->mutex_enter(dn_dbufs_mtx of dnoneB)

Thread B is evicting  dbufs that are related to dnodeB
dnode_evict_dbufs(dnodeB) enters dn_dbufs_mtx, after that walks the AVL of dbufs and calls:
dbuf_destroy()->...->dbuf_evict_notify()->dbuf_evict_one()->select a dbuf from cache->dbuf_destroy()->mutex_enter(dn_dbufs_mtx of dnodeA)
> $C
ffffd001eb9b8570 vpanic()
ffffd001eb9b8610 0xfffffffffbb2d77a()
ffffd001eb9b8680 mutex_vector_enter+0x3a3(ffffd06372a15498)
ffffd001eb9b86e0 dbuf_destroy+0x1b3(ffffd063729427e8)
ffffd001eb9b8700 dbuf_evict_one+0xae()
ffffd001eb9b8720 dbuf_evict_notify+0xb9()
ffffd001eb9b87f0 dbuf_rele_and_unlock+0x2b1(ffffd063748e0258, ffffd063b28b3e10)
ffffd001eb9b8830 dbuf_rele+0x30(ffffd063748e0258, ffffd063b28b3e10)
ffffd001eb9b8890 dbuf_destroy+0x168(ffffd063b28b3e10)
ffffd001eb9b89c0 dnode_evict_dbufs+0x120(ffffd063b432b560, ffffffff)
ffffd001eb9b8cb0 dmu_objset_evict_dbufs+0x83(ffffd06372540b00)
ffffd001eb9b8d10 zfsvfs_teardown+0x142(ffffd063c42db000, 1)
ffffd001eb9b8d70 zfs_umount+0x10c(ffffd063a44b6028, 0, ffffd06332a1b528)
ffffd001eb9b8da0 fsop_unmount+0x1b(ffffd063a44b6028, 0, ffffd06332a1b528)
ffffd001eb9b8df0 dounmount+0x57(ffffd063a44b6028, 0, ffffd06332a1b528)
ffffd001eb9b8e40 umount2_engine+0x96(ffffd063a44b6028, 0, ffffd06332a1b528, 1)
ffffd001eb9b8ec0 umount2+0x163(819e1e8, 0)
ffffd001eb9b8f10 _sys_sysenter_post_swapgs+0x149()

History

#1

Updated by Electric Monk about 1 year ago

  • % Done changed from 0 to 100
  • Status changed from New to Closed

git commit c2919acbea007fa95c709b60d073db9a24526e01

commit  c2919acbea007fa95c709b60d073db9a24526e01
Author: Matthew Ahrens <mahrens@delphix.com>
Date:   2018-07-19T15:54:33.000Z

    9577 remove zfs_dbuf_evict_key tsd
    Reviewed by: George Wilson <george.wilson@delphix.com>
    Reviewed by: Serapheim Dimitropoulos <serapheim.dimitro@delphix.com>
    Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed by: Andy Stormont <astormont@racktopsystems.com>
    Approved by: Richard Lowe <richlowe@richlowe.net>

Also available in: Atom PDF