Project

General

Profile

Bug #8544

syseventd's zfs_mod dies trying to concurrently access libzfs_ns_avl

Added by Yuri Pankov about 2 years ago. Updated about 2 years ago.

Status:
Rejected
Priority:
Normal
Assignee:
-
Category:
-
Start date:
2017-07-27
Due date:
% Done:

0%

Estimated time:
Difficulty:
Bite-size
Tags:

Description

we have syseventd dying intermittently with the following trace:

Loading modules: [ libc.so.1 libumem.so.1 libtopo.so.1 libavl.so.1 libuutil.so.1 libsysevent.so.1 libnvpair.so.1 libcmdutils.so.1 ld.so.1 ]
> > ::status
debugging core file of syseventd (32-bit) from NodeA
file: /usr/lib/sysevent/syseventd
initial argv: /usr/lib/sysevent/syseventd
threading model: native threads
status: process terminated by SIGSEGV (Segmentation Fault), addr=60fe0381
> > ::stack
libuutil.so.1`avl_first+0xd(8139540, 3, fdd8efc8, fe79d77f, 8, fe9f4000)
libzfs.so.1`zpool_iter+0x4c(81136d0, fe9e3562, fe9f4198, fe9e2c55, fef60000, fec74a40)
zfs_mod.so`zfs_enum_pools+0x2f(0, 0, 0, 0)
libc.so.1`_thrp_setup+0x88(fec74a40)
libc.so.1`_lwp_start(fec74a40, 0, 0, 0, 0, 0)

apparently, there's a low possibility race between zpool_iter() and namespace_reload() - one thread tries to access libzfs_ns_avl contents while another one is actively modifying it:

stack pointer for thread 17: fdd8ef58
[ fdd8ef58 libuutil.so.1`avl_first+0xd() ]
  fdd8efa8 libzfs.so.1`zpool_iter+0x4c()
  fdd8efc8 zfs_mod.so`zfs_enum_pools+0x2f()
  fdd8efe8 libc.so.1`_thrp_setup+0x88()
  fdd8eff8 libc.so.1`_lwp_start()
stack pointer for thread 18: fde9d5d8
[ fde9d5d8 libc.so.1`strcmp+0x191() ]
  fde9d618 libnvpair.so.1`nvlist_remove_all+0x55()
  fde9d658 libnvpair.so.1`nvlist_add_common+0x2f6()
  fde9d688 libnvpair.so.1`nvlist_copy_pairs+0x45()
  fde9d6b8 libnvpair.so.1`nvlist_copy_embedded+0x4b()
  fde9d6f8 libnvpair.so.1`nvlist_add_common+0x286()
  fde9d728 libnvpair.so.1`nvlist_copy_pairs+0x45()
  fde9d758 libnvpair.so.1`nvlist_copy_embedded+0x4b()
  fde9d798 libnvpair.so.1`nvlist_add_common+0x222()
  fde9d7c8 libnvpair.so.1`nvlist_copy_pairs+0x45()
  fde9d808 libnvpair.so.1`nvlist_xdup+0x49()
  fde9d828 libnvpair.so.1`nvlist_dup+0x26()
  fde9ea48 libzfs.so.1`namespace_reload+0x328()
  fde9ea98 libzfs.so.1`zpool_iter+0x2a()
  fde9eef8 zfs_mod.so`devid_iter+0xdb()
  fde9ef28 zfs_mod.so`zfs_deliver_add+0x63()
  fde9ef78 zfs_mod.so`zfs_deliver_event+0x239()
  fde9efc8 client_deliver_event_thr+0x109()
  fde9efe8 libc.so.1`_thrp_setup+0x88()
  fde9eff8 libc.so.1`_lwp_start()

History

#1

Updated by Yuri Pankov about 2 years ago

  • Subject changed from libzfs namespace needs proper locking to syseventd's zfs_mod dies trying to concurrently access libzfs_ns_avl
  • Category deleted (zfs - Zettabyte File System)
  • Status changed from In Progress to Feedback
  • Assignee deleted (Yuri Pankov)
  • % Done changed from 50 to 0
#2

Updated by Yuri Pankov about 2 years ago

  • Status changed from Feedback to Rejected

Also available in: Atom PDF