Project

General

Profile

Bug #7662

assertion triggered in syseventd zfs_mod.so

Added by Hans Rosenfeld about 4 years ago. Updated about 4 years ago.

Status:
Closed
Priority:
Normal
Category:
cmd - userland programs
Start date:
2016-12-12
Due date:
% Done:

100%

Estimated time:
Difficulty:
Bite-size
Tags:
needs-triage
Gerrit CR:

Description

I've recently been given a syseventd core dump with the following stack:

feb8fc08 libc.so.1`_lwp_kill+0x15(4, 6, feb8fc38, fef65000, fef65000, feb8fca0)
feb8fc28 libc.so.1`raise+0x2b(6, 0, feb8fc40, feeb9890, 0, 0)
feb8fc78 libc.so.1`abort+0x10e(feb8fca0, feb8fca0, 83, fdc53dcc, fdc53d08, 58)
feb8fea8 0xfee6c597(fdc53dcc, fdc53d08, 58, fdc53853)
feb8fed8 libcmdutils.so.1`list_destroy+0x57(fe8741a0)
feb8ff08 zfs_mod.so`slm_fini+0x9f(2)
feb8ff48 unload_modules+0xc4(1, 805520c, feb8ff74, feb8ffac, feb8ffbc, 0)
feb8ff68 syseventd_fini+0x41(1, 1000000, 0, feb8ffac, feb8ffbc, 0)
feb8ff88 hup_handler+0x2f(1, feb8ffbc, 0, 0)
feb8ffc8 sigwait_thr+0x55(0, 0, 0, 0)
feb8ffe8 libc.so.1`_thrp_setup+0x88(fec91240)
feb8fff8 libc.so.1`_lwp_start(fec91240, 0, 0, 0, 0, 0)

The abort is caused by an assertion in list_destroy(), where it checks that the list is empty, which it isn't in this case. I conclude that there is a race against the thread used to enumerate the pools (zfs_enum_pools(), created by slm_init()). Apparently that thread is still active and adding pools to the list while we're already in slm_fini() removing entries from the list. When slm_fini() finished removing all entries it waits for the thread to complete, which can still add entries to the list while slm_fini() waits. Then slm_fini() destroys the list and run into the assertion.

I think the fix for this is to move the thr_join() call all the way up to the beginning of the function..

#1

Updated by Electric Monk about 4 years ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit cead1df3b072a636f2f644cb38b83059f9f01fe8

commit  cead1df3b072a636f2f644cb38b83059f9f01fe8
Author: Hans Rosenfeld <hans.rosenfeld@nexenta.com>
Date:   2016-12-20T10:38:45.000Z

    7662 assertion triggered in syseventd zfs_mod.so
    Reviewed by: Roman Strashkin <roman.strashkin@nexenta.com>
    Reviewed by: Yuri Pankov <yuri.pankov@nexenta.com>
    Reviewed by: Marcel Telka <marcel@telka.sk>
    Reviewed by: Dan McDonald <danmcd@omniti.com>
    Reviewed by: Igor Kozhukhov <igor@dilos.org>
    Approved by: Richard Lowe <richlowe@richlowe.net>

Also available in: Atom PDF