Project

General

Profile

Bug #10504

zfs-test deadlist_lock leaves behind orphan destructive dtrace probe

Added by Kody Kantor 4 months ago. Updated 4 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
zfs - Zettabyte File System
Start date:
2019-03-04
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage

Description

I noticed a machine I had been running the zfs-test suite on was running slowly and consuming some CPU despite me not having interacted with the machine in a number of hours. prstat(1) showed that a dtrace process was burning the most CPU on the system.

The dtrace process in question:
kkantor@openindiana:~$ ptree 2900
2900 dtrace -qwn fbt::bpobj_decr_empty:entry { chill(500000000); }

That's imposing a pretty hefty 500ms penalty on that function call.

A 'git grep' showed that this comes from the deadlist_lock.ksh script, which forks this process and attempts to kill it before the test returns. The problem is that the script sets DTRACE_PID to the pid of the subshell used to start the dtrace script, IIUC. So the 'kill -9' in the cleanup routine kills a process that has already exited.

History

#1

Updated by Kody Kantor 4 months ago

Testing notes

Wrote a minimal runfile that only runs the specific test. Executed the test, and verified that no dtrace processes were lingering.

kkantor@openindiana:/opt/zfs-tests$ ./bin/zfstest -c ./runfiles/kkantor.run
Test: /opt/zfs-tests/tests/functional/snapshot/setup (run as root) [00:00] [PASS]
Test: /opt/zfs-tests/tests/functional/snapshot/deadlist_lock (run as root) [00:33] [PASS]
Test: /opt/zfs-tests/tests/functional/snapshot/cleanup (run as root) [00:00] [PASS]

Results Summary
PASS 3

Running Time: 00:00:35
Percent passed: 100.0%
Log directory: /var/tmp/test_results/20190305T151558
kkantor@openindiana:/opt/zfs-tests$ ps auxwww | grep dtr | grep -v grep
kkantor@openindiana:/opt/zfs-tests$

#2

Updated by Electric Monk 4 months ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit 051a710dcbf091c85ceb0f5d86bd684a64639477

commit  051a710dcbf091c85ceb0f5d86bd684a64639477
Author: Kody Kantor <kody@kkantor.com>
Date:   2019-03-07T14:54:35.000Z

    10504 zfs-test deadlist_lock leaves behind orphan destructive dtrace probe
    Reviewed by: Toomas Soome <tsoome@me.com>
    Reviewed by: Andy Fiddaman <andy@omniosce.org>
    Approved by: Dan McDonald <danmcd@joyent.com>

Also available in: Atom PDF