Project

General

Profile

Bug #7505

dtrace helpers leaked during fork when lwp_create fails

Added by Alek Pinchuk almost 4 years ago. Updated almost 4 years ago.

Status:
Closed
Priority:
Normal
Category:
DTrace
Start date:
2016-10-24
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

When running our stress tests on NS we noticed leaks in dtrace subsystem which after 24 hours had gobbled up 1GB.
Rerunning with kmem_flags enabled for about 24 hrs, here's what we got.

::findleaks ! sort -n +1 -2
------------------------------
CACHE LEAKED BUFCTL CALLER
fffff042a5a67008 1 fffff045598b6018 ddi_prop_decode_alloc+0x29
fffff042a5a67008 1 fffff045598b7cc8 ddi_prop_decode_alloc+0x29
fffff042a5a67008 1 fffff045598b7da0 ddi_prop_decode_alloc+0x29
fffff042a5a67008 1 fffff045598b7e78 ddi_prop_decode_alloc+0x29
fffff042a5a67008 1 fffff045bfd24058 devid_scsi_init+0x6d
fffff042a5a6a008 1 fffff045597b5988 devid_scsi_init+0x6d
fffff042a5a6a008 1 fffff045597b5b38 ddi_prop_decode_alloc+0x29
<elided for clarity>
fffff042a5a90008 857739 fffff1c4fbf60bf0 dtrace_difo_duplicate+0x47
fffff042a5a81008 1002949 fffff04c198f2dd0 dtrace_difo_duplicate+0xe2
fffff042a5a67008 1026060 fffff301d25e3738 dtrace_difo_duplicate+0x72
fffff042a5a67008 1097019 fffff28cb67c4810 dtrace_difo_duplicate+0x47
fffff042a5a67008 1116948 fffff259efd29708 dtrace_difo_duplicate+0xe2
fffff042a5a61008 1122206 fffff04809010a70 dtrace_difo_duplicate+0x72
fffff042a5a61008 1125635 fffff05cd4f76450 dtrace_difo_duplicate+0xa7
fffff042a5a6a008 1230050 fffff064be45d010 dtrace_difo_duplicate+0xa7
fffff042a5a76008 2267906 fffff264cb9ef7b0 dtrace_difo_duplicate+0x32
Total 13106171 <tel:13106171> buffers, 894942372 bytes

fffff264cb9ef7b0::bufctl -v
ADDR BUFADDR TIMESTAMP THREAD
CACHE LASTLOG CONTENTS
fffff264cb9ef7b0 fffff05e08a7f700 881008ca98044 fffff83db37433e0
fffff042a5a76008 fffff04397d93b40 0
kmem_cache_alloc_debug+0x2e0
kmem_cache_alloc+0x2d0
kmem_zalloc+0x47
dtrace_difo_duplicate+0x32
dtrace_helpers_duplicate+0xec
cfork+0x9e2
forksys+0x3c

rmustacc pointed out the source of this leak in an email exchange
"I took a look at the code in cfork() and it looks like it's possible for a
fork failure to not properly clean up the duplication. It looks like
we're missing something in the forklwperr label path."

The fix relatively simple, it adds cleanup to the forklwperr path. See attached patch.


Files

cfork_leak.patch (4.67 KB) cfork_leak.patch Alek Pinchuk, 2016-10-24 09:54 PM

History

#1

Updated by Rich Lowe almost 4 years ago

  • Subject changed from dtrace helpers can lean when cfork() fails to dtrace helpers can leak when cfork() fails
#2

Updated by Alek Pinchuk almost 4 years ago

  • Assignee deleted (Alek Pinchuk)
#3

Updated by Robert Mustacchi almost 4 years ago

  • Subject changed from dtrace helpers can leak when cfork() fails to dtrace helpers leaked during fork when lwp_create fails
  • Assignee set to Patrick Mooney
  • % Done changed from 80 to 100
#4

Updated by Electric Monk almost 4 years ago

  • Status changed from In Progress to Closed

git commit 9d7cab140913ca66246ee319c3fba0feb52604a8

commit  9d7cab140913ca66246ee319c3fba0feb52604a8
Author: Patrick Mooney <pmooney@pfmooney.com>
Date:   2016-11-17T01:03:59.000Z

    7505 dtrace helpers leaked during fork when lwp_create fails
    Reviewed by: Bryan Cantrill <bryan@joyent.com>
    Reviewed by: Ryan Zezeski <rpz@joyent.com>
    Reviewed by: Adam Leventhal <adam.leventhal@gmail.com>
    Reviewed by: Sanjay Nadkarni <sanjay.nadkarni@nexenta.com>
    Reviewed by: Alek Pinchuk <pinchuk.alek@gmail.com>
    Approved by: Richard Lowe <richlowe@richlowe.net>

Also available in: Atom PDF