Project

General

Profile

Bug #5176

lock contention on godfather zio

Added by Matthew Ahrens about 5 years ago. Updated about 5 years ago.

Status:
Closed
Priority:
Normal
Category:
zfs - Zettabyte File System
Start date:
2014-09-17
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage

Description

With a workload that issues a lot of prefetch i/os (e.g. sequential read of 8KB
blocks), we see lock contention on the "godfather" zio, because all prefetch
i/os are children of this one io.

By creating multiple "godfathers", we can improve performance by ~5%.

On this sequential 8KB read workload, this lock is the 2nd hottest (see zio_*)
below:

Adaptive mutex spin: 18788455 events in 100.056 seconds (187780 events/sec)

Count indv cuml rcnt nsec Lock Caller
-------------------------------------------------------------------------------
5615389 30% 30% 0.00 14648 ARC_mru+0x58 add_reference+0x7b
4955720 26% 56% 0.00 14410 ARC_mru+0x58 remove_reference+0x63
3326997 18% 74% 0.00 17920 ARC_mru+0x58 arc_change_state+0x255
908109 5% 79% 0.00 5853 0xffffff431f139580 zio_remove_child+0x50
847154 5% 83% 0.00 2874 0xffffff431f425990 vdev_queue_io_remove+0x4d
845653 5% 88% 0.00 4601 0xffffff431f139580 zio_add_child+0x61
506744 3% 91% 0.00 5592 0xffffff431f139580 zio_notify_parent+0x4f

History

#1

Updated by Electric Monk about 5 years ago

  • % Done changed from 0 to 100
  • Status changed from New to Closed

git commit 6f834bc197c703a6568554c889157fb345bac079

commit  6f834bc197c703a6568554c889157fb345bac079
Author: Matthew Ahrens <mahrens@delphix.com>
Date:   2014-10-06T01:11:24.000Z

    5176 lock contention on godfather zio
    Reviewed by: Adam Leventhal <ahl@delphix.com>
    Reviewed by: Alex Reece <alex.reece@delphix.com>
    Reviewed by: Christopher Siden <christopher.siden@delphix.com>
    Reviewed by: George Wilson <george.wilson@delphix.com>
    Reviewed by: Richard Elling <richard.elling@gmail.com>
    Reviewed by: Bayard Bell <Bayard.Bell@nexenta.com>
    Approved by: Garrett D'Amore <garrett@damore.org>

Also available in: Atom PDF