Project

General

Profile

Bug #8199

multi-threaded dmu_object_alloc()

Added by Matthew Ahrens over 2 years ago. Updated 10 months ago.

Status:
Closed
Priority:
Normal
Assignee:
-
Category:
-
Start date:
2017-05-10
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage

Description

dmu_object_alloc() is single-threaded, so when multiple threads are
creating files in a single filesystem, they spend a lot of time waiting
for the os_obj_lock. To improve performance of multi-threaded file
creation, we must make dmu_object_alloc() typically not grab any
filesystem-wide locks.

The solution is to have a “next object to allocate” for each CPU. Each
of these “next object”s is in a different block of the dnode object, so
that concurrent allocation holds dnodes in different dbufs. When a
thread’s “next object” reaches the end of a chunk of objects (by default
4 blocks worth -- 128 dnodes), it will be reset to the per-objset
os_obj_next, which will be increased by a chunk of objects (128). Only
when manipulating the os_obj_next will we need to grab the os_obj_lock.
This decreases lock contention dramatically, because each thread only
needs to grab the os_obj_lock briefly, once per 128 allocations.

This results in a 70% performance improvement to multi-threaded object
creation (where each thread is creating objects in its own directory),
from 67,000/sec to 115,000/sec, with 8 CPUs.

History

#1

Updated by Electric Monk 10 months ago

  • % Done changed from 0 to 100
  • Status changed from New to Closed

git commit 54811da5ac6b517992fdc173df5d605e4e61fdc0

commit  54811da5ac6b517992fdc173df5d605e4e61fdc0
Author: Toomas Soome <tsoome@me.com>
Date:   2019-02-13T16:42:18.000Z

    8423 Implement large_dnode pool feature
    8199 multi-threaded dmu_object_alloc()
    7432 Large dnode pool feature
    Reviewed by: Jerry Jelinek <jerry.jelinek@joyent.com>
    Reviewed by: Jason King <jason.king@joyent.com>
    Approved by: Dan McDonald <danmcd@joyent.com>

Also available in: Atom PDF