Project

General

Profile

Bug #5138

add tunable for maximum number of blocks freed in one txg

Added by Matthew Ahrens about 5 years ago. Updated about 5 years ago.

Status:
Closed
Priority:
Normal
Category:
zfs - Zettabyte File System
Start date:
2014-09-07
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage

Description

By limiting the maximum number of blocks freed during a single transaction
group, we can limit how much of the dedup table has to be brought into memory.
The e-mail thread referenced below suggests that destroying large numbers of
blocks in a single txg caused ZFS to run out of memory because the entire DDT
was brought into memory.


Originally proposed on the illumos-zfs mailing list, e-mail subject = "zfs
destroy memory usage / swapping to zvol", author = mahrens:

Ideally, we would add a tunable that is "maximum number of blocks to free in
one txg". Something like this patch should work; set it to around
100,000 for your workload.

History

#1

Updated by Electric Monk about 5 years ago

  • % Done changed from 0 to 100
  • Status changed from New to Closed

git commit af3465da8fa420c4ec701e3e57704d537a6f755b

commit  af3465da8fa420c4ec701e3e57704d537a6f755b
Author: Max Grossman <max.grossman@delphix.com>
Date:   2014-09-11T00:31:57.000Z

    5138 add tunable for maximum number of blocks freed in one txg
    Reviewed by: Adam Leventhal <adam.leventhal@delphix.com>
    Reviewed by: Mattew Ahrens <mahrens@delphix.com>
    Reviewed by: Josef 'Jeff' Sipek <jeffpc@josefsipek.net>
    Reviewed by: Richard Elling <richard.elling@gmail.com>
    Reviewed by: George Wilson <george.wilson@delphix.com>
    Approved by: Dan McDonald <danmcd@omniti.com>

Also available in: Atom PDF