Project

General

Profile

Actions
Copy link

Bug #10330

closed

merge recent ZoL vdev and metaslab changes

Added by Jerry Jelinek almost 5 years ago. Updated almost 5 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Jerry Jelinek
Category:
zfs - Zettabyte File System
Start date:
2019-01-31
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage
Gerrit CR:
External Bug:

Description

3 recent ZoL changes in the vdev and metaslab code which we can pull over:
PR 8324 c853f382db 8324 Change target size of metaslabs from 256GB to 16GB
PR 8290 b194fab0fb 8290 Factor metaslab_load_wait() in metaslab_load()
PR 8286 419ba59145 8286 Update vdev_is_spacemap_addressable() for new spacemap encoding

Actions
Copy link
 #1

Updated by Electric Monk almost 5 years ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit a0b03b161c4df3cfc54fbc741db09b3bdc23ffba

commit  a0b03b161c4df3cfc54fbc741db09b3bdc23ffba
Author: Serapheim Dimitropoulos <serapheimd@gmail.com>
Date:   2019-02-11T19:07:53.000Z

    10330 merge recent ZoL vdev and metaslab changes
    Reviewed by: Matt Ahrens <mahrens@delphix.com>
    Reviewed by: Brian Behlendorf <behlendorf1@llnl.gov>
    Reviewed by: Don Brady <don.brady@delphix.com>
    Reviewed by: Gergő Mihály Doma <domag02@gmail.com>
    Reviewed by: Kody Kantor <kody.kantor@joyent.com>
    Approved by: Joshua M. Clulow <josh@sysmgr.org>

Actions
Copy link
 #2

Updated by Joshua M. Clulow almost 5 years ago

Notes from original commits and review

PR 8324 c853f382db Change target size of metaslabs from 256GB to 16GB

Pull Request: https://github.com/zfsonlinux/zfs/pull/8324

Commit: https://github.com/zfsonlinux/zfs/commit/c853f382db731e15a87512f4ef1101d14d778a55

= Old behavior

For vdev sizes 100GB to 50TB we keep ~200 metaslabs per
vdev and the metaslab size grows from 512MB to 256GB.
For vdev's bigger than that we start increasing the
number of metaslabs until we hit the 128K limit.

= New Behavior

For vdev sizes 100GB to 3TB we keep ~200 metaslabs per
vdev and the metaslab size grows from 512MB to 16GB.
For vdev's bigger than that we start increasing the
number of metaslabs until we hit the 128K limit.

= Reasoning

The old behavior makes metaslabs grow in size when
the vdev range is between 3TB (ms_size 16GB) and
32PB (ms_size 256GB). Even though keeping the number
of metaslabs is good in terms of potential number of
I/Os per TXG, these bigger metaslabs take longer
to be loaded and after they are loaded they can
take up a lot of memory because of their range trees.

This change tries to put a boundary in memory and
loading time for the specific range of vdev sizes.

PR 8290 b194fab0fb Factor metaslab_load_wait() in metaslab_load()

Pull Request: https://github.com/zfsonlinux/zfs/pull/8290

Commit: https://github.com/zfsonlinux/zfs/commit/b194fab0fb6caad18711abccaff3c69ad8b3f6d3

Most callers that need to operate on a loaded metaslab, always
call metaslab_load_wait() before loading the metaslab just in
case someone else is already doing the work.

Factoring metaslab_load_wait() within metaslab_load() makes the
later more robust, as callers won't have to do the load-wait
check explicitly every time they need to load a metaslab.

PR 8286 419ba59145 Update vdev_is_spacemap_addressable() for new spacemap encoding

Pull Request: https://github.com/zfsonlinux/zfs/pull/8286

Commit: https://github.com/zfsonlinux/zfs/commit/419ba5914552c6185afbe1dd17b3ed4b0d526547

Since the new spacemap encoding was ported to ZoL that's no longer 
a limitation. This patch updates vdev_is_spacemap_addressable() 
that was performing that check.

It also updates the appropriate test to ensure that the same 
functionality is tested.  The test does so by creating pools that 
don't have the new spacemap encoding enabled - just the checkpoint
feature. This patch also reorganizes that same tests in order to 
cut in half its memory consumption.
Actions
Copy link

Also available in: Atom PDF