Project

General

Profile

Bug #5408

managing ZFS cache devices requires lots of RAM

Added by Matthew Ahrens almost 5 years ago. Updated over 4 years ago.

Status:
Closed
Priority:
Normal
Category:
zfs - Zettabyte File System
Start date:
2014-12-06
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage

Description

Currently, every buffer cached in the L2ARC is accompanied by a 240-byte header in memory, leading to very high memory consumption when using very large cache devices. These changes significantly reduce this overhead.

Currently:

L1-only header = 176 bytes
L1 + L2 or L2-only header = 176 bytes + 32 byte checksum + 32 byte l2hdr = 240 bytes

Memory-optimized:

L1-only header = 176 bytes
L1 + L2 header = 176 bytes + 32 byte checksum = 208 bytes
L2-only header = 96 bytes + 32 byte checksum = 128 bytes

So overall:

Trunk   Optimized
--------------+
L1-only | 176 B | 176 B | (same)
--------------+
L1 & L2 | 240 B | 208 B | (saved 32 bytes)
--------------+
L2-only | 240 B | 128 B | (saved 116 bytes)
--------------+

For an average blocksize of 8KB, this means that for the L2ARC, the ratio of metadata to data has gone down from about 2.92% to 1.56%. For a 'storage optimized' EC2 instance with 1600GB of SSD and 60GB of RAM, this means that we expect a completely full L2ARC to use (1600 GB * 0.0156) / 60GB = 41% of the available memory, down from 78%.

History

#1

Updated by Electric Monk over 4 years ago

  • % Done changed from 0 to 100
  • Status changed from New to Closed

git commit 89c86e32293a30cdd7af530c38b2073fee01411c

commit  89c86e32293a30cdd7af530c38b2073fee01411c
Author: Chris Williamson <Chris.Williamson@delphix.com>
Date:   2014-12-30T03:12:23.000Z

    5408 managing ZFS cache devices requires lots of RAM
    Reviewed by: Christopher Siden <christopher.siden@delphix.com>
    Reviewed by: George Wilson <george.wilson@delphix.com>
    Reviewed by: Matthew Ahrens <mahrens@delphix.com>
    Reviewed by: Don Brady <dev.fs.zfs@gmail.com>
    Reviewed by: Josef 'Jeff' Sipek <josef.sipek@nexenta.com>
    Approved by: Garrett D'Amore <garrett@damore.org>

Also available in: Atom PDF