Project

General

Profile

Actions
Copy link

Feature #3035

closed

LZ4 compression support in ZFS and GRUB

Added by Sašo Kiselkov almost 11 years ago. Updated over 10 years ago.

Status:
Closed
Priority:
Normal
Assignee:
Sašo Kiselkov
Category:
zfs - Zettabyte File System
Start date:
2012-07-26
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage
Gerrit CR:
External Bug:

Description

LZ4 is a new high-speed BSD-licensed compression algorithm written by Yann Collet that delivers very high compression and decompression performance compared to lzjb (>50% faster on compression, >80% faster on decompression and around 3x faster on compression of incompressible data), while giving better compression ratio. For detailed benchmark numbers see the attached spreadsheet.
As such, this algorithm seems like a good candidate to integrate into Illumos' ZFS.

Files

Download all files
lz4_compression_bench.ods (22.7 KB) lz4_compression_bench.ods Benchmarks of ZFS compression performance on a 64-bit Athlon II Neo N36L CPU Sašo Kiselkov, 2012-07-26 01:14 PM
lz4_compression_bench.ods (31.2 KB) lz4_compression_bench.ods Sašo Kiselkov, 2012-12-19 12:39 PM
lz4_compression_bench.ods (31 KB) lz4_compression_bench.ods Sašo Kiselkov, 2013-01-06 06:33 PM
Actions
Copy link
 #1

Updated by Sašo Kiselkov almost 11 years ago

Webrev at http://cr.illumos.org/~webrev/skiselkov/3035/
Source can be pulled from hg staging repo at http://62.65.188.67:8001/

Actions
Copy link
 #2

Updated by Sašo Kiselkov over 10 years ago

Integrated upstream changes from LZ4 r85 - about a 5% increase in decompression speed.

Actions
Copy link
 #3

Updated by Sašo Kiselkov over 10 years ago

Integrated lz4hc into a new webrev available at:
http://cr.illumos.org/~webrev/skiselkov/3035_lz4hc/
The attached file shows the performance of the lz4hc implementation in relation to gzip.

Actions
Copy link
 #4

Updated by Christopher Siden over 10 years ago

  • Status changed from New to Closed
commit a6f561b4aee75d0d028e7b36b151c8ed8a86bc76
Author: Sašo Kiselkov <skiselkov.ml@gmail.com>
Date:   Wed Jan 16 12:36:06 2013 -0800

    3035 LZ4 compression support in ZFS and GRUB
    Reviewed by: Matthew Ahrens <mahrens@delphix.com>
    Reviewed by: Christopher Siden <christopher.siden@delphix.com>
    Reviewed by: George Wilson <george.wilson@delphix.com>
    Approved by: Christopher Siden <csiden@delphix.com>
Actions
Copy link

Also available in: Atom PDF