Feature #3035

LZ4 compression support in ZFS and GRUB

Added by Sašo Kiselkov over 4 years ago. Updated about 4 years ago.

Status:ClosedStart date:2012-07-26
Priority:NormalDue date:
Assignee:Sašo Kiselkov% Done:

100%

Category:zfs - Zettabyte File System
Target version:-
Difficulty:Medium Tags:needs-triage

Description

LZ4 is a new high-speed BSD-licensed compression algorithm written by Yann Collet that delivers very high compression and decompression performance compared to lzjb (>50% faster on compression, >80% faster on decompression and around 3x faster on compression of incompressible data), while giving better compression ratio. For detailed benchmark numbers see the attached spreadsheet.
As such, this algorithm seems like a good candidate to integrate into Illumos' ZFS.

lz4_compression_bench.ods - Benchmarks of ZFS compression performance on a 64-bit Athlon II Neo N36L CPU (22.7 KB) Sašo Kiselkov, 2012-07-26 01:14 PM

lz4_compression_bench.ods (31.2 KB) Sašo Kiselkov, 2012-12-19 12:39 PM

lz4_compression_bench.ods (31 KB) Sašo Kiselkov, 2013-01-06 06:33 PM

History

#1 Updated by Sašo Kiselkov over 4 years ago

Webrev at http://cr.illumos.org/~webrev/skiselkov/3035/
Source can be pulled from hg staging repo at http://62.65.188.67:8001/

#2 Updated by Sašo Kiselkov over 4 years ago

Integrated upstream changes from LZ4 r85 - about a 5% increase in decompression speed.

#3 Updated by Sašo Kiselkov about 4 years ago

Integrated lz4hc into a new webrev available at:
http://cr.illumos.org/~webrev/skiselkov/3035_lz4hc/
The attached file shows the performance of the lz4hc implementation in relation to gzip.

#4 Updated by Christopher Siden about 4 years ago

  • Status changed from New to Closed
commit a6f561b4aee75d0d028e7b36b151c8ed8a86bc76
Author: Sašo Kiselkov <skiselkov.ml@gmail.com>
Date:   Wed Jan 16 12:36:06 2013 -0800

    3035 LZ4 compression support in ZFS and GRUB
    Reviewed by: Matthew Ahrens <mahrens@delphix.com>
    Reviewed by: Christopher Siden <christopher.siden@delphix.com>
    Reviewed by: George Wilson <george.wilson@delphix.com>
    Approved by: Christopher Siden <csiden@delphix.com>

Also available in: Atom