Project

General

Profile

Actions

Bug #12829

closed

Add Zstandard to /etc/magic

Added by Andy Fiddaman over 1 year ago. Updated over 1 year ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
cmd - userland programs
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Bite-size
Tags:
Gerrit CR:

Description

At present, file test.zstd outputs data or sometimes other incorrect strings.

Zstandard files have well chosen magic bytes at the start, file should be able to correctly identify them.

Actions #1

Updated by Andy Fiddaman over 1 year ago

The format is described in https://tools.ietf.org/html/rfc8478#section-3.1.1

Magic_Number: 4 bytes, little-endian format. Value: 0xFD2FB528.

The magic number was selected to be less probable to find at the
beginning of an arbitrary file. It avoids trivial patterns (0x00,
0xFF, repeated bytes, increasing bytes, etc.), contains byte values
outside of ASCII range, and doesn't map into UTF-8 space, all of
which reduce the likelihood of its appearance at the top of a text
file.

Actions #2

Updated by Andy Fiddaman over 1 year ago

Testing:

% file /tmp/kayak_r151035.zfs.zst
/tmp/kayak_r151035.zfs.zst:     Zstandard compressed file

also file on a variety of other formats, although not exhaustive.
The new signature is pretty low down in the file and very specific so the risk of it overriding another entry is very low.

Actions #3

Updated by Electric Monk over 1 year ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit 61de0de2b543856b77c5769b0d8981f58a7f26bd

commit  61de0de2b543856b77c5769b0d8981f58a7f26bd
Author: Andy Fiddaman <omnios@citrus-it.co.uk>
Date:   2020-06-12T19:09:50.000Z

    12829 Add Zstandard to /etc/magic
    Reviewed by: Toomas Soome <tsoome@me.com>
    Reviewed by: Robert Mustacchi <rm@fingolfin.org>
    Approved by: Dan McDonald <danmcd@joyent.com>

Actions

Also available in: Atom PDF