Bug #6370


ZFS send fails to transmit some holes

Added by Matthew Ahrens almost 6 years ago. Updated over 5 years ago.

zfs - Zettabyte File System
Start date:
Due date:
% Done:


Estimated time:
Gerrit CR:


In certain circumstances, "zfs send -i" (incremental send) can produce a stream which will result in incorrect sparse file contents on the target.

The problem manifests as regions of the received file that should be sparse (and read a zero-filled) actually contain data from a file that was deleted (and which happened to share this file's object ID).

Note: this can happen only with filesystems (not zvols, because they do not free (or reuse) object IDs).

Note: This can happen only if, since the incremental source (FromSnap), a file was deleted and then another file was created, and the new file is sparse (i.e. has areas that were never written to and should be implicitly zero-filled).

We suspect that this was introduced by 4370 (applies only if hole_birth feature is enabled), and made worse by 5243 (applies if hole_birth feature is disabled, and we never send any holes).

The bug is caused by the hole birth feature. When an object is deleted and replaced, all the holes in the object have birth time zero. However, zfs send cannot tell that the holes are new since the file was replaced, so it doesn't send them in an incremental. As a result, you can end up with invalid data when you receive incremental send streams. As a short-term fix, we can always send holes with birth time 0 (unless it's a zvol or a dataset where we can guarantee that no objects have been reused).

Related issues

Related to illumos gate - Feature #4370: avoid transmitting holes during zfs sendClosedChristopher Siden2013-11-30

Related to illumos gate - Bug #5243: zdb -b could be much fasterClosedMatthew Ahrens2014-10-18

Actions #1

Updated by Electric Monk over 5 years ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit 286ef71398fb54b1d5007d6f45aa4320a9e0ede2

commit  286ef71398fb54b1d5007d6f45aa4320a9e0ede2
Author: Paul Dagnelie <>
Date:   2016-03-09T21:10:29.000Z

    6370 ZFS send fails to transmit some holes
    Reviewed by: Matthew Ahrens <>
    Reviewed by: Chris Williamson <>
    Reviewed by: Stefan Ring <>
    Reviewed by: Steven Burgess <>
    Reviewed by: Arne Jansen <>
    Approved by: Robert Mustacchi <>

Actions #2

Updated by Marcel Telka over 1 year ago

  • Related to Feature #4370: avoid transmitting holes during zfs send added
Actions #3

Updated by Marcel Telka over 1 year ago

  • Related to Bug #5243: zdb -b could be much faster added

Also available in: Atom PDF