Project

General

Profile

Bug #2901

ZFS receive fails for exabyte sparse files

Added by Simon Klinkert over 7 years ago. Updated about 7 years ago.

Status:
Resolved
Priority:
Normal
Category:
zfs - Zettabyte File System
Start date:
2012-06-20
Due date:
% Done:

100%

Estimated time:
Difficulty:
Bite-size
Tags:
needs-triage

Description

Our zettabyte file system has a problem with exabyte sparse files...

zfs create zg0/test1
dd if=/dev/zero of=/zg0/test1/sparsefile bs=1 count=1 seek=8000000000000000000 2> /dev/null
zfs snapshot zg0/test1@snap1
dd if=/dev/zero of=/zg0/test1/sparsefile bs=1 count=1 seek=8000000000000000001 2> /dev/null
zfs snapshot zg0/test1@snap2
zfs send zg0/test1@snap1 > /tmp/s.snap1
zfs send -i zg0/test1@snap1  zg0/test1@snap2 > /tmp/s.snap2
zfs create zg0/test2
zfs receive -d zg0/test2 < /tmp/s.snap1
cannot receive new filesystem stream: invalid backup stream

The problem is that the size of this file is too large for an uint64_t in function restore_free(). Thus, restore_free() returns EINVAL.

The EINVAL cames from here (I verified it with dtrace): http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/fs/zfs/dmu_send.c#1331

   1331     if (drrf->drr_length != -1ULL &&
   1332         drrf->drr_offset + drrf->drr_length < drrf->drr_offset)
   1333         return (EINVAL);


Related issues

Related to illumos gate - Bug #8943: ZFS send fails to dump objects with size > 128PiBNew2017-12-28

Actions

History

#1

Updated by Matthew Ahrens over 7 years ago

The size of the file is < 2^63, so it is not actually too large for a uint64_t.

from zstreamdump:

WRITE object = 8 type = 19 checksum type = 7
offset = 8000000000000000000 length = 131072 props = 200ff00ff
FREE object = 8 offset = 8000000000000131072 length = -8000000000000131072

This FREE record is the offending one. The problem is that dump_free() does not check for overflow. If offset + length overflows, it should set length to -1, indicating to free until the end of the file. It will be called with an overflowing offset+length from backup_cb, where the 'span' is very large because the tree of blocks can accomodate > 2^64 bytes.

#2

Updated by Matthew Ahrens over 7 years ago

  • Difficulty changed from Medium to Bite-size
#3

Updated by Dan McDonald about 7 years ago

  • Assignee set to Simon Klinkert
  • % Done changed from 0 to 100
#4

Updated by Dan McDonald about 7 years ago

  • Status changed from New to Resolved
#5

Updated by Ezomori Nozomu over 1 year ago

  • Related to Bug #8943: ZFS send fails to dump objects with size > 128PiB added

Also available in: Atom PDF