Project

General

Profile

Bug #6729

incremental replication stream of a fs tree with lots of snapshots trips assert in zfs recv

Added by Lauri Tirkkonen over 4 years ago. Updated almost 2 years ago.

Status:
New
Priority:
Normal
Assignee:
-
Category:
zfs - Zettabyte File System
Start date:
2016-03-07
Due date:
% Done:

0%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage
Gerrit CR:

Description

The assertion in question is:

Assertion failed: ilen <= SPA_MAXBLOCKSIZE, file ../common/libzfs_sendrecv.c, line 1955, function recv_read

ilen comes from the dmu_replay_record_t's drr_payloadlen member:

> $c
libc.so.1`_lwp_kill+7(1, 6, 80467a8, fee91cdd, fef63000, 8046800)
libc.so.1`raise+0x22(6, 0, 80467e8, fee6edb9)
libc.so.1`abort+0xf3(8046800, 8046800, 6c, fed732cc, 65737341, 6f697472)
libc.so.1`_assert(fedabf16, fedabefa, 7a3, feda93ad)
libzfs.so.1`recv_read+0x3d(80a4548, 0, 80a8008, 1278128, 0, 8047380)
libzfs.so.1`recv_read_nvlist+0x4a(80a4548, 0, 1278128, 804721c, 0, 8047380)
libzfs.so.1`zfs_receive_package+0x100(80a4548, 0, 8047da0, 8047bdc, 80478d8, 8047380)
libzfs.so.1`zfs_receive_impl+0x527(80a4548, 8047da0, 0, 8047bdc, 0, 0)
libzfs.so.1`zfs_receive+0xc3(80a4548, 8047da0, 80a1f88, 8047bdc, 0, 0)
zfs_do_receive+0x3dd(3, 8047cac, 80768a0, 801, 0, 3)
main+0x22c(fee10180, fef6d6a8, 8047c9c, 80557f7, 4, 8047ca8)
_start+0x83(4, 8047d94, 8047d98, 8047d9d, 8047da0, 0)
> 80478d8::print dmu_replay_record_t
{
    drr_type = 0 (DRR_BEGIN)
    drr_payloadlen = 0x1278128
[ ... ]

zstreamdump reveals that all snapshot names of the source datasets are
present in the stream, which is what I suppose is making the record larger than
recv anticipates. Here's one way to repro: create a bunch of filesystems and
recursively snapshot them lots of times using long snapshot names:

# zfs create rpool/foo
# for i in $(seq 1 160); do zfs create rpool/foo/$i; done
# for i in $(seq 1 1000); do zfs snapshot -r rpool/foo@abcdefghijklmnopqrstvwxyz$i; done

Then generate an incremental replication stream between any two snapshots just
created, and attempt to receive it (doesn't matter where, and receive -n is
fine):

# zfs send -Ri abcdefghijklmnopqrstuvwxyz999 rpool/foo@abcdefghijklmnopqrstuvwxyz1000 > /var/tmp/foo.stream
# zfs recv -n rpool/foo < /var/tmp/foo.stream
Assertion failed: ilen <= SPA_MAXBLOCKSIZE, file ../common/libzfs_sendrecv.c, line 1955, function recv_read
zsh: IOT instruction (core dumped)  zfs recv -n rpool/foo < /var/tmp/foo.stream

History

#1

Updated by Chip Schweiss almost 2 years ago

Anyone looking at this?

By taking advantage of Channel Programs, this bug is becoming more frequent.

Here's a core dump from my latest encounter:

ftp://ftp.nrg.wustl.edu/pub/zfs/core_zfs_recv.gz

Also available in: Atom PDF