Project

General

Profile

Bug #5139

SEEK_HOLE failed to report a hole at end of file

Added by Matthew Ahrens about 5 years ago. Updated about 5 years ago.

Status:
Closed
Priority:
Normal
Category:
zfs - Zettabyte File System
Start date:
2014-09-07
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
needs-triage

Description

If the file has contents up to exactly block 128^N (e.g. 1MB w/8K blocks, as with
this file), then we will misreport the hole at the end of the file as starting
at the "logical" end of the file according to the ZPL (5MB in this case),
rather than the end of the allocated blocks according to the DMU (1MB in this
case).

The DMU and ZPL can have different ideas of where the end of an object/file is.
This in itself is fine, but the problem in this case is that confusion results
when the DMU's "end of object" is smaller than the ZPL's "end of file".

In particular, the DMU has one indirect block, and all of its block pointers
are filled in (not holes) pointing to data blocks. So when asked to find a
hole, the DMU sees that there are no holes in the object, and returns ESRCH.
In this case the object ends at 1MB (128 * 8K). The ZPL interprets this as "no
holes before the end of the file", so it inserts the "virtual hole" at the end
of the file, which in this case is at 5MB. So we have missed the hole which is
between the "end of object" and "end of file".

The fix is to handle the "virtual hole" in the DMU. If no hole is found, the
DMU will return a hole at the end of the file, rather than an error.

bug can be reproduced with the following script, where "truncate" uses truncate() to set the file length, and "seek" uses lseek(SEEK_DATA/SEEK_HOLE) to list all the holes.

#!/bin/bash -x

zfs destroy test/fs

zfs create test/fs
zfs set recordsize=1k test/fs

dd if=/dev/zero of=/test/fs/1k bs=1k count=1
/home/mahrens/truncate /test/fs/1k 1000000
/home/mahrens/seek /test/fs/1k
echo "bug exhibits hole offset of 10000000 (should be 1k)"

dd if=/dev/zero of=/test/fs/2k bs=2k count=1
/home/mahrens/truncate /test/fs/2k 1000000
/home/mahrens/seek /test/fs/2k

dd if=/dev/zero of=/test/fs/127k bs=127k count=1
/home/mahrens/truncate /test/fs/127k 1000000
/home/mahrens/seek /test/fs/127k

dd if=/dev/zero of=/test/fs/128k bs=128k count=1
/home/mahrens/truncate /test/fs/128k 1000000
/home/mahrens/seek /test/fs/128k
echo "bug exhibits hole offset of 10000000 (should be 128k)"

dd if=/dev/zero of=/test/fs/129k bs=129k count=1
/home/mahrens/truncate /test/fs/129k 1000000
/home/mahrens/seek /test/fs/129k

dd if=/dev/zero of=/test/fs/16383k bs=16383k count=1
/home/mahrens/truncate /test/fs/16383k 100000000
/home/mahrens/seek /test/fs/16383k

dd if=/dev/zero of=/test/fs/16384k bs=16384k count=1
/home/mahrens/truncate /test/fs/16384k 100000000
/home/mahrens/seek /test/fs/16384k
echo "bug exhibits hole offset of 10000000 (should be 16M)"

dd if=/dev/zero of=/test/fs/16385k bs=16385k count=1
/home/mahrens/truncate /test/fs/16385k 100000000
/home/mahrens/seek /test/fs/16385k

zfs set recordsize=8k test/fs

dd if=/dev/zero of=/test/fs/1016k bs=1016k count=1
/home/mahrens/truncate /test/fs/1016k 10000000
/home/mahrens/seek /test/fs/1016k

dd if=/dev/zero of=/test/fs/1024k bs=1024k count=1
/home/mahrens/truncate /test/fs/1024k 10000000
/home/mahrens/seek /test/fs/1024k
echo "bug exhibits hole offset of 10000000 (should be 1M)"

dd if=/dev/zero of=/test/fs/1032k bs=1032k count=1
/home/mahrens/truncate /test/fs/1032k 10000000
/home/mahrens/seek /test/fs/1032k

-----------------
truncate.c

#include <unistd.h>
#include <stdlib.h>

int main(int argc, char *argv[]) {
truncate(argv1, atoi(argv2));
}

-----------------
hole.c

#include <unistd.h>
#include <stdio.h>
#include <fcntl.h>
#include <sys/stat.h>

int main(int argc, char *argv[]) {
int fd;
int offset = 0;

fd = open(argv[1], O_RDONLY);
while (1) {
offset = lseek(fd, offset, SEEK_HOLE);
printf("hole %d\\n", offset);
if (offset == -1)
break;
offset = lseek(fd, offset, SEEK_DATA);
printf("data
%d\\n", offset);
if (offset == -1)
break;
}
return (0);
}

History

#1

Updated by Electric Monk about 5 years ago

  • % Done changed from 0 to 100
  • Status changed from New to Closed

git commit 0fbc0cd0e52a11f6c4397a1714f94412cbf98b60

commit  0fbc0cd0e52a11f6c4397a1714f94412cbf98b60
Author: Matthew Ahrens <mahrens@delphix.com>
Date:   2014-09-11T07:07:42.000Z

    5139 SEEK_HOLE failed to report a hole at end of file
    Reviewed by: Adam Leventhal <adam.leventhal@delphix.com>
    Reviewed by: Alex Reece <alex.reece@delphix.com>
    Reviewed by: Christopher Siden <christopher.siden@delphix.com>
    Reviewed by: George Wilson <george.wilson@delphix.com>
    Reviewed by: Max Grossman <max.grossman@delphix.com>
    Reviewed by: Peng Dai <peng.dai@delphix.com>
    Reviewed by: Richard Elling <richard.elling@gmail.com>
    Approved by: Dan McDonald <danmcd@omniti.com>

Also available in: Atom PDF