Project

General

Profile

Actions

Bug #14633

closed

lib9p: unlinkat() does not work on 9p share

Added by Andy Fiddaman 4 months ago. Updated 4 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
bhyve
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Bite-size
Tags:
Gerrit CR:

Description

A user reported that trying to do a recursive `rm -rf` on a directory tree on a 9p mount in a Linux guest is failing.

A simple reproducer which fails under Fedora 35 and Ubuntu 21.10:

# mkdir -p aaaa/bbbb
# touch aaaa/bbbb/cccc
# rm -rf aaa
rm: cannot remove 'aaaa': File exists

It turns out that rm in these modern Linux distributions is using unlinkat(), and this is failing on the
file:

# strace rm -rf /a/aaaa 2>&1 | grep unlinkat
unlinkat(5, "cccc", 0)                  = -1 EINVAL (Invalid argument)
unlinkat(4, "bbbb", AT_REMOVEDIR)       = -1 EINVAL (Invalid argument)
unlinkat(AT_FDCWD, "/a/aaaa", AT_REMOVEDIR) = -1 EEXIST (File exists)

and the debug output from lib9p

[DEBUG]  l9p_dispatch_request: Treaddir tag=0 fid=5 offset=0 count=8168
...
DEBUG]  l9p_dispatch_request: Twalk tag=0 fid=5 newfid=6 wname="cccc" 
[DEBUG]  open_fid: authinfo 1290a50 now used by 6
...
[DEBUG]  l9p_dispatch_request: Tunlinkat tag=0 dirfd=5 name="cccc" flags=0x0
[DEBUG]  l9p_respond: Rlerror tag=0 errnum=22 (Invalid argument)

Tracing through, the EINVAL from the attempt to remove the file is because the code asserts
that the directory fid is not open, which it often is for the unlinkat() call.

Actions #1

Updated by Andy Fiddaman 4 months ago

  • Description updated (diff)
Actions #2

Updated by Andy Fiddaman 4 months ago

When using unlinkat() with a directory opened with Linux's O_PATH, it succeeds, otherwise it fails:

        fd = open("/a", O_RDONLY | O_DIRECTORY);
        r = unlinkat(fd, "cccc", 0);
        printf("Ret %d errno %d\n", r, errno);
...
# ./test
Ret -1 errno 22
        fd = open("/a", O_RDONLY | O_DIRECTORY | O_PATH);
        r = unlinkat(fd, "cccc", 0);
        printf("Ret %d errno %d\n", r, errno);
...
# ./test
Ret 0 errno 0

Using AT_FDCWD also works:

        r = unlinkat(AT_FDCWD, "cccc", 0);
        printf("Ret %d errno %d\n", r, errno);
...
# cd /a; ./test
Ret 0 errno 0
Actions #3

Updated by Electric Monk 4 months ago

  • Gerrit CR set to 2111
Actions #4

Updated by Andy Fiddaman 4 months ago

In addition to this, Tunlinkat is part of 9P2000.L and is expected to return error numbers consistent with a Linux implementation. The illumos unlinkat(, AT_REMOVEDIR) call returns EEXIST when the directory is not empty, whereas Linux expects the ENOTEMPTY error code in this case.

Actions #6

Updated by Andy Fiddaman 4 months ago

Testing notes:

With the fix in place, each of the tests shown above succeeds. In a Linux guest, the unlinkat call is able to remove a file with AT_FDCWD and with a file descriptor that was opened with, and without O_PATH.

I also verified that attempting to unlink a non-empty directory now returns the linux ENOTEMPTY value (which differs from the illumos one - 39 versus 93).

[root@bhyvetest ~]# mount | grep /a
bob on /a type 9p (rw,relatime,sync,dirsync,uname=root,access=client,trans=virtio)
[root@bhyvetest ~]# cd /a
[root@bhyvetest ~]# mkdir -p b/c
[root@bhyvetest a]# strace -e unlinkat ./test
unlinkat(3, "b", AT_REMOVEDIR)          = -1 ENOTEMPTY (Directory not empty)
Ret -1 errno 39
Actions #7

Updated by Electric Monk 4 months ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit 1e6b83029f8d7ea1ade06314dc14e2fbd0cd2bcb

commit  1e6b83029f8d7ea1ade06314dc14e2fbd0cd2bcb
Author: Andy Fiddaman <omnios@citrus-it.co.uk>
Date:   2022-04-23T21:18:08.000Z

    14633 lib9p: unlinkat() does not work on 9p share
    Reviewed by: Marco van Wieringen <mvw@planets.elm.net>
    Reviewed by: Jorge Schrauwen <registration@blackdot.be>
    Reviewed by: Andrew Stormont <andyjstormont@gmail.com>
    Approved by: Gordon Ross <gordon.w.ross@gmail.com>

Actions

Also available in: Atom PDF