illumos gate

While working on #11947, I tried to port and run the existing ZoL zfs diff tests to illumos. With a few minor tweaks, they all ran successfully except one test (distilled here):

zfs create -o dnodesize=4k -o encryption=on -o keyformat=passphrase test/testfs
...enter passphrase...
zfs snapshot test/testfs@1
touch /test/testfs/foo
zfs snapshot test/testfs@2
zfs diff -Ft test/testfs@1 test/testfs@2

The zfs diff command fails with something similar to:

1573328760.330576842    +    F    /test/testfs/foo
Unable to determine path or stats for object 3 in test/testfs@2: File exists

When the same sequence is run on a non-encrypted dataset:

zfs create -o dnodesize=4k test/testfs
zfs snapshot test/testfs@1
touch /test/testfs/foo
zfs snapshot test/testfs@2
zfs diff -Ft test/testfs@1 test/testfs@2

The zfs diff command succeeds:

1573328843.769111672    +    F    /test/testfs/foo
1573328843.768976616    M    /    /test/testfs/

Using a bit of dtrace -- dtrace -n 'zfs::set-error /arg0==17/ { stack() }' -- showed that dnode_hold_impl was the source of EEXIST (17). Looking at the code, the following bit of code is responsible:

    if (flag & DNODE_MUST_BE_ALLOCATED) {
        slots = 1;

        while (dn == DN_SLOT_UNINIT) {
            dnode_slots_hold(dnc, idx, slots);
            dnh = &dnc->dnc_children[idx];

            if (DN_SLOT_IS_PTR(dnh->dnh_dnode)) {
                dn = dnh->dnh_dnode;
                break;
            } else if (dnh->dnh_dnode == DN_SLOT_INTERIOR) {
                DNODE_STAT_BUMP(dnode_hold_alloc_interior);
                dnode_slots_rele(dnc, idx, slots);
                dbuf_rele(db, FTAG);
                return (SET_ERROR(EEXIST));

This can further be confirmed by examining the dnode_hold_alloc_interior zfs kstat and seeing that it is incrementing after the error.

Also interesting is that running zdb shows that there isn't an object 3 on the dataset or either snapshot (though perhaps it's a detail of the zfs diff implementation I'm just not familiar enough with).