illumos gate

The nbl_conflict() calls in vnode.c (together with related code around the calls) aims to protect files by preventing certain operation if there is conflicting reservation established on the file. For example, vn_renameat() lookups the target file, and if there is such a file available, then the NBL_REMOVE check is done to make sure we are not in conflict. If there is such a conflict, the rename operation is denied with EACCES:

1728    if (targvp && (fvp != targvp)) {
1729        nbl_start_crit(targvp, RW_READER);
1730        in_crit_targ = 1;
1731        if (nbl_conflict(targvp, NBL_REMOVE, 0, 0, 0, NULL)) {
1732            error = EACCES;
1733            goto out;
1734        }
1735    }

If there is no conflict, the rename proceeds and calls the actual rename operation for particular filesystem:

1746    /*
1747     * Do the rename.
1748     */
1749    (void) pn_fixslash(&tpn);
1750    error = VOP_RENAME(fromvp, fpn.pn_path, tovp, tpn.pn_path, CRED(),
1751        NULL, 0);

Later, in the particular VOP_RENAME() implementation (e.g. zfs_rename(), nfs4rename(), etc.), the affected directories are locked to prevent changes there and the actual rename is attempted. Unfortunately, there is a window between the nbl_conflict() call in vn_renameat() and the directory locks in functions like zfs_rename() or nfs4rename(). In this window the content of the directory could change and new conflicting target file could get created by another thread or process. Such a file didn't existed when we checked for the conflict in the vn_renameat(). If this happens, the newly created conflicting target file would be silently (and incorrectly) removed by vn_renameat().

The solution for this would be to move the nbl_conflict() check after the affected directories are locked inside zfs_rename(), nfs4rename(), etc. to make sure we do the check with the actual state of affected directory and this state cannot change until the operation is completed.

Similarly affected are all nbl_conflict() calls in vnode.c.