zfs_rmnode() fails to clean up znodes
Take a look at the following comment at http://src.illumos.org/source/xref/illumos-gate/usr/src/uts/common/fs/zfs/zfs_dir.c#652
In function zfs_rmnode() :
651 /* 652 * Not enough space to delete the file. Leave it in the 653 * unlinked set, leaking it until the fs is remounted (at 654 * which point we'll call zfs_unlinked_drain() to process it). 655 */
Steps to reproduce:
zfs create tank/test1 zfs set quota=10M tank/test1 mkfile 10M /tank/test1/testfile less /tank/test1/testfile (to keep a filehandle open) rm /tank/test1/testfile zfs snapshot tank/test1@snap1 <exit less> zfs destroy tank/test1@snap1 Result: No free space in tank/test1.
The workaround is to remount the filesystem but that's not really possible for me on a customer machine.
One "fix" might be a cleanup thread for every zpool which calls zfs_unlinked_drain(). In this case, zfs_unlinked_drain() must skip znodes with z_sync_cnt > 0.