Bug #907

Recursive zfsdev_ioctl importing a pool with a zvol mirror vdev

Added by Albert Lee almost 10 years ago. Updated over 9 years ago.

Start date:
Due date:
% Done:


Estimated time:
Gerrit CR:


I attached a zvol as a temporary mirror for a root pool (don't ask why...).

zpool import from live media results in a consistent panic:

fec4a3b4 kmdb_enter+0xa()
fec4a3c4 debug_enter+0x29(fe92457c, c35a2388, c48214c0, fe891a29)
fec4a424 panicsys+0x477(fe923cbc, c35a2388, fec4a43c, 1)
c35a235c vpanic+0xc3(fe923cbc, c35a2388, c35a239c, f5fa1b89)
c35a237c panic+0x12(fe923cbc, fe923c04, c4b59878, c48214c0, c48214c0, 0)
c35a23ac mutex_panic+0x5e(fe923c04, c4b59878, fec29488, c35a2430)
c35a240c mutex_vector_enter+0x1db(c4b59878)
c35a245c zfs`spa_get_stats+0x13f(c6953000, c35a2480, c6953400, 800)
c35a248c zfs`zfs_ioc_pool_stats+0x21(c6953000, 1, 1, 80100000)
c35a24cc zfs`zfsdev_ioctl+0x15e()
c35a24fc cdev_ioctl+0x31(1dc0000, 5a05, c5a71000, 80100000, c26a0e68, c35a2590)
c35a254c ldi_ioctl+0x99(c4071a50, 5a05, c5a71000, 80000000, c26a0e68, c35a2590)
c35a25ac dev`devzvol_handle_ioctl+0xcf(5a05, c5a71000, 0, c40a3600)
c35a25cc dev`devzvol_objset_check+0xb9(c3f91ef0, c35a25f0, 3, febd55ba)
c35a260c dev`devzvol_lookup+0xce(c40a2180, c35a2704, c35a26ec, c35a28a4, 0, 
c35a266c fop_lookup+0xb0(c40a2180, c35a2704, c35a26ec, c35a28a4, 0, c28f2ac0)
c35a282c lookuppnvp+0x294(c35a28a4, 0, 1, 0, c35a2964, c28f2ac0)
c35a287c lookuppnatcred+0xea(c35a28a4, 0, 1, 0, c35a2964, c28f2ac0)
c35a28fc lookupnameatcred+0x47()
c35a292c lookupnameat+0x2c(c40719a9, 1, 1, 0, c35a2964, c28f2ac0)
c35a298c ldi_vp_from_name+0x70(c40719a8, c35a29b0, c35a29cc, fe997cae)
c35a29cc ldi_open_by_name+0x43(c40719a8, 1, c26a0e68, c3bc14a8, c24d6fa8, 0)
c35a2a4c zfs`vdev_disk_open+0x1b6(c398b7c0, c35a2a84, c35a2a8c, 0)
c35a2aac zfs`vdev_open+0xfc(c398b7c0, 4, 4, fea032b8)
c35a2aec zfs`vdev_open_children+0x7c(c46e0800, c398b7c0, c398bb20, 0)
c35a2b3c zfs`vdev_mirror_open+0x3c(c46e0800, c35a2b74, c35a2b7c, 0)
c35a2b9c zfs`vdev_open+0xfc(c46e0800, 4, c35a2c0c, f5f93bdb)
c35a2bdc zfs`vdev_open_children+0xb1(c3995140, c3d151a8, 2, c398b7c0)
c35a2c1c zfs`vdev_root_open+0x3c(c3995140, c35a2c54, c35a2c5c, f5fa1838)
c35a2c7c zfs`vdev_open+0xfc(c3995140, 7f, f60098ad)
c35a2cfc zfs`spa_load_impl+0x1ec(c6909340, bd2db59, 432e60cd, c69c4858, 3)
c35a2d4c zfs`spa_load+0x129(c6909340, 3, 0, 1)
c35a2d8c zfs`spa_tryimport+0x86(c69c4740)
c35a2dcc zfs`zfs_ioc_pool_tryimport+0x49(c6956000, c26a0288, 1590, 100003)
c35a2e0c zfs`zfsdev_ioctl+0x15e()
c35a2e3c cdev_ioctl+0x31(1dc0000, 5a06, 80420b0, 100003, c26a0288, c35a2ef8)
c35a2e6c specfs`spec_ioctl+0x52(c67d0a80, 5a06, 80420b0, 100003, c26a0288, 
c35a2ebc fop_ioctl+0x49(c67d0a80, 5a06, 80420b0, 100003, c26a0288, c35a2ef8)
c35a2f7c ioctl+0x171()
c35a2fa4 sys_sysenter+0x1a2()
[0]> c40719a8/S
0xc40719a8:     /dev/zvol/dsk/seapool/syspool-mirror

No pools have been imported at the time zpool import is run, so /dev/zvol/dsk/seapool/syspool-mirror does not exist, yet devzvol_lookup is called somehow.

spa_namespace_lock is the mutex being grabbed recursively, FWIW.


Updated by Albert Lee almost 10 years ago

By chance, Google turns up this gem for what appears to be an even less likely configuration:

ffffff000916af20 unix:mutex_panic+73 ()
ffffff000916af80 unix:mutex_vector_enter+190 ()
ffffff000916b000 zfs:spa_get_stats+14f ()
ffffff000916b040 zfs:zfs_ioc_pool_stats+32 ()
ffffff000916b0c0 zfs:zfsdev_ioctl+15e ()
ffffff000916b140 genunix:cdev_ioctl+6c ()
ffffff000916b190 genunix:ldi_ioctl+a4 ()
ffffff000916b1f0 dev:devzvol_handle_ioctl+c7 ()
ffffff000916b230 dev:devzvol_objset_check+b6 ()
ffffff000916b2b0 dev:devzvol_lookup+ac ()
ffffff000916b350 genunix:fop_lookup+ed ()
ffffff000916b590 genunix:lookuppnvp+28f ()
ffffff000916b630 genunix:lookuppnatcred+11b ()
ffffff000916b720 genunix:lookupnameatcred+97 ()
ffffff000916b7b0 genunix:lookupnameat+69 ()
ffffff000916b810 genunix:ldi_vp_from_name+8d ()
ffffff000916b880 genunix:ldi_open_by_name+57 ()
ffffff000916b930 zfs:vdev_disk_open+1c7 ()
ffffff000916b980 zfs:vdev_open+86 ()
ffffff000916ba00 zfs:spa_load_l2cache+1bb ()
ffffff000916bae0 zfs:spa_load_impl+791 ()
ffffff000916bb70 zfs:spa_load+123 ()
ffffff000916bbc0 zfs:spa_tryimport+97 ()
ffffff000916bc00 zfs:zfs_ioc_pool_tryimport+45 ()
ffffff000916bc80 zfs:zfsdev_ioctl+15e ()
ffffff000916bd00 genunix:cdev_ioctl+6c ()
ffffff000916bd40 specfs:spec_ioctl+5a ()
ffffff000916bdc0 genunix:fop_ioctl+7b ()
ffffff000916bec0 genunix:ioctl+18e ()
ffffff000916bf10 unix:brand_sys_sysenter+1c9 ()


Updated by Albert Lee almost 10 years ago

Albert Lee wrote:

No pools have been imported at the time zpool import is run, so /dev/zvol/dsk/seapool/syspool-mirror does not exist, yet devzvol_lookup is called somehow.

Nevermind, devzvol_lookup is entered via /dev/zvol/dsk, if you look at its vnode argument.


Updated by Albert Lee over 9 years ago

  • Difficulty set to Medium
  • Tags set to needs-triage

vdev_open_children has a comment apparently stating this is intentional:

      * in order to handle pools on top of zvols, do the opens
      * in a single thread so that the same thread holds the
      * spa_namespace_lock

Also available in: Atom PDF