Bug #907


Recursive zfsdev_ioctl importing a pool with a zvol mirror vdev

Added by Albert Lee about 11 years ago. Updated over 10 years ago.

Start date:
Due date:
% Done:


Estimated time:
Gerrit CR:


I attached a zvol as a temporary mirror for a root pool (don't ask why...).

zpool import from live media results in a consistent panic:

fec4a3b4 kmdb_enter+0xa()
fec4a3c4 debug_enter+0x29(fe92457c, c35a2388, c48214c0, fe891a29)
fec4a424 panicsys+0x477(fe923cbc, c35a2388, fec4a43c, 1)
c35a235c vpanic+0xc3(fe923cbc, c35a2388, c35a239c, f5fa1b89)
c35a237c panic+0x12(fe923cbc, fe923c04, c4b59878, c48214c0, c48214c0, 0)
c35a23ac mutex_panic+0x5e(fe923c04, c4b59878, fec29488, c35a2430)
c35a240c mutex_vector_enter+0x1db(c4b59878)
c35a245c zfs`spa_get_stats+0x13f(c6953000, c35a2480, c6953400, 800)
c35a248c zfs`zfs_ioc_pool_stats+0x21(c6953000, 1, 1, 80100000)
c35a24cc zfs`zfsdev_ioctl+0x15e()
c35a24fc cdev_ioctl+0x31(1dc0000, 5a05, c5a71000, 80100000, c26a0e68, c35a2590)
c35a254c ldi_ioctl+0x99(c4071a50, 5a05, c5a71000, 80000000, c26a0e68, c35a2590)
c35a25ac dev`devzvol_handle_ioctl+0xcf(5a05, c5a71000, 0, c40a3600)
c35a25cc dev`devzvol_objset_check+0xb9(c3f91ef0, c35a25f0, 3, febd55ba)
c35a260c dev`devzvol_lookup+0xce(c40a2180, c35a2704, c35a26ec, c35a28a4, 0, 
c35a266c fop_lookup+0xb0(c40a2180, c35a2704, c35a26ec, c35a28a4, 0, c28f2ac0)
c35a282c lookuppnvp+0x294(c35a28a4, 0, 1, 0, c35a2964, c28f2ac0)
c35a287c lookuppnatcred+0xea(c35a28a4, 0, 1, 0, c35a2964, c28f2ac0)
c35a28fc lookupnameatcred+0x47()
c35a292c lookupnameat+0x2c(c40719a9, 1, 1, 0, c35a2964, c28f2ac0)
c35a298c ldi_vp_from_name+0x70(c40719a8, c35a29b0, c35a29cc, fe997cae)
c35a29cc ldi_open_by_name+0x43(c40719a8, 1, c26a0e68, c3bc14a8, c24d6fa8, 0)
c35a2a4c zfs`vdev_disk_open+0x1b6(c398b7c0, c35a2a84, c35a2a8c, 0)
c35a2aac zfs`vdev_open+0xfc(c398b7c0, 4, 4, fea032b8)
c35a2aec zfs`vdev_open_children+0x7c(c46e0800, c398b7c0, c398bb20, 0)
c35a2b3c zfs`vdev_mirror_open+0x3c(c46e0800, c35a2b74, c35a2b7c, 0)
c35a2b9c zfs`vdev_open+0xfc(c46e0800, 4, c35a2c0c, f5f93bdb)
c35a2bdc zfs`vdev_open_children+0xb1(c3995140, c3d151a8, 2, c398b7c0)
c35a2c1c zfs`vdev_root_open+0x3c(c3995140, c35a2c54, c35a2c5c, f5fa1838)
c35a2c7c zfs`vdev_open+0xfc(c3995140, 7f, f60098ad)
c35a2cfc zfs`spa_load_impl+0x1ec(c6909340, bd2db59, 432e60cd, c69c4858, 3)
c35a2d4c zfs`spa_load+0x129(c6909340, 3, 0, 1)
c35a2d8c zfs`spa_tryimport+0x86(c69c4740)
c35a2dcc zfs`zfs_ioc_pool_tryimport+0x49(c6956000, c26a0288, 1590, 100003)
c35a2e0c zfs`zfsdev_ioctl+0x15e()
c35a2e3c cdev_ioctl+0x31(1dc0000, 5a06, 80420b0, 100003, c26a0288, c35a2ef8)
c35a2e6c specfs`spec_ioctl+0x52(c67d0a80, 5a06, 80420b0, 100003, c26a0288, 
c35a2ebc fop_ioctl+0x49(c67d0a80, 5a06, 80420b0, 100003, c26a0288, c35a2ef8)
c35a2f7c ioctl+0x171()
c35a2fa4 sys_sysenter+0x1a2()
[0]> c40719a8/S
0xc40719a8:     /dev/zvol/dsk/seapool/syspool-mirror

No pools have been imported at the time zpool import is run, so /dev/zvol/dsk/seapool/syspool-mirror does not exist, yet devzvol_lookup is called somehow.

spa_namespace_lock is the mutex being grabbed recursively, FWIW.

Actions #1

Updated by Albert Lee about 11 years ago

By chance, Google turns up this gem for what appears to be an even less likely configuration:

ffffff000916af20 unix:mutex_panic+73 ()
ffffff000916af80 unix:mutex_vector_enter+190 ()
ffffff000916b000 zfs:spa_get_stats+14f ()
ffffff000916b040 zfs:zfs_ioc_pool_stats+32 ()
ffffff000916b0c0 zfs:zfsdev_ioctl+15e ()
ffffff000916b140 genunix:cdev_ioctl+6c ()
ffffff000916b190 genunix:ldi_ioctl+a4 ()
ffffff000916b1f0 dev:devzvol_handle_ioctl+c7 ()
ffffff000916b230 dev:devzvol_objset_check+b6 ()
ffffff000916b2b0 dev:devzvol_lookup+ac ()
ffffff000916b350 genunix:fop_lookup+ed ()
ffffff000916b590 genunix:lookuppnvp+28f ()
ffffff000916b630 genunix:lookuppnatcred+11b ()
ffffff000916b720 genunix:lookupnameatcred+97 ()
ffffff000916b7b0 genunix:lookupnameat+69 ()
ffffff000916b810 genunix:ldi_vp_from_name+8d ()
ffffff000916b880 genunix:ldi_open_by_name+57 ()
ffffff000916b930 zfs:vdev_disk_open+1c7 ()
ffffff000916b980 zfs:vdev_open+86 ()
ffffff000916ba00 zfs:spa_load_l2cache+1bb ()
ffffff000916bae0 zfs:spa_load_impl+791 ()
ffffff000916bb70 zfs:spa_load+123 ()
ffffff000916bbc0 zfs:spa_tryimport+97 ()
ffffff000916bc00 zfs:zfs_ioc_pool_tryimport+45 ()
ffffff000916bc80 zfs:zfsdev_ioctl+15e ()
ffffff000916bd00 genunix:cdev_ioctl+6c ()
ffffff000916bd40 specfs:spec_ioctl+5a ()
ffffff000916bdc0 genunix:fop_ioctl+7b ()
ffffff000916bec0 genunix:ioctl+18e ()
ffffff000916bf10 unix:brand_sys_sysenter+1c9 ()

Actions #2

Updated by Albert Lee about 11 years ago

Albert Lee wrote:

No pools have been imported at the time zpool import is run, so /dev/zvol/dsk/seapool/syspool-mirror does not exist, yet devzvol_lookup is called somehow.

Nevermind, devzvol_lookup is entered via /dev/zvol/dsk, if you look at its vnode argument.

Actions #3

Updated by Albert Lee over 10 years ago

  • Difficulty set to Medium
  • Tags set to needs-triage

vdev_open_children has a comment apparently stating this is intentional:

      * in order to handle pools on top of zvols, do the opens
      * in a single thread so that the same thread holds the
      * spa_namespace_lock


Also available in: Atom PDF