illumos gate

This is triggered by the RAID, Inc. 96-bay JBOD.

# /usr/lib/fm/fmd/fmd -o fg=true -o client.debug=true
fmd: [ loading modules ... ABORT: attempted zero-length allocation: No such device or address
Abort (core dumped)

# mdb core
Loading modules: [ fmd libumem.so.1 libc.so.1 libnvpair.so.1 libtopo.so.1 libuutil.so.1 libavl.so.1 libcmdutils.so.1 libsysevent.so.1 ld.so.1 ]
> $C
08046298 libc.so.1`_lwp_kill+0x15(1, 6, 80462e8, fef42000, fef42000, 8046320)
080462b8 libc.so.1`raise+0x2b(6, 0, 80462d0, feec1b59, 0, 0)
08046308 libc.so.1`abort+0x10e(fede2a40, fef44cb8, 8046348, 6, 524f4241, 61203a54)
08046738 libses.so.1`ses_panic(fdde6758, 8046764, 80467d8, fdb6b67a, 83f1048, fdb6c398)
08046758 libses.so.1`ses_realloc(fdde6758, 0, 83f5078, fdde6130, fddf7000, fdb6658f)
08046778 libses.so.1`ses_alloc+0x27(0, feb80000, 6, 10, ee0, 80f4627)
080467a8 libses.so.1`ses_zalloc+0x1e(0, 0, 73, fdb6659d, 83f5050, 8)
08046828 ses2.so`elem_parse_aes_misc+0x91(80f44f4, 83f1048, 8, fdb65d85)
08046878 ses2.so`elem_parse_aes+0xfc(82bd388, 83f5148, 80468e8, fdb80eae)
08046898 ses2.so`ses2_fill_element_node+0x37(82bd388, 83f5148, 8303ed8, 4)
080468c8 ses2.so`ses2_node_parse+0x53(82bd388, 83f5148, e, fddf7000)
080468e8 libses.so.1`ses_fill_node+0x22(83f5148, 83f5208, fdde38ae, fdde394c)
08046908 libses.so.1`ses_fill_tree+0x21(83f5148, 82bd548, 83e9cc8, fdde394c)
08046928 libses.so.1`ses_fill_tree+0x33(82bd648, 82bd448, 8046958, fdde394c)
08046948 libses.so.1`ses_fill_tree+0x33(82bd548, 82a5270, 8046988, fdde394c)
08046968 libses.so.1`ses_fill_tree+0x33(82bd448, 0, 18, fddf7000)
08046988 libses.so.1`ses_fill_snap+0x22(82c08d0, 80, 0, fdde56eb)
080469d8 libses.so.1`ses_snap_new+0x325(82bd408, 0, 8046a08, fdde3006)
08046a08 libses.so.1`ses_open_scsi+0xc4(1, 82a51a0, 8046a90, fed71c1b, 80e9468, fede4042)
08046a58 libses.so.1`ses_open+0x98(1, 8046a90, 0, feecedd3, 43, fde1fc58)
08046ea8 ses.so`ses_process_dir+0x133(fde20159, 83d8ed8, 0, fed77e40)
08046ed8 ses.so`ses_enum+0xc1(80e9468, 83aeb58, 8356570, 0, 400, 0)
08046f28 libtopo.so.1`topo_mod_enumerate+0xc4(80e9468, 83aeb58, 82d4a88, 8356570, 0, 400)
08046f78 libtopo.so.1`enum_run+0xe9(80e9a18, 83d77c8, a, fed7b1dd)
08046fc8 libtopo.so.1`topo_xml_range_process+0x13e(80e9a18, 82bb0b0, 83d77c8, 8046ff8)
08047018 libtopo.so.1`tf_rdata_new+0x135(80e9a18, 81c8790, 82bb0b0, 83aeb58)
08047078 libtopo.so.1`topo_xml_walk+0x246(80e9a18, 81c8790, 82bb830, 83aeb58, 80e9a18, 83d5bc0)
080470d8 libtopo.so.1`topo_xml_walk+0x1b2(80e9a18, 81c8790, 82b0b28, 83aeb58)
08047118 libtopo.so.1`dependent_create+0x127(80e9a18, 81c8790, 83d6ab0, 82b0b28, 83aeb58, fed7b1f9)
08047158 libtopo.so.1`dependents_create+0x64(80e9a18, 81c8790, 83d6ab0, 82b0da8, 83aeb58, 81bd0d8)
08047208 libtopo.so.1`pad_process+0x51e(80e9a18, 83d79a8, 82b0da8, 83aeb58, 83d79d0, 8356340)
08047268 libtopo.so.1`topo_xml_range_process+0x31f(80e9a18, 82b0da8, 83d79a8, 8047298)
080472b8 libtopo.so.1`tf_rdata_new+0x135(80e9a18, 81c8790, 82b0da8, 81bd258)
08047318 libtopo.so.1`topo_xml_walk+0x246(80e9a18, 81c8790, 82a37a0, 81bd258, 80e5f40, fed8c000)
08047348 libtopo.so.1`topo_xml_enum+0x67(80e9a18, 81c8790, 81bd258, feac2000)
08047478 libtopo.so.1`topo_file_load+0x139(80e9a18, 81bd258, fe20c127, fe20bda2, 0, 82a6000)
080474a8 libtopo.so.1`topo_mod_enummap+0x26(80e9a18, 81bd258, fe20c127, fe20bda2, 80e9a18, fe20b11c)
080474f8 x86pi.so`x86pi_enum_start+0xc5(80e9a18, 8047520, 8047528, fe205580, 80e9a18, 80e9a18)
08047548 x86pi.so`x86pi_enum+0x55(80e9a18, 81bd258, 81a6a70, 0, 0, 0)
08047598 libtopo.so.1`topo_mod_enumerate+0xc4(80e9a18, 81bd258, 80cdf38, 81a6a70, 0, 0)
080475e8 libtopo.so.1`enum_run+0xe9(80e9b68, 82a5fa8, a, fed7b1dd)
08047638 libtopo.so.1`topo_xml_range_process+0x13e(80e9b68, 82a3f70, 82a5fa8, 8047668)
08047688 libtopo.so.1`tf_rdata_new+0x135(80e9b68, 81c8bd0, 82a3f70, 81bd258)
080476e8 libtopo.so.1`topo_xml_walk+0x246(80e9b68, 81c8bd0, 81c7108, 81bd258, 80e5f40, fed8c000)
08047718 libtopo.so.1`topo_xml_enum+0x67(80e9b68, 81c8bd0, 81bd258, 81a6ab0)
08047848 libtopo.so.1`topo_file_load+0x139(80e9b68, 81bd258, 80d4f38, 81a6a80, 0, 2c)
08047888 libtopo.so.1`topo_tree_enum+0x89(80e5f40, 81c5318, 80478b8, fe70e6f8, 81b5310, 80e5f40)
080478a8 libtopo.so.1`topo_tree_enum_all+0x20(80e5f40, 81b5310, 80478e8, fed71087)
080478e8 libtopo.so.1`topo_snap_create+0x13d(80e5f40, 804793c, 0, fed7118d, 807c010, 21)
08047918 libtopo.so.1`topo_snap_hold+0x56(80e5f40, 0, 804793c, 80c9f08, 0, 8047ab8)
08047958 fmd_topo_update+0x9f(80c9f08, 8085dfa, 8047a58, 80601f7, 0, 0)
08047968 fmd_topo_init+0xb(0, 0, 0, 0, 2, 80992f8)
08047a58 fmd_run+0x118(809a8c0, ffffffff, 0, 0)
08047ad8 main+0x344(8047acc, fef4f348, 8047b0c, 805fdd3, 5, 8047b18)
08047b0c _start+0x83(5, 8047c2c, 8047c40, 8047c43, 8047c4b, 8047c4e)

Disabling all modules will not allow FMD to continue running. It will still die as soon as the JBOD is attached.

Core dump is available at:

ftp://ftp.nrg.wustl.edu/pub/zfs/fmd.core