Bug #3541


vdev_open returns EINVAL after upgrade to oi_151a

Added by Piotr Jasiukajtis over 8 years ago. Updated about 8 years ago.

zfs - Zettabyte File System
Start date:
Due date:
% Done:


Estimated time:
Gerrit CR:


After upgrade from OpenSolaris to OpenIndiana (oi_b151a) I get a panic on HP blade.
It could be related to Cpqary3 driver, but I use 2.4.6 version so I think it should work fine.

You can see a notice on a screenshot: zfs_parse_bootfs: error 22 so I checked where we receive this error.
I started at zfs_parse_bootfs and goes down to:

In my case vdev_open returns EINVAL (0x16).
This is not finished yet but I filed a bug for a reference.


zfs_parse_bootfs_args.png (116 KB) zfs_parse_bootfs_args.png Piotr Jasiukajtis, 2013-02-08 10:07 AM
panic.png (40.1 KB) panic.png Piotr Jasiukajtis, 2013-02-08 10:07 AM
vdev_open_-_stack.png (107 KB) vdev_open_-_stack.png Piotr Jasiukajtis, 2013-02-08 10:38 AM
end_of_vdev_open.png (93.6 KB) end_of_vdev_open.png Piotr Jasiukajtis, 2013-02-08 10:57 AM
vdev_open_child_-_returned_EINVAL_from_vdev_open.png (103 KB) vdev_open_child_-_returned_EINVAL_from_vdev_open.png Piotr Jasiukajtis, 2013-02-08 02:19 PM
vdev_set_state_-_regs.png (88.8 KB) vdev_set_state_-_regs.png Piotr Jasiukajtis, 2013-02-08 09:14 PM
vdev_set_state_-_stack.png (41.2 KB) vdev_set_state_-_stack.png Piotr Jasiukajtis, 2013-02-08 09:14 PM

Related issues

Related to illumos gate - Bug #2671: zpool import should not fail if vdev ashift has increasedResolvedGeorge Wilson2012-05-02

Actions #2

Updated by Piotr Jasiukajtis over 8 years ago

Just for reference, vdev_open from oi_151a:

> vdev_open::dis
vdev_open:                      pushq  %rbp
vdev_open+1:                    movq   %rsp,%rbp
vdev_open+4:                    subq   $0x8,%rsp
vdev_open+8:                    movq   %rdi,-0x8(%rbp)
vdev_open+0xc:                  pushq  %rbx
vdev_open+0xd:                  pushq  %r12
vdev_open+0xf:                  pushq  %r13
vdev_open+0x11:                 pushq  %r14
vdev_open+0x13:                 subq   $0x18,%rsp
vdev_open+0x17:                 movq   %rdi,%r13
vdev_open+0x1a:                 movq   0x50(%r13),%r12
vdev_open+0x1e:                 xorq   %r14,%r14
vdev_open+0x21:                 movq   %r14,-0x40(%rbp)
vdev_open+0x25:                 movq   %r14,-0x38(%rbp)
vdev_open+0x29:                 movq   %r14,0x240(%r13)
vdev_open+0x30:                 movb   $0x0,0x4de(%r13)
vdev_open+0x38:                 movb   $0x0,0x4df(%r13)
vdev_open+0x40:                 movq   %r13,%rdi
vdev_open+0x43:                 call   -0x2108  <vdev_get_min_asize>
vdev_open+0x48:                 movq   %rax,0x28(%r13)
vdev_open+0x4c:                 cmpq   $0x0,0x478(%r13)
vdev_open+0x54:                 jne    +0xe     <vdev_open+0x64>
vdev_open+0x56:                 cmpq   $0x0,0x468(%r13)
vdev_open+0x5e:                 jne    +0x344   <vdev_open+0x3a8>
vdev_open+0x64:                 cmpq   $0x0,0x460(%r13)
vdev_open+0x6c:                 jne    +0x31b   <vdev_open+0x38d>
vdev_open+0x72:                 movq   0x48(%r13),%r8
vdev_open+0x76:                 leaq   -0x40(%rbp),%rsi
vdev_open+0x7a:                 leaq   -0x38(%rbp),%rdx
vdev_open+0x7e:                 movq   %r13,%rdi
vdev_open+0x81:                 xorl   %eax,%eax
vdev_open+0x83:                 call   *(%r8)
vdev_open+0x86:                 movl   %eax,%ebx
vdev_open+0x88:                 xorl   %eax,%eax
vdev_open+0x8a:                 movl   %eax,0x2fc(%r13)
vdev_open+0x91:                 cmpl   %eax,+0x43305e1(%rip)    <zio_injection_enabled>
vdev_open+0x97:                 je     +0x1a    <vdev_open+0xb3>
vdev_open+0x99:                 testl  %ebx,%ebx
vdev_open+0x9b:                 jne    +0x2b7   <vdev_open+0x358>
vdev_open+0xa1:                 movq   %r13,%rdi
vdev_open+0xa4:                 xorq   %rsi,%rsi
vdev_open+0xa7:                 movl   $0x6,%edx
vdev_open+0xac:                 call   +0x272ef <zio_handle_device_injection>
vdev_open+0xb1:                 movl   %eax,%ebx
vdev_open+0xb3:                 testl  %ebx,%ebx
vdev_open+0xb5:                 jne    +0x29d   <vdev_open+0x358>
vdev_open+0xbb:                 movq   %r14,0x478(%r13)
vdev_open+0xc2:                 cmpq   $0x0,0x468(%r13)
vdev_open+0xca:                 jne    +0x268   <vdev_open+0x338>
vdev_open+0xd0:                 cmpq   $0x0,0x470(%r13)
vdev_open+0xd8:                 jne    +0x16    <vdev_open+0xf0>
vdev_open+0xda:                 movq   %r13,%rdi
vdev_open+0xdd:                 movl   $0x1,%esi
vdev_open+0xe2:                 movl   $0x7,%edx
vdev_open+0xe7:                 xorl   %ecx,%ecx
vdev_open+0xe9:                 call   +0x2da2  <vdev_set_state>
vdev_open+0xee:                 jmp    +0x17    <vdev_open+0x107>
vdev_open+0xf0:                 movq   %r13,%rdi
vdev_open+0xf3:                 movl   $0x1,%esi
vdev_open+0xf8:                 movl   $0x6,%edx
vdev_open+0xfd:                 movl   $0xa,%ecx
vdev_open+0x102:                call   +0x2d89  <vdev_set_state>
vdev_open+0x107:                cmpq   $0x0,0x408(%r13)
vdev_open+0x10f:                jne    +0x21c   <vdev_open+0x331>
vdev_open+0x115:                leaq   +0x431a48c(%rip),%r8     <vdev_missing_ops>
vdev_open+0x11c:                cmpq   0x48(%r13),%r8
vdev_open+0x120:                je     +0x20b   <vdev_open+0x331>
vdev_open+0x126:                movq   0x88(%r13),%r10
vdev_open+0x12d:                testq  %r10,%r10
vdev_open+0x130:                je     +0x43    <vdev_open+0x175>
vdev_open+0x132:                xorq   %r8,%r8
vdev_open+0x135:                xorq   %r9,%r9
vdev_open+0x138:                movq   0x80(%r13),%r11
vdev_open+0x13f:                prefetcht0 0x100(%r11,%r9)
vdev_open+0x148:                movq   (%r11,%r9),%rax
vdev_open+0x14c:                cmpq   $0x7,0x38(%rax)
vdev_open+0x151:                jne    +0xe     <vdev_open+0x161>
vdev_open+0x153:                incq   %r8
vdev_open+0x156:                addq   $0x8,%r9
vdev_open+0x15a:                cmpq   %r10,%r8
vdev_open+0x15d:                jb     -0x20    <vdev_open+0x13f>
vdev_open+0x15f:                jmp    +0x14    <vdev_open+0x175>
vdev_open+0x161:                movq   %r13,%rdi
vdev_open+0x164:                movl   $0x1,%esi
vdev_open+0x169:                movl   $0x6,%edx
vdev_open+0x16e:                xorl   %ecx,%ecx
vdev_open+0x170:                call   +0x2d1b  <vdev_set_state>
vdev_open+0x175:                movq   -0x40(%rbp),%r8
vdev_open+0x179:                andq   $0xfffffffffffc0000,%r8
vdev_open+0x180:                movq   %r8,-0x40(%rbp)
vdev_open+0x184:                cmpq   $0x0,0x88(%r13)
vdev_open+0x18c:                je     +0x36    <vdev_open+0x1c4>
vdev_open+0x18e:                cmpq   $0x0,0x78(%r13)
vdev_open+0x193:                je     +0x2a    <vdev_open+0x1bf>
vdev_open+0x195:                cmpq   $0x3b80000,%r8
vdev_open+0x19c:                jae    +0x21    <vdev_open+0x1bf>
vdev_open+0x19e:                movq   %r13,%rdi
vdev_open+0x1a1:                movl   $0x1,%esi
vdev_open+0x1a6:                movl   $0x4,%edx
vdev_open+0x1ab:                movl   $0x5,%ecx
vdev_open+0x1b0:                call   +0x2cdb  <vdev_set_state>
vdev_open+0x1b5:                movl   $0x4f,%ebx
vdev_open+0x1ba:                jmp    +0x207   <vdev_open+0x3c6>
vdev_open+0x1bf:                xorq   %r9,%r9
vdev_open+0x1c2:                jmp    +0x1a    <vdev_open+0x1de>
vdev_open+0x1c4:                cmpq   $0x4000000,%r8
vdev_open+0x1cb:                jb     +0x13f   <vdev_open+0x310>
vdev_open+0x1d1:                leaq   0xffffffffffb80000(%r8),%r10
vdev_open+0x1d8:                movq   %r8,%r9
vdev_open+0x1db:                movq   %r10,%r8
vdev_open+0x1de:                movq   %r9,0x410(%r13)
vdev_open+0x1e5:                cmpq   0x28(%r13),%r8
vdev_open+0x1e9:                jb     +0x100   <vdev_open+0x2ef>
vdev_open+0x1ef:                movq   0x20(%r13),%r10
vdev_open+0x1f3:                testq  %r10,%r10
vdev_open+0x1f6:                jne    +0x1c    <vdev_open+0x214>
vdev_open+0x1f8:                movq   %r8,0x20(%r13)
vdev_open+0x1fc:                movq   -0x38(%rbp),%r9
vdev_open+0x200:                movq   0x30(%r13),%r10
vdev_open+0x204:                cmpq   %r10,%r9
vdev_open+0x207:                cmovq.b %r10,%r9
vdev_open+0x20b:                movq   %r8,%r10
vdev_open+0x20e:                movq   %r9,0x30(%r13)
vdev_open+0x212:                jmp    +0x12    <vdev_open+0x226>
vdev_open+0x214:                movq   -0x38(%rbp),%r9
vdev_open+0x218:                movq   0x70(%r13),%r11
vdev_open+0x21c:                cmpq   0x30(%r11),%r9
vdev_open+0x220:                ja     +0xa8    <vdev_open+0x2ce>
vdev_open+0x226:                movq   %r13,%rdi
vdev_open+0x229:                cmpq   $0x7,0x38(%rdi)
vdev_open+0x22e:                jne    +0x1d    <vdev_open+0x24d>
vdev_open+0x230:                cmpq   %r8,%r10
vdev_open+0x233:                jae    +0x18    <vdev_open+0x24d>
vdev_open+0x235:                cmpl   $0x0,0x2f8(%rdi)
vdev_open+0x23c:                jne    +0xb     <vdev_open+0x249>
vdev_open+0x23e:                cmpq   $0x0,0x750(%r12)
vdev_open+0x247:                je     +0x4     <vdev_open+0x24d>
vdev_open+0x249:                movq   %r8,0x20(%rdi)
vdev_open+0x24d:                call   -0x22a2  <vdev_set_min_asize>
vdev_open+0x252:                movq   0x48(%r13),%r8
vdev_open+0x256:                cmpl   $0x0,0x50(%r8)
vdev_open+0x25b:                je     +0x4e    <vdev_open+0x2ab>
vdev_open+0x25d:                movq   %r13,%rdi
vdev_open+0x260:                xorq   %rsi,%rsi
vdev_open+0x263:                call   -0x838   <vdev_probe>
vdev_open+0x268:                movq   %rax,%rdi
vdev_open+0x26b:                call   +0x234f0 <zio_wait>
vdev_open+0x270:                movl   %eax,%ebx
vdev_open+0x272:                testl  %ebx,%ebx
vdev_open+0x274:                jne    +0x3c    <vdev_open+0x2b2>
vdev_open+0x276:                movq   0x48(%r13),%r8
vdev_open+0x27a:                cmpl   $0x0,0x50(%r8)
vdev_open+0x27f:                je     +0x2a    <vdev_open+0x2ab>
vdev_open+0x281:                cmpb   $0x0,0x5ce(%r12)
vdev_open+0x28a:                jne    +0x1f    <vdev_open+0x2ab>
vdev_open+0x28c:                movq   %r13,%rdi
vdev_open+0x28f:                xorq   %rsi,%rsi
vdev_open+0x292:                xorq   %rdx,%rdx
vdev_open+0x295:                call   +0x10a6  <vdev_resilver_needed>
vdev_open+0x29a:                testl  %eax,%eax
vdev_open+0x29c:                je     +0xd     <vdev_open+0x2ab>
vdev_open+0x29e:                movq   %r12,%rdi
vdev_open+0x2a1:                movl   $0x10,%esi
vdev_open+0x2a6:                call   -0xc0eb  <spa_async_request>
vdev_open+0x2ab:                xorl   %ebx,%ebx
vdev_open+0x2ad:                jmp    +0x114   <vdev_open+0x3c6>
vdev_open+0x2b2:                movq   %r13,%rdi
vdev_open+0x2b5:                movl   $0x1,%esi
vdev_open+0x2ba:                movl   $0x5,%edx
vdev_open+0x2bf:                movl   $0xa,%ecx
vdev_open+0x2c4:                call   +0x2bc7  <vdev_set_state>
vdev_open+0x2c9:                jmp    +0xf8    <vdev_open+0x3c6>
vdev_open+0x2ce:                movq   %r13,%rdi
vdev_open+0x2d1:                movl   $0x1,%esi
vdev_open+0x2d6:                movl   $0x4,%edx
vdev_open+0x2db:                movl   $0x6,%ecx
vdev_open+0x2e0:                call   +0x2bab  <vdev_set_state>
vdev_open+0x2e5:                movl   $0x16,%ebx
vdev_open+0x2ea:                jmp    +0xd7    <vdev_open+0x3c6>
vdev_open+0x2ef:                movq   %r13,%rdi
vdev_open+0x2f2:                movl   $0x1,%esi
vdev_open+0x2f7:                movl   $0x4,%edx
vdev_open+0x2fc:                movl   $0x6,%ecx
vdev_open+0x301:                call   +0x2b8a  <vdev_set_state>
vdev_open+0x306:                movl   $0x16,%ebx
vdev_open+0x30b:                jmp    +0xb6    <vdev_open+0x3c6>
vdev_open+0x310:                movq   %r13,%rdi
vdev_open+0x313:                movl   $0x1,%esi
vdev_open+0x318:                movl   $0x4,%edx
vdev_open+0x31d:                movl   $0x5,%ecx
vdev_open+0x322:                call   +0x2b69  <vdev_set_state>
vdev_open+0x327:                movl   $0x4f,%ebx
vdev_open+0x32c:                jmp    +0x95    <vdev_open+0x3c6>
vdev_open+0x331:                xorl   %ebx,%ebx
vdev_open+0x333:                jmp    +0x8e    <vdev_open+0x3c6>
vdev_open+0x338:                movl   0x600(%r13),%ecx
vdev_open+0x33f:                movq   %r13,%rdi
vdev_open+0x342:                movl   $0x1,%esi
vdev_open+0x347:                movl   $0x5,%edx
vdev_open+0x34c:                call   +0x2b3f  <vdev_set_state>
vdev_open+0x351:                movl   $0x6,%ebx
vdev_open+0x356:                jmp    +0x6e    <vdev_open+0x3c6>
vdev_open+0x358:                movq   %r13,%rdi
vdev_open+0x35b:                movq   0x240(%rdi),%r8
vdev_open+0x362:                cmpq   $0x0,0x478(%rdi)
vdev_open+0x36a:                je     +0xd     <vdev_open+0x379>
vdev_open+0x36c:                cmpq   $0x1,%r8
vdev_open+0x370:                je     +0x7     <vdev_open+0x379>
vdev_open+0x372:                movq   %r14,0x478(%rdi)
vdev_open+0x379:                movslq %r8d,%rcx
vdev_open+0x37c:                movl   $0x1,%esi
vdev_open+0x381:                movl   $0x4,%edx
vdev_open+0x386:                call   +0x2b05  <vdev_set_state>
vdev_open+0x38b:                jmp    +0x39    <vdev_open+0x3c6>
vdev_open+0x38d:                movq   %r13,%rdi
vdev_open+0x390:                movl   $0x1,%esi
vdev_open+0x395:                movl   $0x2,%edx
vdev_open+0x39a:                xorl   %ecx,%ecx
vdev_open+0x39c:                call   +0x2aef  <vdev_set_state>
vdev_open+0x3a1:                movl   $0x6,%ebx
vdev_open+0x3a6:                jmp    +0x1e    <vdev_open+0x3c6>
vdev_open+0x3a8:                movl   0x600(%r13),%ecx
vdev_open+0x3af:                movq   %r13,%rdi
vdev_open+0x3b2:                movl   $0x1,%esi
vdev_open+0x3b7:                movl   $0x5,%edx
vdev_open+0x3bc:                call   +0x2acf  <vdev_set_state>
vdev_open+0x3c1:                movl   $0x6,%ebx
vdev_open+0x3c6:                movl   %ebx,%eax
vdev_open+0x3c8:                addq   $0x18,%rsp
vdev_open+0x3cc:                popq   %r14
vdev_open+0x3ce:                popq   %r13
vdev_open+0x3d0:                popq   %r12
vdev_open+0x3d2:                popq   %rbx
vdev_open+0x3d3:                leave  
vdev_open+0x3d4:                ret    

Actions #3

Updated by Piotr Jasiukajtis over 8 years ago

> vdev_open_child::dis
vdev_open_child:                pushq  %rbp
vdev_open_child+1:              movq   %rsp,%rbp
vdev_open_child+4:              subq   $0x8,%rsp
vdev_open_child+8:              movq   %rdi,-0x8(%rbp)
vdev_open_child+0xc:            pushq  %r12
vdev_open_child+0xe:            movq   %rdi,%r12
vdev_open_child+0x11:           movq   %gs:0x18,%rax
vdev_open_child+0x1a:           movq   %rax,0x308(%r12)
vdev_open_child+0x22:           movq   %r12,%rdi
vdev_open_child+0x25:           call   +0x2fe   <vdev_open> ####### returned EINVAL 
vdev_open_child+0x2a:           movl   %eax,0x300(%r12)
vdev_open_child+0x32:           xorq   %r8,%r8
vdev_open_child+0x35:           movq   %r8,0x308(%r12)
vdev_open_child+0x3d:           popq   %r12
vdev_open_child+0x3f:           leave  
vdev_open_child+0x40:           ret    
Actions #4

Updated by Piotr Jasiukajtis over 8 years ago

1146    /*
1147     * Reset the vdev_reopening flag so that we actually close
1148     * the vdev on error.
1149     */
1150    vd->vdev_reopening = B_FALSE;
1151    if (zio_injection_enabled && error == 0)
1152        error = zio_handle_device_injection(vd, NULL, ENXIO);
1154    if (error) {
1155        if (vd->vdev_removed &&
1156            vd->vdev_stat.vs_aux != VDEV_AUX_OPEN_FAILED)
1157            vd->vdev_removed = B_FALSE;
1159        vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN, <-------- 
1160            vd->vdev_stat.vs_aux);
1161        return (error);
1162    }

Arguments on the screenshot:
rdi = vd
rsi = B_TRUE

Actions #5

Updated by Piotr Jasiukajtis over 8 years ago

rcx register is loaded with 6, so it's actually VDEV_AUX_BAD_LABEL:

1228    /*
1229     * Make sure the allocatable size hasn't shrunk.
1230     */
1231    if (asize < vd->vdev_min_asize) {
1232        vdev_set_state(vd, B_TRUE, VDEV_STATE_CANT_OPEN,
1233            VDEV_AUX_BAD_LABEL);
1234        return (EINVAL);
1235    }

Actions #6

Updated by Albert Lee over 8 years ago

This related to, but not the same as, #2671 which relates to ashift shrinking.

Actions #7

Updated by Piotr Jasiukajtis about 8 years ago

Albert Lee wrote:

This related to, but not the same as, #2671 which relates to ashift shrinking.

Corresponding changeset:


Also available in: Atom PDF