Project

General

Profile

Actions

Feature #14471

closed

vioblk could raise dynamic lun expansion sysevents

Added by Andy Fiddaman 6 months ago. Updated 6 months ago.

Status:
Closed
Priority:
Low
Assignee:
Category:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

I've been doing some testing of the feature in bhyve where expanding the underlying storage for a vioblk device causes a configuration changed interrupt to be delivered.

When this occurs and the size of the disk has increased, the driver could send a DLE (dynamic LUN expansion) sysevent so that other parts of the system (e.g. ZFS) can take appropriate action.


Related issues

Related to illumos gate - Feature #14469: nvme could raise dynamic lun expansion syseventsClosedAndy Fiddaman

Actions
Related to illumos gate - Bug #14022: zpool online -e breaks access to poolClosedAndy Fiddaman

Actions
Actions #1

Updated by Andy Fiddaman 6 months ago

  • Related to Feature #14469: nvme could raise dynamic lun expansion sysevents added
Actions #2

Updated by Electric Monk 6 months ago

  • Gerrit CR set to 2002
Actions #3

Updated by Andy Fiddaman 6 months ago

  • Related to Bug #14022: zpool online -e breaks access to pool added
Actions #4

Updated by Andy Fiddaman 6 months ago

I've tested this in a bhyve VM with various vioblk disks and pool configurations, resizing the ZFS Volume from outside the VM and checking that the pool can be expanded (checked with both the autoexpand pool property and manual expansion with online -e on a system with #14022

I tested both both virtio/vioblk using MSI-X interrupts and also modified the vioblk driver to use fixed interrupts/ISR and confirmed that the events were still delivered as expected.

I've used the sysevent utility from smartos-live to watch the DLE events in real-time, and also run the syseventd in debug mode to watch processing there.

Here is a test with a pool on a GPT-partitioned disk, first without a UEFI boot partition:


root@bhyvetest:~# diskinfo
TYPE    DISK                    VID      PID              SIZE          RMV SSD
-       c1t0d0                  Virtio   Block Device       10.00 GiB   no  no
-       c2t0d0                  Virtio   Block Device       10.00 GiB   no  no

root@bhyvetest:~# zpool create bob c2t0d0
root@bhyvetest:~# prtvtoc -s /dev/rdsk/c2t0d0p0
*                            First       Sector      Last
* Partition  Tag  Flags      Sector       Count      Sector  Mount Directory
       0      4    00          256    20954847    20955102
       8     11    00     20955103       16384    20971486
root@bhyvetest:~# zpool list bob
NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
bob   9.50G  1.12M  9.50G        -         -     0%     0%  1.00x  ONLINE  -
root@bhyvetest:~# zdb | grep phys_path
            phys_path: '/pci@0,0/pci1af4,2@5/blkdev@0,0:a'          <-- note 'a'

... volume resize, outside the VM

root@bhyvetest:~# zpool list bob
NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
bob   9.50G  1.12M  9.50G        -       11G     0%     0%  1.00x  ONLINE  -
root@bhyvetest:~# zpool online -e bob c2t0d0
root@bhyvetest:~# zpool list bob
NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
bob   20.5G  1.38M  20.5G        -         -     0%     0%  1.00x  ONLINE  -

and with a UEFI boot partition:

root@bhyvetest:~# zpool create -B bob c2t0d0
root@bhyvetest:~# prtvtoc -s /dev/rdsk/c2t0d0p0
*                            First       Sector      Last
* Partition  Tag  Flags      Sector       Count      Sector  Mount Directory
       0     12    00          256      524288      524543
       1      4    00       524544    43499231    44023774
       8     11    00     44023775       16384    44040158
root@bhyvetest:~# zpool list bob
NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
bob   20.5G   744K  20.5G        -        1G     0%     0%  1.00x  ONLINE  -
root@bhyvetest:~# zdb | grep phys_path
            phys_path: '/pci@0,0/pci1af4,2@5/blkdev@0,0:b'          <-- note 'b'

... volume resize, outside the VM

root@bhyvetest:~# zpool online -e bob c2t0d0
root@bhyvetest:~# zpool list bob
NAME   SIZE  ALLOC   FREE  CKPOINT  EXPANDSZ   FRAG    CAP  DEDUP  HEALTH  ALTROOT
bob   21.5G  1.10M  21.5G        -         -     0%     0%  1.00x  ONLINE  -

Actions #5

Updated by Electric Monk 6 months ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit 501bc5c03a221e7e601a0cfa80ab0972bcf20330

commit  501bc5c03a221e7e601a0cfa80ab0972bcf20330
Author: Andy Fiddaman <omnios@citrus-it.co.uk>
Date:   2022-02-21T19:31:10.000Z

    14471 vioblk could raise dynamic lun expansion sysevents
    Reviewed by: Robert Mustacchi <rm+illumos@fingolfin.org>
    Reviewed by: Jason King <jason.brian.king+illumos@gmail.com>
    Approved by: Dan McDonald <danmcd@joyent.com>

Actions

Also available in: Atom PDF