Project

General

Profile

Actions
Copy link

Bug #14787

closed

bhyve should block leases on drv purge

Added by Patrick Mooney 3 months ago. Updated 2 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Patrick Mooney
Category:
bhyve
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:
2202
External Bug:

Description

When purging (forcing them to release their holds) consumers of the vmm_drv interface which have attached to a given instance, bhyve should also go through the logic which blocks leases from being established through those existing vmm_hold references. Any lingering leases at that point (such as those used from the viona ring threads) will then be notified so they can drop their lease, cleaning out references to the to-be-destroyed vmm instance.

This was found while testing bits drafted for #14755. It was possible for bhyve consumer processes to get hung up on the way down, since the ring worker threads were not notified of the pending destruction.

Actions
Copy link
 #1

Updated by Electric Monk 3 months ago

  • Gerrit CR set to 2202
Actions
Copy link
 #2

Updated by Patrick Mooney 3 months ago

I dtraced the relevant calls to vmm_block_lease while destroying a vmm instance which was attached to a propolis process (although was not actively running vCPUs at the time):

 16  67004            vmm_lease_block:entry   vmm_lease_block
              vmm`vmm_drv_purge+0x8a
              vmm`vmm_do_vm_destroy_locked+0x64
              vmm`vmmdev_do_vm_destroy+0xae
              vmm`vmm_ctl_ioctl+0xe2
              vmm`vmm_ioctl+0x10b
              genunix`cdev_ioctl+0x2b
              specfs`spec_ioctl+0x45
              genunix`fop_ioctl+0x5b
              genunix`ioctl+0x153
              unix`sys_syscall+0x283

 16  67008          vmm_lease_unblock:entry   vmm_lease_unblock
              vmm`vmm_drv_purge+0x92
              vmm`vmm_do_vm_destroy_locked+0x64
              vmm`vmmdev_do_vm_destroy+0xae
              vmm`vmm_ctl_ioctl+0xe2
              vmm`vmm_ioctl+0x10b
              genunix`cdev_ioctl+0x2b
              specfs`spec_ioctl+0x45
              genunix`fop_ioctl+0x5b
              genunix`ioctl+0x153
              unix`sys_syscall+0x283

This shows the expected purge of vmm_drv leases while the instance is marked for destruction. The machine in question was dumped after this to check for kmem leaks (not that any were expected). ::findleaks was clean after the test and dump

Actions
Copy link
 #3

Updated by Patrick Mooney 3 months ago

It should be noted that this ticket was filed to provide additional background for changes made as part of #14755. This was not a case of broken behavior observed in the wild, but rather a desire to make the vmm_drv portion of instance destruction more lively when it came to leases held by viona worker threads.

Actions
Copy link
 #4

Updated by Electric Monk 2 months ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit a26f9c149bc8e4c9206303674cdef16edec1ca70

commit  a26f9c149bc8e4c9206303674cdef16edec1ca70
Author: Patrick Mooney <pmooney@pfmooney.com>
Date:   2022-07-14T00:47:06.000Z

    14755 viona should expose more ring state
    14756 expose viona kernel interface version
    14787 bhyve should block leases on drv purge
    Reviewed by: Garrett D'Amore <garrett@damore.org>
    Reviewed by: Luqman Aden <luqman@oxide.computer>
    Approved by: Richard Lowe <richlowe@richlowe.net>

Actions
Copy link

Also available in: Atom PDF