Project

General

Profile

Actions

Bug #13860

closed

bhyve nvme with ramdisk backing store no longer works after 13674

Added by Andy Fiddaman about 2 months ago. Updated about 2 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
bhyve
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Bite-size
Tags:
Gerrit CR:

Description

It has always been possible to create bhyve nvme devices with a ramdisk backing store via a command line option such as:

-s 7,nvme,ram=1024

However, after 13674, this no longer works:

bhyve-r36: Could not open backing file: ram=1024: No such file or directory
nvme: Could not open backing file: No such file or directory
device emulation initialization error: No such file or directory
Actions #1

Updated by Electric Monk about 2 months ago

  • Gerrit CR set to 1537
Actions #2

Updated by Andy Fiddaman about 2 months ago

bhyve(1M) - https://man.omnios.org/man1m/bhyve - says:

NVMe devices:

path
Accepted device paths are: /dev/blockdev or /path/to/image or ram=size_in_MiB.

and bhyve_config(4) - https://man.omnios.org/bhyve_config - says

NVMe Controller Settings
Each NVMe controller supports a single storage device. The device can be backed either by a memory disk described by the ram variable, or a block device using the block device settings described above. In addition, each controller supports the following settings:

Name    Format    Default    Description
ram    integer        If set, allocate a memory disk as the backing store. The value of this variable is the size of the memory disk in megabytes.

However, attempting to start bhyve with a path of ram=1024 generates this configuration:

pci.0.7.0.device=nvme
pci.0.7.0.path=ram=1024

where the nvme backend is expecting a ram attribute.

Actions #3

Updated by Andy Fiddaman about 2 months ago

Before the referenced patch:

% bhyve -s 7,nvme,ram=1024 -o config.dump=1 xx
acpi_tables=false
memory.size=256M
x86.strictmsr=true
pci.0.7.0.device=nvme
pci.0.7.0.path=ram=1024
config.dump=1
name=xx

and after:

% bhyve -s 7,nvme,ram=1024 -o config.dump=1 xx
acpi_tables=false
memory.size=256M
x86.strictmsr=true
pci.0.7.0.device=nvme
pci.0.7.0.path=ram=1024
pci.0.7.0.ram=1024
config.dump=1
name=xx

and the VM boots and shows a present 1GiB NVMe device:

% pfexec diskinfo
TYPE    DISK                    VID      PID              SIZE          RMV SSD
-       c2t0d0                  Virtio   Block Device      300.00 GiB   no  no
NVME    c3t589CFC2075170001d0   NVMe     bhyve-NVMe          1.00 GiB   no  yes
Actions #4

Updated by Andy Fiddaman about 2 months ago

Upstream change for reference: https://reviews.freebsd.org/D30717

Actions #5

Updated by Andy Fiddaman about 2 months ago

I also tested that using nvme with block devices still functions as expected.
The legacy (CLI flags) parser produces expected output (unchanged from before):

# bhyve -s 7,nvme,/dev/zvol/rdsk/data/omniosce/hdd-r151038 -o config.dump=1 xx | grep pci
pci.0.7.0.device=nvme
pci.0.7.0.path=/dev/zvol/rdsk/data/omniosce/hdd-r151038

And the VM is running with the NVMe disk:

r151038% pfexec diskinfo
TYPE    DISK                    VID      PID              SIZE          RMV SSD
NVME    c2t589CFC206D150001d0   NVMe     bhyve-NVMe        300.00 GiB   no  yes
Actions #6

Updated by Electric Monk about 2 months ago

  • Status changed from In Progress to Closed
  • % Done changed from 0 to 100

git commit 76e6cd87e3c5c67bcf83c6a51c5fc85ce41e031a

commit  76e6cd87e3c5c67bcf83c6a51c5fc85ce41e031a
Author: Andy Fiddaman <omnios@citrus-it.co.uk>
Date:   2021-06-10T14:24:06.000Z

    13860 bhyve nvme with ramdisk backing store no longer works after 13674
    Reviewed by: Jason King <jason.brian.king@gmail.com>
    Reviewed by: Hans Rosenfeld <rosenfeld@grumpf.hope-2000.org>
    Reviewed by: Toomas Soome <tsoome@me.com>
    Approved by: Robert Mustacchi <rm@fingolfin.org>

Actions

Also available in: Atom PDF