Project

General

Profile

Actions

Bug #15256

closed

viona doesn't setup datapath for non-VNIC devices

Added by Luqman Aden 6 months ago. Updated 5 months ago.

Status:
Closed
Priority:
Normal
Assignee:
Category:
-
Start date:
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:
External Bug:

Description

A viona instance is usually used with a VNIC but should work atop any MAC provider. But today it relies on private behaviour between the MAC layer and VNIC driver:

/*
 * mac_client.c:
 * 
 * When VNICs are created on top of an underlying MAC, this causes
 * a layering of two MAC instances. Since the lower MAC already
 * does the switching and demultiplexing to its MAC clients, the
 * upper MAC would simply have to pass packets to the layer below
 * or above it, which would introduce overhead. In order to avoid
 * this overhead, the MAC layer implements a pass-through mechanism
 * for VNICs. When a VNIC opens the lower MAC instance, it saves
 * the MAC client handle it optains from the MAC layer. When a MAC
 * client opens a VNIC (upper MAC), the MAC layer detects that
 * the MAC being opened is a VNIC, and gets the MAC client handle
 * that the VNIC driver obtained from the lower MAC. This exchange
 * is done through a private capability between the MAC layer
 * and the VNIC driver. The upper MAC then returns that handle
 * directly to its MAC client. Any operation done by the upper
 * MAC client is now done on the lower MAC client handle, which
 * allows the VNIC driver to be completely bypassed for the
 * performance sensitive data-path.
 */

The VNIC driver opens a MAC client handle to the lower MAC and configures it with a unicast address setting up a datapath. Viona implicitly relies on this so attempting to use a non-VNIC device will result in packets not being delivered from/to the underlying MAC provider.

Actions #1

Updated by Electric Monk 6 months ago

  • Gerrit CR set to 2558
Actions #2

Updated by Luqman Aden 5 months ago

Testing notes:

Both c-bhyve & propolis work as before using viona w/ a vnic device (via dladm create-vnic)

Also tested adding a secondary mac to the vnic (via dladm set-linkprop -p secondary-macs=xx:xx:xx:xx:xx:xx vnic0) and creating a macvlan dev with the secondary mac inside a linux guest with both c-bhyve and propolis.

Both c-bhyve & propolis work as before using viona w/ a vlan device (via dladm create-vlan)

No changes were made to c-bhyve and so it continues to only expect a VNIC as before. I did modify propolis to remove the VNIC restriction and used viona successfully atop an xde (our out-of-tree mac provider) link.

Actions #3

Updated by Luqman Aden 5 months ago

Patrick also tested w/ vnics atop ixgbe links and was able to get roughly line-rate in a c-bhyve guest.

Actions #4

Updated by Electric Monk 5 months ago

  • Status changed from New to Closed
  • % Done changed from 0 to 100

git commit 1165c30927ddd6622eb8e71a57510eaea0bfb947

commit  1165c30927ddd6622eb8e71a57510eaea0bfb947
Author: Luqman Aden <luqman@oxide.computer>
Date:   2023-01-07T14:12:30.000Z

    15256 viona doesn't setup datapath for non-VNIC devices
    Reviewed by: Robert Mustacchi <rm@fingolfin.org>
    Reviewed by: Patrick Mooney <pmooney@pfmooney.com>
    Approved by: Gordon Ross <gordon.w.ross@gmail.com>

Actions

Also available in: Atom PDF