Project

General

Profile

Actions

Bug #7269

closed

vmxnet3s declares wrong sdu

Added by Robert Mustacchi almost 5 years ago. Updated almost 5 years ago.

Status:
Closed
Priority:
Normal
Category:
driver - device drivers
Start date:
2016-08-03
Due date:
% Done:

100%

Estimated time:
Difficulty:
Medium
Tags:
Gerrit CR:

Description

A user was having QEMU abort. If we look at the stack, we found the following:

> $C
fffffd7fffdecfa0 libc.so.1`_lwp_kill+0xa()
fffffd7fffdecfd0 libc.so.1`raise+0x20(6)
fffffd7fffded020 libc.so.1`abort+0x98()
fffffd7fffded0b0 vnic_receive_iov+0x2fc(f6c2e0, fffffd7fffdf5320, 1)
fffffd7fffded150 qemu_vlan_deliver_packet_iov+0xe0(c1079380, 0, fffffd7fffdf5320, 1, f67200)
fffffd7fffded1e0 qemu_net_queue_deliver_iov+0x72(f67240, c1079380, 0, fffffd7fffdf5320, 1)
fffffd7fffded270 qemu_net_queue_send_iov+0x92(f67240, c1079380, 0, fffffd7fffdf5320, 1, 4edb21)
fffffd7fffded2e0 qemu_sendv_packet_async+0xbe(c1079380, fffffd7fffdf5320, 1, 4edb21)
fffffd7fffdf9360 virtio_net_flush_tx+0x19b(c106b400, c1077fc0)
fffffd7fffdf93a0 virtio_net_tx_timer+0x8f(c106b400)
fffffd7fffdf93f0 qemu_run_timers+0xb0(f62620)
fffffd7fffdf9400 qemu_run_all_timers+0x51()
fffffd7fffdff490 main_loop_wait+0x480(0)
fffffd7fffdff4c0 kvm_main_loop+0x119()
fffffd7fffdff4e0 main_loop+0x17()
fffffd7fffdff820 main+0x2da4(2e, fffffd7fffdff848, fffffd7fffdff9c0)
fffffd7fffdff830 _start+0x6c()
> *errno/p
mdb: failed to read data from target: no mapping for address
0x22:           

The error number is ERANGE. The only way that we can return ERANGE from the framed I/O write functions is if the size of the data is outside the valid SDU. Let's see how much data we tried to write:

> f6c2e0::print VNICState vns_wfio[]
vns_wfio = {
    vns_wfio->fio_version = 0x1
    vns_wfio->fio_nvpf = 0x1
    vns_wfio->fio_nvecs = 0x1
    vns_wfio->fio_vecs = [ ... ]
}
> f6c2e0::print VNICState vns_wfio->fio_vecs[0]
mdb: index 0 is outside of array bounds [0 .. ffffffff]
vns_wfio->fio_vecs[0] = {
    vns_wfio->fio_vecs[0].fv_buf = 0xb9a6be02
    vns_wfio->fio_vecs[0].fv_buflen = 0x2a
    vns_wfio->fio_vecs[0].fv_actlen = 0x3e
}

So, 0x2a is certainly not going to exceed the upper bound. However, this corresponds to a VNIC over the vmxnet3s driver. Based on some additional information we saw that the minimum SDU was being advertised as 60 bytes!

If we look at the actual code in vmxnet3_main.c, we'll see that the min sdu is being set to the minimum MTU.

1459         macr->m_min_sdu = VMXNET3_MIN_MTU;

While the maximum SDU is the same thing as the MTU, It's actualy not the case that the minimum MTU is the same thing as the minimum SDU. The SDU represents the minimum amount of data that can be sent. With most devices, they'll pad out automatically.

In fact, we know it can send these smaller frames just fine as the IP and ARP modules are basicaly ignoring the minimum SDU information from the DL_INFO_REQ. Only vnd is honoring it which is leading us down here.

Actions #1

Updated by Electric Monk almost 5 years ago

  • Status changed from New to Closed

git commit f061e8dcd16c62605d130648b5c5e5181f6868f9

commit  f061e8dcd16c62605d130648b5c5e5181f6868f9
Author: Robert Mustacchi <rm@joyent.com>
Date:   2016-08-13T15:50:26.000Z

    7269 vmxnet3s declares wrong sdu
    Reviewed by: Patrick Mooney <patrick.mooney@joyent.com>
    Reviewed by: Sebastien Roy <sebastien.roy@delphix.com>
    Reviewed by: Garrett D'Amore <garrett@damore.org>
    Approved by: Gordon Ross <gordon.w.ross@gmail.com>

Actions

Also available in: Atom PDF