illumos gate

Upstreaming OS-4600 for bhyve:

Joshua Clulow hit a case where he had a zone and a KVM instance on the same VLAN. However, whenever the KVM instance tried to receive traffic from zones on the same box, it failed, despite the fact that it could receive traffic locally.

We were able to observe that the problem was that we were missing their IP Level checksums. The problem here is that the vnics in question were created over a device that supported hardware checksums. This meant that the vnics advertised that they supported hardware checksums and thus the ip stack did not do checksums on its own.

Normally this is fine, because the packets would have checksums put on them when they left the box or mac would fix the checksum before looping it back internally. However, the promiscuous callback hooks fire before this happens.

It is insufficient to just have vnd try and fix up the checksum, instead, we want to add a new mac promisc option which allows us to fix these kinds of issues up.

In addition, as a side effect we need to make sure that when we copy a block we actually copy the relevant checksum information across to the new data block so it isn't lost when being copied.