mac_unregister does not clear mi_dstaddr_set
From the evaluation in the Joyent bug report:
The final problem here is one of how mac interacts with its kmem_cache i_mac_impl_cachep. Recall that a kmem_cache's constructor and destructor are used to set invariants in the object. Everything else is the responsibility of the user of the kmem_cache. What ended up happening is that in the past, various iptun devices were created. Now iptun itself is special in that it forces every packet that goes over it to go over a specific mac address and this is inforced in mac via the mac_header function and the variables mac_dstaddr_set and mac_dstaddr. The mac_dstaddr_set is a boolean_t to determine whether or not the address in mac_dstaddr is to be used.
mac_unregister correctly bzeros the contents of mac_dstaddr. However, it does not zero out mac_dstaddr_set. That means it goes back into the kmem_cache with that value set. Because of this, we can come along and create the mac_impl_t which corresponds to some other mac client, a physical nic, or much more likely, a vnic, and this value will be set for us, but the alternate mac address will all be zeros meaning that our packet effectively gets black holed. The solution here is to simply zero that field.