Packets encapsulated under VXLAN are not fragmented/reassembled properly

[aserdean]

Issuing a command with a packet size greater than the external interface MTU results
in dropped packets.

Steps to reproduce:

ping -l 1600 10.0.6.25

The request is sent and fragmented but no reply comes back (this happens from
Hyper-V to Hyper-V but also from KVM to Hyper-V).

[nithinrajub]

Alan,
It is recommended that we configure the VIF MTU to be smaller than the physical NIC's MTU. Something like 100 bytes lesser will do. This will give enough room for the tunnel header to be added.

I agree, that we should be able to take care of this better.

[Samuel-Ghinet]

Currently, in order for tunneling to work, the mtu must be configured in every VM (by the client / user of the VM) to account for max additional bytes added by encapsulation protocol(s).

Also, any mtu change that the user / client may do in his OS (VM) may break tunneling functionality.

For production I believe we cannot:
a) expect the users of the VMs to change the MTU
b) allow the users to break tunneling by making changes in their configuration.

Expected functionality:
If original_packet_size + encap_size > external_nic_mtu => must:
a) encapsulate then fragment the buffer (if the payload ip header has not DF set)
b) send icmp4 error back to the VM if ipv4 and DF is set
c) send icmp6 error back to the VM if ipv6

[EitanEliahu]

I came across this "Windows Server 2012 R2 Network Virtualization (NVGRE) MTU reduction" 👍
http://social.technet.microsoft.com/wiki/contents/articles/23833.windows-server-2012-r2-network-virtualization-nvgre-mtu-reduction.aspx
It seems that while the synthetic NIC MTU is reduced but if fragmentation is needed it will be done.

[Samuel-Ghinet]

Eitan,

MS has a component (quite new, actually) called Network Virtualization, where they implemented the NVGRE (not the same as the GRE that we have). As I understand from the article, they have modified their switch implementation to automatically take into account their NVGRE.

I believe that no custom implementation of a tunneling could actually make use of their mechanism.
Unless we convince the MS guys to modify the hyper-v switch implementation as to allow registering custom-made tunneling protocols (to take advantage of MTU and possibly other things that might be), that is.

[EitanEliahu]

The only part we need here is how to reduce to automatically reduce the VNIC MTU. I am not sure if this is part of the NVGRE implementation (probably not).

[Samuel-Ghinet]

From what I read from the article you gave, it automatically reduces the MTU as considered required for NVGRE.

The only alternative would be if you have a WMI / powershell cmdlet by which you can reduce the MTU of a given vmNic. I once sought one for doing that, but did not find. I don't remember whether it was for Win Server 2012 or 2012R2.

See if you can find some way to configure the MTU of a vmNic via powershell. This might save us the trouble :)

[aserdean]

This issue has been solved

[sacheth003]

@aserdean Can you provide some guidance into what was done to solve this issue? I am facing some similar problems.