Optimizing the Virtual Network with VXLAN Overlay Offloading

Virtualized and overlays networks are not a new concept in networking, but have come to the forefront as being a core building block for building virtual networks, software defined networks, service function chaining, and abstracting large-scale virtualization. Many standards have evolved to abstract the underlying network or physical network infrastructure to create virtual networks.  Some of the tunneling techniques such as,  VXLAN, NVGRE, and SST use completely different encapsulation methods to overcome curtain network limitations.  But while each tunneling protocol can be identified in their different approaches they can share some similarities in performance.


Because each of these tunneling protocols wraps a UDP header around the original packet using the offloading to accelerate perform may not always work. Tunneling protocols introduce an additional layer of packet processing. Adding and removing protocol headers causes the CPU to do additional work for each packet.  Also, some of the offload capabilities of the NIC can not be used because the inner packet is no longer accessible. Checksum offloading and TCP segmentation offload (TSO) have been designed for standard packet headers and VXLAN, for example  is encapsulated with a UDP header invalidating many of these offloading techniques and putting more cycles on the CPU that otherwise could be performed more efficiently by hardware offloading.  

In general, the performance may vary based on the underlying hardware used when VXLAN is configured in Open vSwitch.  But worry not, the ixgbe Linux driver features VXLAN Overlay HW Offloading support and is enabled in the driver by default.  To view and configure VXLAN on a VXLAN-overlay offload enabled devices for Linux use the following ethtool commands:

This command displays the offloads and the current state:

ethtool -k ethX

This command enables/disables VXLAN support in the driver:

ethtool -K ethX tx-udp_tnl-segmentation [off|on]

To add tunnel acceleration to a virtual network see “Creating Overlay Networks Using Intel Ethernet Converged Network Adapters” to get started with VXLAN and GRE using Open vSwitch.

Download the latest Intel Ethernet driver source code and check if  for your network adapters support VXLAN Overlay HW Offloading.

For more complete information about compiler optimizations, see our Optimization Notice.