DPDK integration with x550 NICs

DPDK integration with x550 NICs

In the past I was able to successfully integrate the dpdk with sr-iov on i350 NICs with ovs-dpdk.

Now we are working to integrate it on the x553 NIC of the atom c3758 processor as well as the Broadwell-de built-in NICs, and it’s not going so well..

I started with a clean setup, installed the last stable CentOS release (7.5), downloaded and compiled the last stable IXGBE driver (5.3.7) and the recent stable dpdk (17.11.3). The OS is configured for mmu and huge pages as follow:

iommu=pt intel_iommu=on default_hugepagesz=1G hugepagesz=1G hugepages=4

also virtualization settings are enabled in bios.

As advised by Intel engineer, I used the “trusted VF” mode for traffic diversion to the VF.

First I tested with a linux bridge that the sr-iov is functioning properly:

>insmod ./ixgbe.ko max_vfs=1,1,1,1

>ip l s eth2 up;

>ip l s enp9s0f1 up;

>ip l s enp9s16 up;

>ip l s enp9s16f1 up;

>brctl addbr brvf;

>brctl addif brvf enp9s16;

>brctl addif brvf enp9s16f1;

>ip l s eth2 vf 0 trust on

>ip l s enp9s0f1 vf 0 trust on

>ip l s eth2 promisc on;

>ip l s enp9s0f1 promisc on;

>ip l s eth2 vf 0 spoof off;

>ip l s enp9s0f1 vf 0 spoof off;

>ip l s brvf up

 

with this configuration both tagged and untagged traffic is forwarded successfully both ways.

Next step, I bound the VFs to the vfio-pci driver and tried using the dpdk “l2fwd” example application: (same steps as we took validating the i350 functionality before)

>modprobe vfio-pci

>~/dpdk-stable-17.11.3/usertools/dpdk-devbind.py --bind=vfio-pci 0000:09:10.0

>~/dpdk-stable-17.11.3/usertools/dpdk-devbind.py --bind=vfio-pci 0000:09:10.1

>~/dpdk-stable-17.11.3/usertools/dpdk-devbind.py --status

Network devices using DPDK-compatible driver

============================================

0000:09:10.0 'X553 Virtual Function 15c5' drv=vfio-pci unused=ixgbevf

0000:09:10.1 'X553 Virtual Function 15c5' drv=vfio-pci unused=ixgbevf

>ip l s eth2 up;

>ip l s enp9s0f1 up;

>ip l s eth2 vf 0 trust on;

>ip l s enp9s0f1 vf 0 trust on

>ip l s eth2 promisc on;

>ip l s enp9s0f1 promisc on;

>ip l s eth2 vf 0 spoof off;

>ip l s enp9s0f1 vf 0 spoof off

>~/dpdk-stable-17.11.3/build/examples/l2fwd/build/l2fwd -l 0-3 -n 4 -- -q 8 -p 3

 

Unfortunately at this point no traffic is received nor forwarded by the l2fwd application. All counters are flat zero.

It’s worth mentioning that the dpdk package was compiled using this quick start guide, and the “testpmd” application, mentioned in the guide, is working properly.

Please advise, do you see any problem in the path I’m taking or in my setup?

Do you have an idea how I can debug this?

4 posts / 0 new
Last post
For more complete information about compiler optimizations, see our Optimization Notice.

Can you make sure that your OvS is properly configured.

https://software.intel.com/en-us/articles/using-open-vswitch-with-dpdk-f...

Hi Yaser,

The Ovs is configured properly and works properly with the i350 ports.

as I mentioned in the original post, even the basic L2fwd example application of the dpdk package, does not receive any packets
 

Could you follow and apply these well tested configurations. And if you still have issues report the error.

https://fast.dpdk.org/doc/perf/DPDK_17_11_Intel_NIC_performance_report.pdf

​Yaser

Leave a Comment

Please sign in to add a comment. Not a member? Join today