在英特尔®架构服务器上使用矢量包处理（VPP）构建快速的网络栈

Build a Fast Network Stack with Vector Packet Processing (VPP) on an Intel® Architecture Server

Introduction

Vector Packet Processing (VPP) Overview

Build and Install VPP

Build the FD.io VPP Binary

Build the Debian* VPP Packages

Configure VPP

Configure huge pages

Configure startup.conf

Run VPP as a Packet Processing Engine

Example 1: Using Kernel Packet Forwarding with iperf3*

Example 2: Using VPP with iperf3

Example 3. Using VPP with the TREX* Realistic Traffic Generator

Example 4: Using VPP with TRex Mixed Traffic Templates

Summary

About the Author

References

产品和性能信息

Build a Fast Network Stack with Vector Packet Processing (VPP) on an Intel® Architecture Server

https://software.intel.com/content/www/us/en/develop/articles/build-a-fast-network-stack-with-vpp-on-an-intel-architecture-server.html

Introduction

This tutorial shows how to install the FD.io Vector Packet Processing (VPP) package and build a packet forwarding engine on a bare metal Intel® Xeon® processor server. Two additional Intel Xeon processor platform systems are used to connect to the VPP host to pass traffic using iperf3* and Cisco’s TRex* Realistic Traffic Generator (TRex*). Intel 40 Gigabit Ethernet (GbE) network interface cards (NICs) are used to connect the hosts.

本教程介绍了如何在裸机英特尔®至强®处理器服务器上安装FD.io矢量数据包处理（VPP）软件包并构建数据包转发引擎。另外两个Intel Xeon处理器平台系统用于连接到VPP主机，以使用iperf3 *和Cisco的TRex *现实流量生成器（TRex *）传递流量。英特尔40 Gb以太网（GbE）网络接口卡（NIC）用于连接主机。

Vector Packet Processing (VPP) Overview

VPP is open source high-performance packet processing software. It leverages the Data Plane Development Kit (DPDK) to take advantage of fast I/O. DPDK provides fast packet processing libraries and user space drivers. It receives and send packets with a minimum number of CPU cycles by bypassing the kernel and using a user poll mode driver. Details on how to configure DPDK can be found in the DPDK Documentation.

VPP是开源的高性能数据包处理软件。它利用Data Plane Development Kit（DPDK）来利用快速I / O。 DPDK提供了快速的数据包处理库和用户空间驱动程序。通过绕过内核并使用用户轮询模式驱动程序，它以最少的CPU周期接收和发送数据包。有关如何配置DPDK的详细信息，请参见DPDK文档。

VPP can be used as a standalone product or as an extended data plane product. It is highly efficient because it scales well on modern Intel® processors and handles packet processing in batches, called vectors, up to 256 packets at a time. This approach ensures that cache hits will be maximized.

VPP可以用作独立产品或扩展数据平面产品。它具有很高的效率，因为它可以在现代英特尔®处理器上很好地扩展并可以批量处理称为向量的数据包处理，一次最多可处理256个数据包。这种方法确保了缓存命中将最大化。

The VPP platform consists of a set of nodes in a directed graph called a packet processing graph. Each node provides a specific network function to packets, and each directed edge indicates the next network function that will handle packets. Instead of processing one packet at a time as the kernel does, the first node in the packet processing graph polls for a burst of incoming packets from a network interface; it collects similar packets into a frame (or vector), and passes the frame to the next node indicated by the directed edge. The next node takes the frame of packets, processes them based on the functionality it provides, passes the frame to the next node, and so on. This process repeats until the last node gets the frame, processes all the packets in the frame based on the functionality it provides, and outputs them on a network interface. When a frame of packets is handled by a node only the first packet in the frame needs to load the CPU’s instructions to the cache; the rest of the packets benefit from the instruction already in the cache. VPP architecture is flexible to allow users to create new nodes, enter them into the packet processing graph, and rearrange the graph.

VPP平台由称为包处理图的有向图中的一组节点组成。每个节点为数据包提供特定的网络功能，每个有向边指示将处理数据包的下一个网络功能。数据包处理图中的第一个节点不是像内核那样一次处理一个数据包，而是轮询来自网络接口的传入数据包的突发。它将相似的数据包收集到一个帧（或向量）中，并将该帧传递到有向边指示的下一个节点。下一个节点获取数据包的帧，根据其提供的功能对其进行处理，然后将帧传递给下一个节点，依此类推。重复此过程，直到最后一个节点获取帧，然后根据其提供的功能处理帧中的所有数据包，然后在网络接口上输出它们。当一个节点处理一个数据包帧时，只有该帧中的第一个数据包才需要将CPU的指令加载到缓存中。其余数据包则受益于缓存中已有的指令。 VPP体系结构非常灵活，允许用户创建新节点，将其输入到数据包处理图中，然后重新排列该图。

Like DPDK, VPP operates in user space. VPP can be used on bare metal, virtual machines (VMs), or containers.

Build and Install VPP

In this tutorial, three systems named csp2s22c03, csp2s22c04, and net2s22c05 are used. The system csp2s22c03, with VPP installed, is used to forward packets, and the systems csp2s22c04 and net2s22c05 are used to pass traffic. All three systems are equipped with Intel® Xeon® processor E5-2699 v4 @ 2.20 GHz, two sockets with 22 cores per socket, and are running 64-bit Ubuntu* 16.04 LTS. The Intel® Ethernet Converged Network Adapter XL710 10/40 GbE is used to connect these systems. Refer to Figure 1 and Figure 2 for configuration diagrams.

在本教程中，使用了三个名为csp2s22c03，csp2s22c04和net2s22c05的系统。已安装VPP的系统csp2s22c03用于转发数据包，而系统csp2s22c04和net2s22c05用于传递流量。这三个系统都配备了英特尔®至强®处理器E5-2699 v4 @ 2.20 GHz，两个插槽（每个插槽22个内核），并运行64位Ubuntu * 16.04 LTS。英特尔®以太网聚合网络适配器XL710 10/40 GbE用于连接这些系统。有关配置图，请参见图1和图2。

Build the FD.io VPP Binary

The instructions in this section describe how to build the VPP package from FD.io. Skip to the next section if you’d like to use the Debian* VPP packages instead.

本节中的说明描述了如何从FD.io构建VPP软件包。如果您想改用Debian * VPP软件包，请跳至下一部分。

With an admin privileges account in csp2s22c03, we download a stable version of VPP (version 17.04 is used in this tutorial), and navigate to the build-root directory to build the image:

使用csp2s22c03中的管理员特权帐户，我们下载稳定版本的VPP（本教程中使用版本17.04），然后导航到build-root目录以构建映像：

csp2s22c03$ git clone -b stable/1704 https://gerrit.fd.io/r/vpp fdio.1704
csp2s22c03$ cd fdio.1704/
csp2s22c03$ make install-dep
csp2s22c03$ make bootstrap
csp2s22c03$ cd build-root
csp2s22c03$ source ./path_setup
csp2s22c03$ make PLATFORM=vpp TAG=vpp vpp-install

To build the image with debug symbols:

csp2s22c03$ make PLATFORM=vpp TAG=vpp_debug vpp-install

After you've configured VPP, you can run the VPP binary from the fdio.1704 directory using the src/vpp/conf/startup.conf configuration file:

csp2s22c03$ cd ..
csp2s22c03$ sudo build-root/build-vpp-native/vpp/bin/vpp -c src/vpp/conf/startup.conf

Build the Debian* VPP Packages

If you prefer to use the Debian VPP packages, follow these instructions to build them:

如果您更喜欢使用Debian VPP软件包，请按照以下说明进行构建：

csp2s22c03$ make PLATFORM=vpp TAG=vpp install-deb
csp2s22c03:~/download/fdio.1704/build-root$ ls -l *.deb
-rw-r--r-- 1 plse plse 1667422 Feb 12 16:41 vpp_17.04.2-2~ga8f93f8_amd64.deb
-rw-r--r-- 1 plse plse 2329572 Feb 12 16:41 vpp-api-java_17.04.2-2~ga8f93f8_amd64.deb
-rw-r--r-- 1 plse plse 23374 Feb 12 16:41 vpp-api-lua_17.04.2-2~ga8f93f8_amd64.deb
-rw-r--r-- 1 plse plse 8262 Feb 12 16:41 vpp-api-python_17.04.2-2~ga8f93f8_amd64.deb
-rw-r--r-- 1 plse plse 44175468 Feb 12 16:41 vpp-dbg_17.04.2-2~ga8f93f8_amd64.deb
-rw-r--r-- 1 plse plse 433788 Feb 12 16:41 vpp-dev_17.04.2-2~ga8f93f8_amd64.deb
-rw-r--r-- 1 plse plse 1573956 Feb 12 16:41 vpp-lib_17.04.2-2~ga8f93f8_amd64.deb
-rw-r--r-- 1 plse plse 1359024 Feb 12 16:41 vpp-plugins_17.04.2-2~ga8f93f8_amd64.deb

In this output:

vpp is the packet engine

vpp-api-java is the Java* binding module

vpp-api-lua is the Lua* binding module

vpp-api-python is the Python* binding module

vpp-dbg is the debug symbol version of VPP

vpp-dev is the development support (headers and libraries)

vpp-lib is the VPP runtime library

vpp-plugins is the plugin module

vpp是数据包引擎
vpp-api-java是Java *绑定模块
vpp-api-lua是Lua *绑定模块
vpp-api-python是Python *绑定模块
vpp-dbg是VPP的调试符号版本
vpp-dev是开发支持（标头和库）
vpp-lib是VPP运行时库
vpp-plugins是插件模块

Next, install the Debian VPP packages. At a minimum, you should install the VPP, vpp-lib, and vpp-plugins packages). We install them on the machine csp2s22c03:

csp2s22c03$ apt list --installed | grep vpp
csp2s22c03$ sudo dpkg -i vpp_17.04.2-2~ga8f93f8_amd64.deb vpp-lib_17.04.2-2~ga8f93f8_amd64.deb vpp-plugins_17.04.2-2~ga8f93f8_amd64.deb

Verify that the VPP packages are installed successfully:

csp2s22c03$ apt list --installed | grep vpp
vpp/now 17.04.2-2~ga8f93f8 amd64 [installed,upgradable to: 18.01.1-release]
vpp-lib/now 17.04.2-2~ga8f93f8 amd64 [installed,upgradable to: 18.01.1-release]
vpp-plugins/now 17.04.2-2~ga8f93f8 amd64 [installed,upgradable to: 18.01.1-release]

Configure VPP

During installation, two configuration files are created: /etc/sysctl.d/80-vpp.conf and /etc/vpp/startup.conf/startup.conf. The /etc/sysctl.d/80-vpp.conf configuration file is used to set up huge pages. The /etc/vpp/startup.conf/startup.conf configuration file is used to start VPP.

Configure huge pages

In the /etc/sysctl.d/80-vpp.conf configuration file, set parameters as follows: the number of 2 MB huge pages vm.nr_hugepages is chosen to be 4096, and vm.max_map_count is 9216 (2.5 * 4096), shared memory max kernel.shmmax is 8,589,934,592 (4096 * 2 * 1024 * 1024).

csp2s22c03$ cat /etc/sysctl.d/80-vpp.conf
# Number of 2MB hugepages desired
vm.nr_hugepages=4096# Must be greater than or equal to (2 * vm.nr_hugepages).
vm.max_map_count=9216# All groups allowed to access hugepages
vm.hugetlb_shm_group=0# Shared Memory Max must be greator or equal to the total size of hugepages.
# For 2MB pages, TotalHugepageSize = vm.nr_hugepages * 2 * 1024 * 1024
# If the existing kernel.shmmax setting  (cat /sys/proc/kernel/shmmax)
# is greater than the calculated TotalHugepageSize then set this parameter
# to current shmmax value.
kernel.shmmax=8589934592

Apply these memory settings to the system and verify the huge pages:

csp2s22c03$ sudo sysctl -p /etc/sysctl.d/80-vpp.conf
vm.nr_hugepages = 4096
vm.max_map_count = 9216
vm.hugetlb_shm_group = 0
kernel.shmmax = 8589934592csp2s22c03$ cat /proc/meminfo
MemTotal:       131912940 kB
MemFree:        116871136 kB
MemAvailable:   121101956 kB
...............................
HugePages_Total:    4096
HugePages_Free:     3840
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB

Configure startup.conf

In the /etc/vpp/startup.conf/startup.conf configuration file, the keyword interactive is added to enable the VPP Command-Line Interface (CLI). Also, four worker threads are selected and run on cores 2, 3, 22, and 23. Note that you can choose the NIC cards to use in this configuration or you can specify them later, as this exercise shows. The modified /etc/vpp/startup.conf/startup.conf configuration file is shown below.

csp2s22c03$ cat /etc/vpp/startup.confunix {nodaemonlog /tmp/vpp.logfull-coredumpinteractive
}api-trace {}api-segment {gid vpp
}{## In the VPP there is one main thread and optionally the user can create worker(s)## The main thread and worker thread(s) can be pinned to CPU core(s) manually or automatically## Manual pinning of thread(s) to CPU core(s)## Set logical CPU core where main thread runsmain-core 1## Set logical CPU core(s) where worker threads are runningcorelist-workers 2-3,22-23
}dpdk {## Change default settings for all intefaces# dev default {## Number of receive queues, enables RSS## Default is 1# num-rx-queues 3## Number of transmit queues, Default is equal## to number of worker threads or 1 if no workers treads# num-tx-queues 3## Number of descriptors in transmit and receive rings## increasing or reducing number can impact performance## Default is 1024 for both rx and tx# num-rx-desc 512# num-tx-desc 512## VLAN strip offload mode for interface## Default is off# vlan-strip-offload on# }## Whitelist specific interface by specifying PCI address# dev 0000:02:00.0## Whitelist specific interface by specifying PCI address and in## addition specify custom parameters for this interface# dev 0000:02:00.1 {#       num-rx-queues 2# }## Change UIO driver used by VPP, Options are: igb_uio, vfio-pci## and uio_pci_generic (default)# uio-driver vfio-pci
}# Adjusting the plugin path depending on where the VPP plugins are:
plugins
{path /usr/lib/vpp_plugins
}

Run VPP as a Packet Processing Engine

In this section, four examples of running VPP are shown. In the first two examples, the iperf3 tool is used to generate traffic, and in the last two examples the TRex Realistic Traffic Generator is used. For comparison purposes, the first example shows packet forwarding using ordinary kernel IP forwarding, and the second example shows packet forwarding using VPP.

在本节中，显示了运行VPP的四个示例。在前两个示例中，使用iperf3工具生成流量，而在后两个示例中，使用TRex现实流量生成器。为了进行比较，第一个示例显示了使用普通内核IP转发的数据包转发，第二个示例显示了使用VPP的数据包转发。

Example 1: Using Kernel Packet Forwarding with iperf3*

In this test, 40 GbE Intel Ethernet Network Adapters are used to connect the three systems. Figure 1 illustrates this configuration.

在此测试中，使用40 GbE英特尔以太网网络适配器连接三个系统。图1说明了此配置。

Figure 1 – VPP runs on a host that connects to two other systems via 40 GbE NICs.

For comparison purposes, in the first test, we configure kernel forwarding in csp2s22c03 and use the iperf3 tool to measure network bandwidth between csp2s22c03 and net2s22c05. In the second test, we start the VPP engine in csp2s22c03 instead of using kernel forwarding.

为了进行比较，在第一个测试中，我们在csp2s22c03中配置内核转发，并使用iperf3工具测量csp2s22c03和net2s22c05之间的网络带宽。在第二个测试中，我们在csp2s22c03中启动VPP引擎，而不是使用内核转发。

On csp2s22c03, we configure the system to have the addresses 10.10.1.1/24 and 10.10.2.1/24 on the two 40-GbE NICs. To find all network interfaces available on the system, use the lshw Linux* command to list all network interfaces and the corresponding slots [0000:xx:yy.z]. For example, the 40-GbE interfaces are ens802f0 and ens802f1.

在csp2s22c03上，我们将系统配置为在两个40 GbE NIC上分别具有地址10.10.1.1/24和10.10.2.1/24。要查找系统上所有可用的网络接口，请使用lshwLinux *命令列出所有网络接口和相应的插槽[0000：xx：yy.z]。例如，40 GbE接口是ens802f0和ens802f1。

csp2s22c03$ sudo lshw -class network -businfo
Bus info          Device      Class          Description
========================================================
pci@0000:03:00.0  enp3s0f0    network        Ethernet Controller 10-Gigabit X540
pci@0000:03:00.1  enp3s0f1    network        Ethernet Controller 10-Gigabit X540
pci@0000:82:00.0  ens802f0    network        Ethernet Controller XL710 for 40GbE
pci@0000:82:00.1  ens802f1    network        Ethernet Controller XL710 for 40GbE
pci@0000:82:00.0  ens802f0d1  network        Ethernet interface
pci@0000:82:00.1  ens802f1d1  network        Ethernet interface

Configure the system to have 10.10.1.1 and 10.10.2.1 on the two 40-GbE NICs ens802f0 and ens802f1, respectively.

csp2s22c03$ sudo ip addr add 10.10.1.1/24 dev ens802f0
csp2s22c03$ sudo ip link set dev ens802f0 up
csp2s22c03$ sudo ip addr add 10.10.2.1/24 dev ens802f1
csp2s22c03$ sudo ip link set dev ens802f1 up

List the route table:

csp2s22c03$ route
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
default         jf111-ldr1a-530 0.0.0.0         UG    0      0        0 enp3s0f1
default         192.168.0.50    0.0.0.0         UG    100    0        0 enp3s0f0
10.10.1.0       *               255.255.255.0   U     0      0        0 ens802f0
10.10.2.0       *               255.255.255.0   U     0      0        0 ens802f1
10.23.3.0       *               255.255.255.0   U     0      0        0 enp3s0f1
link-local      *               255.255.0.0     U     1000   0        0 enp3s0f1
192.168.0.0     *               255.255.255.0   U     100    0        0 enp3s0f0

csp2s22c03$ ip route
default via 10.23.3.1 dev enp3s0f1
default via 192.168.0.50 dev enp3s0f0  proto static  metric 100
10.10.1.0/24 dev ens802f0  proto kernel  scope link  src 10.10.1.1
10.10.2.0/24 dev ens802f1  proto kernel  scope link  src 10.10.2.1
10.23.3.0/24 dev enp3s0f1  proto kernel  scope link  src 10.23.3.67
169.254.0.0/16 dev enp3s0f1  scope link  metric 1000
192.168.0.0/24 dev enp3s0f0  proto kernel scope link src 192.168.0.142 metric 100

On csp2s22c04, we configure the system to have the address 10.10.1.2 and use the interface ens802 to route IP packets 10.10.2.0/24. Use the lshw Linux command to list all network interfaces and the corresponding slots [0000:xx:yy.z]. For example, the interface ens802d1 (ens802) is connected to slot [82:00.0]:

在csp2s22c04上，我们将系统配置为具有地址10.10.1.2，并使用接口ens802路由IP数据包10.10.2.0/24。使用lshw Linux命令列出所有网络接口和相应的插槽[0000：xx：yy.z]。例如，接口ens802d1（ens802）连接到插槽[82：00.0]：

csp2s22c04$ sudo lshw -class network -businfo
Bus info          Device      Class       Description
=====================================================
pci@0000:03:00.0  enp3s0f0    network     Ethernet Controller 10-Gigabit X540-AT2
pci@0000:03:00.1  enp3s0f1    network     Ethernet Controller 10-Gigabit X540-AT2
pci@0000:82:00.0  ens802d1    network     Ethernet Controller XL710 for 40GbE QSFP+
pci@0000:82:00.0  ens802      network     Ethernet interface

For kernel forwarding, set 10.10.1.2 to the interface ens802, and add a static route for IP packet 10.10.2.0/24:

csp2s22c04$ sudo ip addr add 10.10.1.2/24 dev ens802
csp2s22c04$ sudo ip link set dev ens802 up
csp2s22c04$ sudo ip route add 10.10.2.0/24 via 10.10.1.1

csp2s22c04$ ifconfig
enp3s0f0  Link encap:Ethernet  HWaddr a4:bf:01:00:92:73inet addr:10.23.3.62  Bcast:10.23.3.255  Mask:255.255.255.0inet6 addr: fe80::a6bf:1ff:fe00:9273/64 Scope:LinkUP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1RX packets:3411 errors:0 dropped:0 overruns:0 frame:0TX packets:1179 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:1000RX bytes:262230 (262.2 KB)  TX bytes:139975 (139.9 KB)ens802    Link encap:Ethernet  HWaddr 68:05:ca:2e:76:e0inet addr:10.10.1.2  Bcast:0.0.0.0  Mask:255.255.255.0inet6 addr: fe80::6a05:caff:fe2e:76e0/64 Scope:LinkUP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1RX packets:0 errors:0 dropped:0 overruns:0 frame:0TX packets:40 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:1000RX bytes:0 (0.0 B)  TX bytes:5480 (5.4 KB)lo        Link encap:Local Loopbackinet addr:127.0.0.1  Mask:255.0.0.0inet6 addr: ::1/128 Scope:HostUP LOOPBACK RUNNING  MTU:65536  Metric:1RX packets:31320 errors:0 dropped:0 overruns:0 frame:0TX packets:31320 errors:0 dropped:0 overruns:0 carrier:0collisions:0 txqueuelen:1RX bytes:40301788 (40.3 MB)  TX bytes:40301788 (40.3 MB)

After setting the route, we can ping from csp2s22c03 to csp2s22c04, and vice versa:

csp2s22c03$ ping 10.10.1.2 -c 3
PING 10.10.1.2 (10.10.1.2) 56(84) bytes of data.
64 bytes from 10.10.1.2: icmp_seq=1 ttl=64 time=0.122 ms
64 bytes from 10.10.1.2: icmp_seq=2 ttl=64 time=0.109 ms
64 bytes from 10.10.1.2: icmp_seq=3 ttl=64 time=0.120 ms

csp2s22c04$ ping 10.10.1.1 -c 3
PING 10.10.1.1 (10.10.1.1) 56(84) bytes of data.
64 bytes from 10.10.1.1: icmp_seq=1 ttl=64 time=0.158 ms
64 bytes from 10.10.1.1: icmp_seq=2 ttl=64 time=0.096 ms
64 bytes from 10.10.1.1: icmp_seq=3 ttl=64 time=0.102 ms

Similarly, on net2s22c05, we configure the system to have the address 10.10.2.2 and use the interface ens803f0 to route IP packets 10.10.1.0/24. Use the lshw Linux command to list all network interfaces and the corresponding slots [0000:xx:yy.z]. For example, the interface ens803f0 is connected to slot [87:00.0]:

NET2S22C05$ sudo lshw -class network -businfo
Bus info          Device      Class          Description
========================================================
pci@0000:03:00.0  enp3s0f0    network    Ethernet Controller 10-Gigabit X540-AT2
pci@0000:03:00.1  enp3s0f1    network    Ethernet Controller 10-Gigabit X540-AT2
pci@0000:81:00.0  ens787f0    network    82599 10 Gigabit TN Network Connection
pci@0000:81:00.1  ens787f1    network    82599 10 Gigabit TN Network Connection
pci@0000:87:00.0  ens803f0    network    Ethernet Controller XL710 for 40GbE QSFP+
pci@0000:87:00.1  ens803f1    network    Ethernet Controller XL710 for 40GbE QSFP+

For kernel forwarding, set 10.10.2.2 to the interface ens803f0, and add a static route for IP packet 10.10.1.0/24:

NET2S22C05$ sudo ip addr add 10.10.2.2/24 dev ens803f0
NET2S22C05$ sudo ip link set dev ens803f0 up
NET2S22C05$ sudo ip r add 10.10.1.0/24 via 10.10.2.1

After setting the route, you can ping from csp2s22c03 to net2s22c05, and vice versa. However, in order to ping between net2s22c05 and csp2s22c04, kernel IP forwarding in csp2s22c03 has to be enabled:

csp2s22c03$ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 0
csp2s22c03$ echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward
csp2s22c03$ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 1

If successful, verify that now you can ping between net2s22c05 and csp2s22c04:

NET2S22C05$ ping 10.10.1.2 -c 3
PING 10.10.1.2 (10.10.1.2) 56(84) bytes of data.
64 bytes from 10.10.1.2: icmp_seq=1 ttl=63 time=0.239 ms
64 bytes from 10.10.1.2: icmp_seq=2 ttl=63 time=0.224 ms
64 bytes from 10.10.1.2: icmp_seq=3 ttl=63 time=0.230 ms\

We use the iperf3 utility to measure network bandwidth between hosts. In this test, we download the iperf3 utility tool on both net2s22c05 and csp2s22c04. On csp2s22c04, we start the iperf3 server with “iperf3 –s”, and then on net2s22c05, we start the iperf3 client to connect to the server:

NET2S22C05$ iperf3 -c 10.10.1.2
Connecting to host 10.10.1.2, port 5201
[  4] local 10.10.2.2 port 54074 connected to 10.10.1.2 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec   936 MBytes  7.85 Gbits/sec  2120    447 KBytes
[  4]   1.00-2.00   sec   952 MBytes  7.99 Gbits/sec  1491    611 KBytes
[  4]   2.00-3.00   sec   949 MBytes  7.96 Gbits/sec  2309    604 KBytes
[  4]   3.00-4.00   sec   965 MBytes  8.10 Gbits/sec  1786    571 KBytes
[  4]   4.00-5.00   sec   945 MBytes  7.93 Gbits/sec  1984    424 KBytes
[  4]   5.00-6.00   sec   946 MBytes  7.94 Gbits/sec  1764    611 KBytes
[  4]   6.00-7.00   sec   979 MBytes  8.21 Gbits/sec  1499    655 KBytes
[  4]   7.00-8.00   sec   980 MBytes  8.22 Gbits/sec  1182    867 KBytes
[  4]   8.00-9.00   sec  1008 MBytes  8.45 Gbits/sec  945    625 KBytes
[  4]   9.00-10.00  sec  1015 MBytes  8.51 Gbits/sec  1394    611 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  9.45 GBytes  8.12 Gbits/sec  16474             sender
[  4]   0.00-10.00  sec  9.44 GBytes  8.11 Gbits/sec                  receiveriperf Done.

Using kernel IP forwarding, iperf3 shows the network bandwidth is about 8.12 Gbits per second.

Example 2: Using VPP with iperf3

First, disable kernel IP forward in csp2s22c03 to ensure the host cannot use kernel forwarding (all the settings in net2s22c05 and csp2s22c04 remain unchanged):

csp2s22c03$ echo 0 | sudo tee /proc/sys/net/ipv4/ip_forward
0
csp2s22c03$ sysctl net.ipv4.ip_forward
net.ipv4.ip_forward = 0

You can use DPDK’s device binding utility (./install-vpp-native/dpdk/sbin/dpdk-devbind) to list network devices and bind/unbind them from specific drivers. The flag “-s/--status” shows the status of devices; the flag “-b/--bind” selects the driver to bind. The status of devices in our system indicates that the two 40-GbE XL710 devices are located at 82:00.0 and 82:00.1. Use the device’s slots to bind them to the driver uio_pci_generic:

您可以使用DPDK的设备绑定实用程序（./install-vpp-native/dpdk/sbin/dpdk-devbind）列出网络设备，并从特定的驱动程序进行绑定/取消绑定。标志“ -s /-status”显示设备的状态；标志“ -b /-bind”选择要绑定的驱动程序。我们系统中设备的状态指示两个40 GbE XL710设备位于82：00.0和82：00.1。使用设备的插槽将它们绑定到驱动程序uio_pci_generic：

csp2s22c03$ ./install-vpp-native/dpdk/sbin/dpdk-devbind -sNetwork devices using DPDK-compatible driver
============================================
<none>Network devices using kernel driver
===================================
0000:03:00.0 'Ethernet Controller 10-Gigabit X540-AT2' if=enp3s0f0 drv=ixgbe unused=vfio-pci,uio_pci_generic *Active*
0000:03:00.1 'Ethernet Controller 10-Gigabit X540-AT2' if=enp3s0f1 drv=ixgbe unused=vfio-pci,uio_pci_generic *Active*
0000:82:00.0 'Ethernet Controller XL710 for 40GbE QSFP+' if=ens802f0d1,ens802f0 drv=i40e unused=uio_pci_generic
0000:82:00.1 'Ethernet Controller XL710 for 40GbE QSFP+' if=ens802f1d1,ens802f1 drv=i40e unused=uio_pci_generic                        Other network devices
=====================
<none>csp2s22c03$ sudo modprobe uio_pci_generic
csp2s22c03$ sudo ./install-vpp-native/dpdk/sbin/dpdk-devbind --bind uio_pci_generic 82:00.0
csp2s22c03$ sudo ./install-vpp-native/dpdk/sbin/dpdk-devbind --bind uio_pci_generic 82:00.1csp2s22c03$ sudo ./install-vpp-native/dpdk/sbin/dpdk-devbind -sNetwork devices using DPDK-compatible driver
============================================
0000:82:00.0 'Ethernet Controller XL710 for 40GbE QSFP+' drv=uio_pci_generic unused=i40e,vfio-pci
0000:82:00.1 'Ethernet Controller XL710 for 40GbE QSFP+' drv=uio_pci_generic unused=i40e,vfio-pciNetwork devices using kernel driver
===================================
0000:03:00.0 'Ethernet Controller 10-Gigabit X540-AT2' if=enp3s0f0 drv=ixgbe unused=vfio-pci,uio_pci_generic *Active*
0000:03:00.1 'Ethernet Controller 10-Gigabit X540-AT2' if=enp3s0f1 drv=ixgbe unused=vfio-pci,uio_pci_generic *Active*

Start the VPP service, and verify that VPP is running:

csp2s22c03$ sudo service vpp start
csp2s22c03$ ps -ef | grep vpp
root     105655      1 98 17:34 ?        00:00:02 /usr/bin/vpp -c /etc/vpp/startup.conf
:w105675 105512  0 17:34 pts/4    00:00:00 grep --color=auto vpp

To access the VPP CLI, issue the command sudo vppctl . From the VPP interface, list all interfaces that are bound to DPDK using the command show interface:

VPP shows that the two 40-Gbps ports located at 82:0:0 and 82:0:1 are bound. Next, you need to assign IP addresses to those interfaces, bring them up, and verify:

vpp# set interface ip address FortyGigabitEthernet82/0/0 10.10.1.1/24
vpp# set interface ip address FortyGigabitEthernet82/0/1 10.10.2.1/24
vpp# set interface state FortyGigabitEthernet82/0/0 up
vpp# set interface state FortyGigabitEthernet82/0/1 up
vpp# show interface address
FortyGigabitEthernet82/0/0 (up):10.10.1.1/24
FortyGigabitEthernet82/0/1 (up):10.10.2.1/24
local0 (dn):

At this point VPP is operational. You can ping these interfaces either from net2s22c05 or csp2s22c04. Moreover, VPP can forward packets whose IP address are 10.10.1.0/24 and 10.10.2.0/24, so you can ping between net2s22c05 and csp2s22c04. Also, you can run iperf3 as illustrated in the previous example, and the result from running iperf3 between net2s22c05 and csp2s22c04 increases to 20.3 Gbits per second.

ET2S22C05$ iperf3 -c 10.10.1.2
Connecting to host 10.10.1.2, port 5201
[  4] local 10.10.2.2 port 54078 connected to 10.10.1.2 port 5201
[ ID] Interval           Transfer     Bandwidth       Retr  Cwnd
[  4]   0.00-1.00   sec  2.02 GBytes  17.4 Gbits/sec  460   1.01 MBytes
[  4]   1.00-2.00   sec  3.28 GBytes  28.2 Gbits/sec    0   1.53 MBytes
[  4]   2.00-3.00   sec  2.38 GBytes  20.4 Gbits/sec  486    693 KBytes
[  4]   3.00-4.00   sec  2.06 GBytes  17.7 Gbits/sec  1099   816 KBytes
[  4]   4.00-5.00   sec  2.07 GBytes  17.8 Gbits/sec  614   1.04 MBytes
[  4]   5.00-6.00   sec  2.25 GBytes  19.3 Gbits/sec  2869   716 KBytes
[  4]   6.00-7.00   sec  2.26 GBytes  19.4 Gbits/sec  3321   683 KBytes
[  4]   7.00-8.00   sec  2.33 GBytes  20.0 Gbits/sec  2322   594 KBytes
[  4]   8.00-9.00   sec  2.28 GBytes  19.6 Gbits/sec  1690  1.23 MBytes
[  4]   9.00-10.00  sec  2.73 GBytes  23.5 Gbits/sec  573    680 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bandwidth       Retr
[  4]   0.00-10.00  sec  23.7 GBytes  20.3 Gbits/sec  13434             sender
[  4]   0.00-10.00  sec  23.7 GBytes  20.3 Gbits/sec                  receiveriperf Done.

The VCC CLI command show run displays the graph runtime statistics. Observe that the average vector per node is 6.76, which means on average, a vector of 6.76 packets is handled in a graph node.

Example 3. Using VPP with the TREX* Realistic Traffic Generator

In this example we use only two systems, csp2s22c03 and net2s22c05, to run the TRex Realistic Traffic Generator. VPP is installed in csp2s22c03 and run as a packet forwarding engine. On net2s22c05, TRex is used to generate both client and server-side traffic. TRex is a high-performance traffic generator. It leverages DPDK to run in user space. Figure 2 illustrates this configuration.

VPP is set up on csp2s22c03 exactly as it was in Example 2. Only the setup on net2s22c05 is modified slightly to run TRex preconfigured traffic files.

Figure 2 – The TRex traffic generator sends packages to the host that has VPP running.

To install TRex, in net2s22c05, download and extract TRex package:

NET2S22C05$ wget --no-cache http://trex-tgn.cisco.com/trex/release/latest
NET2S22C05$ tar -xzvf latest
NET2S22C05$ cd v2.37
NET2S22C05$ sudo ./dpdk_nic_bind.py -sNetwork devices using DPDK-compatible driver
============================================
0000:87:00.0 'Ethernet Controller XL710 for 40GbE QSFP+' drv=vfio-pci unused=i40e
0000:87:00.1 'Ethernet Controller XL710 for 40GbE QSFP+' drv=vfio-pci unused=i40eNetwork devices using kernel driver
===================================
0000:03:00.0 'Ethernet Controller 10-Gigabit X540-AT2' if=enp3s0f0 drv=ixgbe unused=vfio-pci *Active*
0000:03:00.1 'Ethernet Controller 10-Gigabit X540-AT2' if=enp3s0f1 drv=ixgbe unused=vfio-pci
0000:81:00.0 '82599 10 Gigabit TN Network Connection' if=ens787f0 drv=ixgbe unused=vfio-pci
0000:81:00.1 '82599 10 Gigabit TN Network Connection' if=ens787f1 drv=ixgbe unused=vfio-pciOther network devices
=====================
<none>

Create the /etc/trex_cfg.yaml configuration file. In this configuration file, the port should match the interfaces available in the target system, which is net2s22c05 in our example. The IP addresses correspond to Figure 2. For more information on the configuration file, please refer to the TRex Manual.

NET2S22C05$ cat /etc/trex_cfg.yaml
### Config file generated by dpdk_setup_ports.py ###
- port_limit: 2version: 2interfaces: ['87:00.0', '87:00.1']port_bandwidth_gb: 40port_info:- ip: 10.10.2.2default_gw: 10.10.2.1- ip: 10.10.1.2default_gw: 10.10.1.1platform:master_thread_id: 0latency_thread_id: 1dual_if:- socket: 1threads: [22,23,24,25,26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43]

Stop the previous VPP session and start it again in order to add a route for new IP addresses 16.0.0.0/8 and 48.0.0.0/8, according to Figure 2. Those IP addresses are needed because TRex generates packets that use these addresses. Refer to the TRex Manual for details on these traffic templates.

csp2s22c03$ sudo service vpp stop
csp2s22c03$ sudo service vpp start
csp2s22c03$ sudo vppctl
_______    _        _   _____  ___
__/ __/ _ \  (_)__    | | / / _ \/ _ \
_/ _// // / / / _ \   | |/ / ___/ ___/
/_/ /____(_)_/\___/   |___/_/  /_/vpp# sho intName               Idx       State          Counter          Count
FortyGigabitEthernet82/0/0        1        down
FortyGigabitEthernet82/0/1        2        down
local0                            0        downvpp#
vpp# set interface ip address FortyGigabitEthernet82/0/0 10.10.1.1/24
vpp# set interface ip address FortyGigabitEthernet82/0/1 10.10.2.1/24
vpp# set interface state FortyGigabitEthernet82/0/0 up
vpp# set interface state FortyGigabitEthernet82/0/1 up
vpp# ip route add 16.0.0.0/8 via 10.10.1.2
vpp# ip route add 48.0.0.0/8 via 10.10.2.2
vpp# clear run

Now, you can generate a simple traffic flow from net2s22c05 using the traffic configuration file cap2/dns.yaml:

NET2S22C05$ sudo ./t-rex-64 -f cap2/dns.yaml -d 1 -l 1000summary stats--------------Total-pkt-drop       : 0 pktsTotal-tx-bytes       : 166886 bytesTotal-tx-sw-bytes    : 166716 bytesTotal-rx-bytes       : 166886 byteTotal-tx-pkt         : 2528 pktsTotal-rx-pkt         : 2528 pktsTotal-sw-tx-pkt      : 2526 pktsTotal-sw-err         : 0 pktsTotal ARP sent       : 4 pktsTotal ARP received   : 2 pktsmaximum-latency   : 35 usecaverage-latency   : 8 useclatency-any-error : OK

On csp2s22c03, the VCC CLI command show run displays the graph runtime statistics:

Example 4: Using VPP with TRex Mixed Traffic Templates

In this example, a more complicated traffic with delay profile on net2s22c05 is generated using the traffic configuration file avl/sfr_delay_10_1g.yaml:

NET2S22C05$ sudo ./t-rex-64 -f avl/sfr_delay_10_1g.yaml -c 2 -m 20 -d 100 -l 1000
summary stats--------------Total-pkt-drop       : 43309 pktsTotal-tx-bytes       : 251062132504 bytesTotal-tx-sw-bytes    : 21426636 bytesTotal-rx-bytes       : 251040139922 byteTotal-tx-pkt         : 430598064 pktsTotal-rx-pkt         : 430554755 pktsTotal-sw-tx-pkt      : 324646 pktsTotal-sw-err         : 0 pktsTotal ARP sent       : 5 pktsTotal ARP received   : 4 pktsmaximum-latency   : 1278 usecaverage-latency   : 9 useclatency-any-error : ERROR

On csp2s22c03, use the VCC CLI command show run to display the graph runtime statistics. Observe that the average vector per node is 10.69 and 14.47:

Summary

This tutorial showed how to download, compile, and install the VPP binary on an Intel® Architecture platform. Examples of /etc/sysctl.d/80-vpp.conf and /etc/vpp/startup.conf/startup.conf configuration files were provided to get the user up and running with VPP. The tutorial also illustrated how to detect and bind the network interfaces to a DPDK-compatible driver. You can use the VPP CLI to assign IP addresses to these interfaces and bring them up. Finally, four examples using iperf3 and TRex were included, to show how VPP processes packets in batches.

About the Author

Loc Q Nguyen received an MBA from University of Dallas, a master’s degree in Electrical Engineering from McGill University, and a bachelor's degree in Electrical Engineering from École Polytechnique de Montréal. He is currently a software engineer with Intel Corporation's Software and Services Group. His areas of interest include computer networking, parallel computing, and computer graphics.

References

Installing VPP binaries from packages
FDIO Quick Start Guide
VPP Command-line Interface (CLI) Guide
DPDK Documentation
TRex Stateful Manual and Getting Started Manual User Guide
TRex Documentation

Product and Performance Information

Intel's compilers may or may not optimize to the same degree for non-Intel microprocessors for optimizations that are not unique to Intel microprocessors. These optimizations include SSE2, SSE3, and SSSE3 instruction sets and other optimizations. Intel does not guarantee the availability, functionality, or effectiveness of any optimization on microprocessors not manufactured by Intel. Microprocessor-dependent optimizations in this product are intended for use with Intel microprocessors. Certain optimizations not specific to Intel microarchitecture are reserved for Intel microprocessors. Please refer to the applicable product User and Reference Guides for more information regarding the specific instruction sets covered by this notice.

产品和性能信息

对于非Intel微处理器而言，Intel的编译器可能会或可能不会针对非Intel微处理器进行相同程度的优化。这些优化包括SSE2，SSE3和SSSE3指令集以及其他优化。英特尔不保证非英特尔制造的微处理器的任何优化的可用性，功能或有效性。该产品中与微处理器有关的优化旨在与Intel微处理器一起使用。不特定于英特尔微体系结构的某些优化保留给英特尔微处理器。有关本通知涵盖的特定指令集的更多信息，请参阅适用的产品《用户和参考指南》。

Notice revision #20110804

在英特尔架构服务器上构建基于矢量包处理（VPP）的快速网络协议栈相关推荐

在英特尔® 架构平台上开发和优化基于 NDK 的 Android 游戏应用
作者:杜伟 Android 原生开发套件 (NDK) 是 Android SDK 的附带工具.借助该工具,您可以使用诸如 C 和 C++ 等本地代码语言实现部分应用. 您可以从以下网址下载该 NDK ...
在英特尔架构 Android* 设备上 HTML5 应用中创建劲酷动画与切换
作者:Tao Wang (Intel) 平板电脑和智能手机等移动设备日益普及,Adobe Flash* 插件需要被替代. 最近 Adobe 宣称 Flash 不再支持移动设备. 因此 HTML5 是未 ...
英特尔架构助力大数据 • 智算一体机方案释放“数智”潜能
在数据爆发式增长的今天,越来越多的组织开始构建大数据与人工智能 (AI) 平台,以期在强化海量数据处理能力的同时,赋能创新技术应用,推动数字化转型战略实施. 然而在构建大数据与 AI 平台的同时,复杂 ...
英特尔下一代服务器cpu消息,向10nm转型英特尔下一代CPU与GPU前瞻
2018英特尔架构日上,英特尔毫不吝啬通过5个小时的展示解开了2021年以前的芯片规划.下一代CPU.显卡.封装技术纷纷浮出水面.而第一波消费级新品会在2019年逐步上市,这一次,英特尔没有打算给我们 ...
关于英特尔® 以太网服务器适配器中 SR-IOV 的常见问题解答
英特尔数据中心解决方案关于英特尔® 以太网服务器适配器中 SR-IOV 的常见问题解答关于英特尔® 以太网适配器中单根 I/O 虚拟化 (SR-IOV) 的常见问题 (FAQ),您可以在此页面找到 ...
在英特尔独立显卡上训练ResNet PyTorch模型
作者:武卓,张晶目录 1.1 英特尔锐炫™独立显卡简介 1.2 蝰蛇峡谷简介 1.3 搭建训练PyTorch模型的开发环境 1.3.1 Windows 版本要求: 1.3.2 下载并安装最新的英特尔 ...
在英特尔独立显卡上训练TensorFlow模型
作者:武卓,张晶目录 1.1 英特尔锐炫™独立显卡简介 1.2 蝰蛇峡谷简介 1.3 搭建训练TensorFlow模型的开发环境 1.3.1 Windows 版本要求: 1.3.2 下载并安装最新的 ...
intel英特尔架构
英特尔架构 http://baike.baidu.com/view/2777794.htm
在英特尔独立显卡上部署YOLOv5 v7.0版实时实例分割模型
作者:贾志刚英特尔物联网创新大使目录 1.1 YOLOv5实时实例分割模型简介 1.2 英特尔®消费级锐炫™ A 系列显卡简介 1.3 在英特尔独立显卡上部署YOLOv5-seg模型的完整流程 ...

在英特尔架构服务器上构建基于矢量包处理（VPP）的快速网络协议栈

Build a Fast Network Stack with Vector Packet Processing (VPP) on an Intel® Architecture Server

Introduction

Vector Packet Processing (VPP) Overview

Build and Install VPP

Build the FD.io VPP Binary

Build the Debian* VPP Packages

Configure VPP

Configure huge pages

Configure startup.conf

Run VPP as a Packet Processing Engine

Example 1: Using Kernel Packet Forwarding with iperf3*

Example 2: Using VPP with iperf3

Example 3. Using VPP with the TREX* Realistic Traffic Generator

Example 4: Using VPP with TRex Mixed Traffic Templates

Summary

About the Author

References

产品和性能信息

在英特尔架构服务器上构建基于矢量包处理（VPP）的快速网络协议栈相关推荐

最新文章

热门文章