FD.io VPP利用iperf3进行UDP灌包测试-英特尔X520万兆网卡
2024-05-20 06:33:26
FD.io VPP：用户文档 iperf3灌包测试
RToax 2020年9月
架构
1. VPP环境配置与启动
1.1. 安装VPP环境
略
1.2. VPP配置文件
启动配置文件startup-iperf3.conf
## Filename: startup-iperf3.conf
## Introduction: VPP start up configure file
## File History:
## 2020.09 rongtao New
##
unix { nodaemongid vpp cli-listen localhost:5002startup-config /usr/share/vpp/scripts/startup.txt}
session { evt_qs_memfd_seg }
socksvr { socket-name /tmp/vpp-api.sock}
api-trace { on }
cpu {main-core 1workers 2scheduler-policy fifoscheduler-priority 50thread-prefix vpp
}buffers {buffers-per-numa 128000
}dpdk {log-level debugdev 0000:3b:00.0 {name dpdk0}dev 0000:3b:00.1 {name dpdk1}no-multi-seg dev default {num-rx-queues 1num-tx-queues 1num-rx-desc 1024}# igb_uio, vfio-pci uio-driver igb_uio# socket-mem 2048,2048no-tx-checksum-offload
}plugins {## Adjusting the plugin path depending on where the VPP plugins arepath /opt/vpp-stable-2005/build-root/install-vpp-native/vpp/lib/vpp_plugins## Disable all plugins by default and then selectively enable specific plugins plugin default { disable }plugin dpdk_plugin.so { enable }plugin acl_plugin.so { enable }
}
1.3. VPP的初始化命令行脚本
为VPP自动化配置考虑，需要在VPP启动配置文件中配置startup-config参数，在上小节中的startup-config配置为/usr/share/vpp/scripts/startup.txt。
/usr/share/vpp/scripts/startup.txt文件如下：
comment {文件描述：该文件用于配置VPP网络，具体配置根据实际情况而定作者：荣涛时间：2020年9月2日网址：https://docs.fd.io/vpp/20.05/d5/d57/clicmd.html
}comment { 设置网卡IP地址 }
set interface ip address dpdk0 10.170.7.169/24
comment {
set interface ip address dpdk1 10.170.6.169/24
}comment { 设置网卡MAC地址 }
comment {
set interface mac address dpdk0 xx:xx:xx:xx:xx:xx
}comment { 设置MTU-最大传输单元 }
comment {
set interface mtu [packet|ip4|ip6|mpls] value interface
}comment { 设置接收模式 }
comment {
set interface rx-mode interface [queue n] [polling | interrupt | adaptive]
}
show interface rx-placementcomment { 设置队列worker }
comment {
set interface rx-placement interface queue 1 worker 0
}comment { 启动网卡 [up|down|punt|enable] }
set interface state dpdk0 up
set interface state dpdk1 upcomment { 设置路由，根据实际情况决定 }
ip route add 10.170.6.0/24 via 10.170.7.254comment { 开启混杂模式 }
set interface promiscuous on dpdk0comment { 启动local0口 }
comment {
set interface state local0 up
}comment { 创建环回地址 }
comment {
create loopback interface
set interface ip address loop0 127.0.0.1/8
set interface state loop0 up
}comment { DPDK相关配置、查询 }
comment {
show dpdk buffer
show dpdk physmem
show dpdk version
}comment { DPDK内存测试 }
comment {
test dpdk buffer allocate 1024
test dpdk buffer
test dpdk buffer free 1024
}comment { 显示有用信息 }
comment { show version }
comment { 查看所有插件 }
comment {
show plugins
}comment { 抓包 }
comment {
pcap trace status
pcap trace tx max 35 intfc dpdk0 file dpdk0.pcap
pcap trace status
pcap trace off
}1.4. VPP启动
启动很简单，我们给vpp传入配置文件startup-iperf3.conf
# vpp -c startup-iperf3.conf一下内容为输出打印，报错部分先忽略。
vpp[19861]: vlib_sort_init_exit_functions:160: order constraint fcn 'dns_init' not f
oundvpp[19861]: vnet_feature_arc_init:271: feature node 'nat44-in2out-output' not found
(before 'ip4-dvr-reinject', arc 'ip4-output')vpp[19861]: dpdk: EAL init args: -c e -n 4 --in-memory --log-level debug --file-pref
ix vpp -w 0000:3b:00.0 -w 0000:3b:00.1 --master-lcore 1 Thread 1 (vpp_wk_0):node dpdk-input:dpdk0 queue 0 (polling)
Thread 2 (vpp_wk_1):node dpdk-input:dpdk1 queue 0 (polling)
我们用telnet命令进入该vpp实例的命令行
# telnet 0 5002
Trying 0.0.0.0...
Connected to 0.
Escape character is '^]'._______ _ _ _____ ___ __/ __/ _ \ (_)__ | | / / _ \/ _ \_/ _// // / / / _ \ | |/ / ___/ ___//_/ /____(_)_/\___/ |___/_/ /_/ vpp#
查看网口信息
vpp# show interface Name Idx State MTU (L3/IP4/IP6/MPLS) Counter Count
dpdk0 1 up 9000/0/0/0 rx packets 342rx bytes 35543tx packets 3tx bytes 126drops 282punt 60ip4 18ip6 24
dpdk1 2 up 9000/0/0/0 rx packets 339rx bytes 35295drops 280punt 59ip4 16ip6 24
local0 0 down 0/0/0/0
vpp#
查看网口IP地址
vpp# show interface address
dpdk0 (up):L3 10.170.7.169/24
dpdk1 (up):
local0 (dn):
这样我们就确定了，10.170.7.169/24即为我们配置过的VPP网卡。
2. UDP灌包测试
2.1. ACL库预加载
在得知10.170.7.169/24即为我们配置过的VPP网卡后，首先配置LD_PRELOAD环境变量，将系统的posix替换为vpp的acl库。
LD_PRELOAD的作用：系统在运行过程中，会首先加载该环境变量指定的函数库（在libc.so之前加载），如果函数库内包含了程序中执行的函数名，该可执行文件的函数将被重定向到LD_PRELOAD指向的函数中。
export LD_PRELOAD=$VPP_ROOT/build-root/build-vpp-native/vpp/lib/libvcl_ldpreload.so
在10.170.7.166服务器上，我们的环境变量如下：
export LD_PRELOAD=/opt/vpp-stable-2005/build-root/build-vpp-native/vpp/lib/libvcl_ldpreload.so
echo $LD_PRELOAD
/opt/vpp-stable-2005/build-root/build-vpp-native/vpp/lib/libvcl_ldpreload.so
注意，当设定环境变量LD_PRELOAD后，该终端/bash下的所有指令，如有使用posix相关函数的，都将会被中定向到VPP的ACL库函数中。此时的socket相关系统指令可能执行失败，如下面的ping指令
ping 10.170.6.69
WARNING: your kernel is veeery old. No problems.
WARNING: setsockopt(IP_RECVTTL): 不支持的操作
WARNING: setsockopt(IP_RETOPTS): 不支持的操作
PING 10.170.6.69 (10.170.6.69) 56(84) bytes of data.
Warning: no SO_TIMESTAMP support, falling back to SIOCGSTAMP
ping: sendmsg: 传输端点尚未连接
ping: recvmsg: 函数未实现
^C
--- 10.170.6.69 ping statistics ---
1 packets transmitted, 0 received, 100% packet loss, time 0msvl_client_disconnect:323: queue drain: 5312.2. 启动iperf3服务端
在任一台服务器或许你中启动iperf3服务端，监听5201端口。端口可变
这里我进行了绑核操作，可以不绑。
iperf3 -s --bind 10.170.6.66 --affinity 1
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------
2.3. 在VPP下启动iperf3 UDP客户端
启动iperf3客户端
# iperf3 -c 10.170.6.66 --bind 10.170.7.169 -u
输出如下：
warning: Warning: UDP block size 1460 exceeds TCP MSS 0, may result in fragmentatio
n / dropsConnecting to host 10.170.6.66, port 5201
[ 33] local 10.170.7.169 port 58322 connected to 10.170.6.66 port 5201
[ ID] Interval Transfer Bandwidth Total Datagrams
[ 33] 0.00-1.00 sec 115 KBytes 946 Kbits/sec 81
[ 33] 1.00-2.00 sec 128 KBytes 1.05 Mbits/sec 90
[ 33] 2.00-3.00 sec 128 KBytes 1.05 Mbits/sec 90
[ 33] 3.00-4.00 sec 128 KBytes 1.05 Mbits/sec 90
[ 33] 4.00-5.00 sec 127 KBytes 1.04 Mbits/sec 89
[ 33] 5.00-6.00 sec 128 KBytes 1.05 Mbits/sec 90
[ 33] 6.00-7.00 sec 128 KBytes 1.05 Mbits/sec 90
[ 33] 7.00-8.00 sec 128 KBytes 1.05 Mbits/sec 90
[ 33] 8.00-9.00 sec 128 KBytes 1.05 Mbits/sec 90
[ 33] 9.00-10.00 sec 127 KBytes 1.04 Mbits/sec 89
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 33] 0.00-10.00 sec 1.24 MBytes 1.04 Mbits/sec 0.469 ms 0/889 (0%)
[ 33] Sent 889 datagramsiperf Done.
vl_client_disconnect:323: queue drain: 531
此时的服务端打印如下：
Accepted connection from 10.170.7.169, port 36991
[ 5] local 10.170.6.66 port 5201 connected to 10.170.7.169 port 58322
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-1.00 sec 117 KBytes 956 Kbits/sec 127.396 ms 0/82 (0%)
[ 5] 1.00-2.00 sec 128 KBytes 1.05 Mbits/sec 0.845 ms 0/90 (0%)
[ 5] 2.00-3.01 sec 128 KBytes 1.05 Mbits/sec 0.195 ms 0/90 (0%)
[ 5] 3.01-4.01 sec 128 KBytes 1.05 Mbits/sec 0.321 ms 0/90 (0%)
[ 5] 4.01-5.00 sec 125 KBytes 1.03 Mbits/sec 0.332 ms 0/88 (0%)
[ 5] 5.00-6.00 sec 130 KBytes 1.06 Mbits/sec 0.499 ms 0/91 (0%)
[ 5] 6.00-7.01 sec 128 KBytes 1.04 Mbits/sec 0.538 ms 0/90 (0%)
[ 5] 7.01-8.01 sec 128 KBytes 1.06 Mbits/sec 0.463 ms 0/90 (0%)
[ 5] 8.01-9.00 sec 128 KBytes 1.06 Mbits/sec 0.434 ms 0/90 (0%)
[ 5] 9.00-10.00 sec 125 KBytes 1.03 Mbits/sec 0.469 ms 0/88 (0%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.00 sec 1.24 MBytes 1.04 Mbits/sec 0.469 ms 0/889 (0%) receiver发现速率根本不对应，通过在网上查找资料，发现需要添加参数-b。相关链接：Iperf results on UDP Bandwidth
参数-b的说明如下
-b, --bandwidth #[KMG][/#] target bandwidth in bits/sec (0 for unlimited)(default 1 Mbit/sec for UDP, unlimited for TCP)(optional slash and packet count for burst mode)
再次测试：
# iperf3 -c 10.170.6.66 --bind 10.170.7.169 -b 1000M -u
输出如下：
warning: Warning: UDP block size 1460 exceeds TCP MSS 0, may result in fragmentatio
n / dropsConnecting to host 10.170.6.66, port 5201
[ 33] local 10.170.7.169 port 1748 connected to 10.170.6.66 port 5201
[ ID] Interval Transfer Bandwidth Total Datagrams
[ 33] 0.00-1.00 sec 109 MBytes 913 Mbits/sec 78148
[ 33] 1.00-2.00 sec 119 MBytes 1000 Mbits/sec 85588
[ 33] 2.00-3.00 sec 119 MBytes 1000 Mbits/sec 85601
[ 33] 3.00-4.00 sec 119 MBytes 1.00 Gbits/sec 85660
[ 33] 4.00-5.00 sec 119 MBytes 1.00 Gbits/sec 85620
[ 33] 5.00-6.00 sec 119 MBytes 999 Mbits/sec 85567
[ 33] 6.00-7.00 sec 119 MBytes 1.00 Gbits/sec 85669
[ 33] 7.00-8.00 sec 119 MBytes 1000 Mbits/sec 85574
[ 33] 8.00-9.00 sec 119 MBytes 1000 Mbits/sec 85603
[ 33] 9.00-10.00 sec 119 MBytes 1.00 Gbits/sec 85683
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 33] 0.00-10.00 sec 1.15 GBytes 991 Mbits/sec 0.060 ms 818675/846746 (97%) [ 33] Sent 846746 datagramsiperf Done.
vl_client_disconnect:323: queue drain: 531
加大带宽再次测试：
# iperf3 -c 10.170.6.66 --bind 10.170.7.169 -b 10000M -uwarning: Warning: UDP block size 1460 exceeds TCP MSS 0, may result in fragmentatio
n / dropsConnecting to host 10.170.6.66, port 5201
[ 33] local 10.170.7.169 port 32072 connected to 10.170.6.66 port 5201
[ ID] Interval Transfer Bandwidth Total Datagrams
[ 33] 0.00-1.00 sec 1.13 GBytes 9.71 Gbits/sec 831282
[ 33] 1.00-2.00 sec 1.18 GBytes 10.1 Gbits/sec 865276
[ 33] 2.00-3.00 sec 1.17 GBytes 10.0 Gbits/sec 859818
[ 33] 3.00-4.00 sec 1.16 GBytes 10.0 Gbits/sec 856510
[ 33] 4.00-5.00 sec 1.16 GBytes 9.96 Gbits/sec 852834
[ 33] 5.00-6.00 sec 1.15 GBytes 9.88 Gbits/sec 845942
[ 33] 6.00-7.00 sec 1.17 GBytes 10.1 Gbits/sec 860781
[ 33] 7.00-8.00 sec 1.16 GBytes 9.95 Gbits/sec 851835
[ 33] 8.00-9.00 sec 1.18 GBytes 10.1 Gbits/sec 865302
[ 33] 9.00-10.00 sec 1.17 GBytes 10.1 Gbits/sec 864098
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 33] 0.00-10.00 sec 11.6 GBytes 9.99 Gbits/sec 0.075 ms 8499899/8551442 (99%
) [ 33] Sent 8551442 datagramsiperf Done.
vl_client_disconnect:323: queue drain: 531索性将带宽增加至40000 Mbps：
# iperf3 -c 10.170.6.66 --bind 10.170.7.169 -b 40000M -u
测试结果如下：
warning: Warning: UDP block size 1460 exceeds TCP MSS 0, may result in fragmentatio
n / dropsConnecting to host 10.170.6.66, port 5201
[ 33] local 10.170.7.169 port 43051 connected to 10.170.6.66 port 5201
[ ID] Interval Transfer Bandwidth Total Datagrams
[ 33] 0.00-1.00 sec 1.99 GBytes 17.1 Gbits/sec 1460305
[ 33] 1.00-2.00 sec 1.99 GBytes 17.1 Gbits/sec 1461188
[ 33] 2.00-3.00 sec 1.99 GBytes 17.1 Gbits/sec 1461091
[ 33] 3.00-4.00 sec 1.99 GBytes 17.1 Gbits/sec 1461270
[ 33] 4.00-5.00 sec 1.99 GBytes 17.1 Gbits/sec 1461472
[ 33] 5.00-6.00 sec 1.99 GBytes 17.1 Gbits/sec 1463297
[ 33] 6.00-7.00 sec 1.99 GBytes 17.1 Gbits/sec 1463611
[ 33] 7.00-8.00 sec 1.99 GBytes 17.1 Gbits/sec 1463020
[ 33] 8.00-9.00 sec 1.99 GBytes 17.1 Gbits/sec 1463278
[ 33] 9.00-10.00 sec 1.99 GBytes 17.1 Gbits/sec 1462751
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Jitter Lost/Total Datagrams
[ 33] 0.00-10.00 sec 19.9 GBytes 17.1 Gbits/sec 0.061 ms 14553747/14612709 (1e+02%) [ 33] Sent 14612709 datagramsiperf Done.
vl_client_disconnect:323: queue drain: 531
此时服务端的丢包率非常大（还好我不用关心）：
Accepted connection from 10.170.7.169, port 34288
[ 5] local 10.170.6.66 port 5201 connected to 10.170.7.169 port 24719
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-1.00 sec 10.3 MBytes 86.3 Mbits/sec 1.032 ms 1432532/1439957 (99%)
[ 5] 1.00-2.01 sec 9.67 MBytes 80.7 Mbits/sec 0.948 ms 1478665/1485608 (1e+02%)
[ 5] 2.01-3.00 sec 9.59 MBytes 81.0 Mbits/sec 0.747 ms 1465574/1472460 (1e+02%)
[ 5] 3.00-4.01 sec 9.29 MBytes 77.3 Mbits/sec 1.048 ms 1476442/1483116 (1e+02%)
[ 5] 4.01-5.00 sec 9.54 MBytes 80.7 Mbits/sec 0.996 ms 1457845/1464696 (1e+02%)
[ 5] 5.00-6.01 sec 9.69 MBytes 80.9 Mbits/sec 0.895 ms 1478246/1485203 (1e+02%)
[ 5] 6.01-7.01 sec 8.66 MBytes 72.5 Mbits/sec 0.924 ms 1474369/1480588 (1e+02%)
[ 5] 7.01-8.01 sec 8.72 MBytes 73.3 Mbits/sec 0.771 ms 1467575/1473841 (1e+02%)
[ 5] 8.01-9.00 sec 8.89 MBytes 74.8 Mbits/sec 0.767 ms 1467219/1473604 (1e+02%)
[ 5] 9.00-10.01 sec 9.31 MBytes 77.9 Mbits/sec 0.980 ms 1472938/1479624 (1e+02%)
[ 5] 10.01-10.25 sec 301 KBytes 10.1 Mbits/sec 0.225 ms 37991/38202 (99%)
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Jitter Lost/Total Datagrams
[ 5] 0.00-10.25 sec 94.0 MBytes 76.9 Mbits/sec 0.225 ms 14709396/14776899 (1e+02%) receiver
-----------------------------------------------------------
Server listening on 5201
-----------------------------------------------------------3. 结论
我们采用VPP方案，使用英特尔X520网卡
# lspci | grep 10G
3b:00.0 Ethernet controller: Intel Corporation Ethernet 10G 2P X520 Adapter (rev 01)
3b:00.1 Ethernet controller: Intel Corporation Ethernet 10G 2P X520 Adapter (rev 01)
测出的发包峰值速率为17.1 Gbits/sec。
FD.io VPP利用iperf3进行UDP灌包测试-英特尔X520万兆网卡相关推荐
- FD.io VPP环境下运行用户应用程序教程
FD.io VPP环境下运行用户应用程序教程 RToax 2020年9月 相关文章:<FD.io VPP利用iperf3进行UDP灌包测试-英特尔X520万兆网卡> 1. VPP简介 VP ...
- FD.io VPP用户文档：会话层架构与VPP应用
FD.io VPP:用户文档 HostStack RToax 2020年9月 HostStack 1. 描述 VPP的主机协议栈是利用VPP传输,会话和应用程序层协议的用户空间实现.它大致包括四个主要 ...
- FD.io VPP 20.09版本正式发布：往期VPP文章回顾+下载地址+相关链接
目录 下载RPM/DEB包 往期文章回顾与推荐 FD.io是一些项目和库的集合,基于DPDK并逐渐演化,支持在通用硬件平台上部署灵活可变的业务.FD.io为软件定义基础设施的开发者提供了一个通用平台, ...
- FD.io VPP基本介绍：理解向量包处理（VPP）
FD.io VPP:用户文档 向量包处理器 RToax 2020年9月 1. 什么是向量包处理器(VPP) FD.io的矢量包处理器(VPP)是一个快速,可扩展的2-4层多平台网络协议栈.它在Linu ...
- FD.io/VPP — Overview
目录 文章目录 目录 FD.io VPP FD.io 官网:https://fd.io FD.io(Fast data – Input/Output)是 Linux 基金会下属的一个开源项目,成立于 ...
- 关于FD.io VPP的最新消息
Table of Contents CuVPP:软件数据平面中基于过滤器的最长前缀匹配 快速数据项目的矢量包处理器(VPP)版本20.05 2020年打破神话的DPDK 在商品硬件上使用Calico ...
- FD.io VPP：用户文档：VPP RPM包的构建与离线安装
VPP RPM包的构建与离线安装 RToax 2020年9月 要开始使用VPP进行开发,您需要获取所需的VPP源代码,然后构建软件包. 1. 设置代理 如果你的服务器本地可以连接互联网,则跳过设置代理 ...
- FD.io VPP 20.05 官方文档 总目录：用户文档+开发文档+命令行
https://docs.fd.io/vpp/20.05/index.html Vector Packet Processing FD.io VPP ▼Vector Packet Processing ...
- FD.io VPP startup.conf配置文件示例：安装后第一次配置
FD.io VPP配置文件详解:https://rtoax.blog.csdn.net/article/details/108056964 ## Filename: startup.conf ## I ...
最新文章
- ASSERT()是干什么用的
- 在C#中利用Keep-Alive处理Socket网络异常断开的方法
- git 删除已add的文件夹_如何删除还没有被add到git的文件（git clean的用法）
- [转]整理一些好的开源项目
- linux 长连接个数限制
- Assembly Essence-- 程序集深入探讨：程序集结构及部署
- STM32工作笔记0037---STM32时钟系统精讲-M3
- 微信小程序图标样式的引用
- acunetix导出html,关于Acunetix v11 WebUI下不得不说的事情！
- java计算机毕业设计基于安卓Android的团务智慧管理APP
- vue使用mavon-editor编辑器，外链保存到本地，且A标签外链和图片外链的处理
- dlna移植android系统,android系统的dlna开发
- 超链接一般有两种表现形式_超级链接有哪些常见的表现形式？
- Win11系统Windows.old能删除吗？Windows.old怎么删？
- vue+koa2移动电商实战
- 基于pytorch的花卉识别小程序
- Kong的插件：Request Size Limiting
- linux命令 ---rm
- NBU：1651 Red packet（二分）【好】
- 如何开启任务计划程序