vpp的网卡直通配置和SRIOV配置方法
本文总结了Centos-KVM作为宿主机，centos-vpp作为客户机时的网卡passthrough和SRIOV的配置方法
配置网卡直通
一、宿主机上修改GRUB，增加intel_iommu=on。刷新GRUB。然后重启宿主机。
[root@kvm-02 ~]# vi /etc/default/grub
[root@kvm-02 ~]# grub2-mkconfig -o /boot/efi/EFI/centos/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-693.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-693.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-20fe28cd4f4b4fa1b140c6a72d10ae05
Found initrd image: /boot/initramfs-0-rescue-20fe28cd4f4b4fa1b140c6a72d10ae05.im
g
done
（注：刷新grub的方法与系统引导方式有关，如果是legacy模式，就是/boot/grub2/grub.cfg，如果是uefi模式，则为/boot/efi/EFI/centos/grub.cfg）
[root@kvm-02 ~]# reboot
等启动完了之后
[root@kvm-02 ~]# cat /proc/cmdline
BOOT_IMAGE=/vmlinuz-3.10.0-693.el7.x86_64 root=/dev/mapper/centos-root ro crashkernel=auto rd.lvm.lv=centos/root rd.lvm.lv=centos/swap intel_iommu=on isolcpus=20-23 nohz_full=20-23 rcu_nocbs=20-23 nmi_watchdog=0 selinux=0 intel_pstate=disable nosoftlockup rhgb quiet
[root@kvm-02 ~]#
二、通过virsh命令将PCI从宿主机分离
[root@kvm-02 ~]# lspci -nn | grep net
09:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 10GbE SFP+ [8086:37d3] (rev 09)
09:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 10GbE SFP+ [8086:37d3] (rev 09)
09:00.2 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 10GbE SFP+ [8086:37d3] (rev 09)
09:00.3 Ethernet controller [0200]: Intel Corporation Ethernet Connection X722 for 10GbE SFP+ [8086:37d3] (rev 09)
2f:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 01)
2f:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 01)
31:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller XXV710 for 25GbE SFP28 [8086:158b] (rev 02)
31:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller XXV710 for 25GbE SFP28 [8086:158b] (rev 02)
58:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 01)
58:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 01)
86:00.0 Ethernet controller [0200]: Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe [14e4:165f]
86:00.1 Ethernet controller [0200]: Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe [14e4:165f]
af:00.0 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 01)
af:00.1 Ethernet controller [0200]: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ [8086:1572] (rev 01)
virsh nodedev认的PCI格式稍微有点区别，用virsh再显示一遍
[root@kvm-02 ~]# virsh nodedev-list --tree | grep 09
| +- pci_0000_09_00_0
| +- pci_0000_09_00_1
| +- pci_0000_09_00_2
| +- pci_0000_09_00_3
+- pci_0000_05_09_0
+- pci_0000_05_09_1
+- pci_0000_05_09_2
+- pci_0000_05_09_3
+- pci_0000_05_09_4
+- pci_0000_05_09_5
+- pci_0000_05_09_6
+- pci_0000_05_09_7
+- pci_0000_2e_09_0
| | +- block_sdc_MTFDDAK480TBY_1AR1ZA_01PE061D7A09450LEN_1CC00A37
| | +- block_sdd_MTFDDAK480TBY_1AR1ZA_01PE061D7A09450LEN_1CFD6740
+- pci_0000_85_09_0
+- pci_0000_85_09_1
+- pci_0000_85_09_2
+- pci_0000_85_09_3
+- pci_0000_85_09_4
+- pci_0000_85_09_5
+- pci_0000_85_09_6
+- pci_0000_85_09_7
+- pci_0000_ae_09_0
将设备从宿主机分离出来
[root@kvm-02 ~]# virsh nodedev-dettach pci_0000_09_00_0
已分离设备 pci_0000_09_00_0
[root@kvm-02 ~]# virsh nodedev-dettach pci_0000_09_00_1
已分离设备 pci_0000_09_00_1
三、通过virsh edit修改虚拟机的xml配置，往虚拟机挂载PCI; 修改完虚拟机配置文件后,运行虚拟机.
比如virsh edit vm115_vnf，在devices里新增hostdev配置（注意bus slot function的编号是和pci编号对应着来的）
<devices>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x000' bus='0x09' slot='0x00' function='0x0' />
</source>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x000' bus='0x09' slot='0x00' function='0x1' />
</source>
</hostdev>
</devices>
修改完后保存退出，再 virsh start vm115_vnf
四、虚拟机里lspci可以看到新挂载的PCI，给新的PCI增加驱动后，vpp即可显示出相应的接口
如下：00:0d.0和00:12.0是新挂上的PCI
[root@vnf1-0 ~]# lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:02.0 VGA compatible controller: Cirrus Logic GD 5446
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:04.0 Ethernet controller: Red Hat, Inc Virtio network device
00:05.0 Ethernet controller: Red Hat, Inc Virtio network device
00:06.0 Ethernet controller: Red Hat, Inc Virtio network device
00:07.0 Ethernet controller: Red Hat, Inc Virtio network device
00:08.0 Ethernet controller: Red Hat, Inc Virtio network device
00:09.0 Ethernet controller: Red Hat, Inc Virtio network device
00:0a.0 Ethernet controller: Red Hat, Inc Virtio network device
00:0b.0 Ethernet controller: Red Hat, Inc Virtio network device
00:0c.0 Ethernet controller: Red Hat, Inc Virtio network device
00:0d.0 Ethernet controller: Intel Corporation Device 37d3 (rev 09)
00:0e.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 03)
00:0e.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 03)
00:0e.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 03)
00:0e.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 03)
00:0f.0 Communication controller: Red Hat, Inc Virtio console
00:10.0 SCSI storage controller: Red Hat, Inc Virtio block device
00:11.0 Unclassified device [00ff]: Red Hat, Inc Virtio memory balloon
00:12.0 Ethernet controller: Intel Corporation Device 37d3 (rev 09)
给新挂的PCI增加igb_uio驱动后，dpdk可以识别出接口。从而vpp就能看到接口了
modprobe uio
insmod /home/dpdk-stable-18.02.2/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
/home/dpdk-stable-18.02.2/usertools/dpdk-devbind.py --bind=igb_uio 0000:00:12.0 0000:00:0d.0 0000:00:0c.0 0000:00:0b.0 0000:00:0a.0 0000:00:09.0 0000:00:08.0 0000:00:07.0 0000:00:06.0 0000:00:05.0 0000:00:04.0
五、virsh nodedev-reattach pci_0000_09_00_0命令可以重新加载被分离的PCI
[root@kvm-02 net]# virsh nodedev-reattach pci_0000_09_00_0
重新附加设备 pci_0000_09_00_0
[root@kvm-02 net]# virsh nodedev-reattach pci_0000_09_00_1
重新附加设备 pci_0000_09_00_1
配置SRIOV
1、linux的引导参数里使能intel_iommu
intel_iommu=on
2、生成VF
linux内核3.8以下的和3.8以上的有区别。3.8以下版本先卸载驱动模块，再重新拉起驱动模块并附加max_vfs参数，网上较多这种配置方法指导。3.8以上的如下：
先查看内核版本
[root@kvm-02 rc.d]# cat /proc/version
Linux version 3.10.0-862.11.6.el7.x86_64 (builder@kbuilder.dev.centos.org) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC) ) #1 SMP Tue Aug 14 21:49:04 UTC 2018
[root@kvm-02 rc.d]#
通过ifconfig查看想生成VF的网卡名。
或者直接查看device文件
[root@kvm-02 ~]# cd /sys/class/net
[root@kvm-02 net]# ls
br0 br1 br10 br11 br12 br2 br3 br4 br5 br6 br7 br8 br9 eno1 eno2 eno3 eno4 enp0s20f0u1u6 enp134s0f0 enp134s0f1 enp175s0f0 enp175s0f1 enp47s0f0 enp47s0f1 enp49s0f0 enp49s0f1 enp88s0f0 enp88s0f1 lo virbr0 virbr0-nic
我需要使用的是网卡eno1和eno2， （4代表要生成4个VF，device最大支持多少VF可通过命令查看： cat /sys/class/net/device name/device/sriov_totalvfs）
echo 4 > /sys/class/net/eno1/device/sriov_numvfs
echo 4 > /sys/class/net/eno2/device/sriov_numvfs
查看pci，可以看到生成的VF（virtual function）
[root@kvm-02 rc.d]# lspci | grep Eth
09:00.0 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GbE SFP+ (rev 09)
09:00.1 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GbE SFP+ (rev 09)
09:00.2 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GbE SFP+ (rev 09)
09:00.3 Ethernet controller: Intel Corporation Ethernet Connection X722 for 10GbE SFP+ (rev 09)
09:02.0 Ethernet controller: Intel Corporation X722 Virtual Function (rev 09)
09:02.1 Ethernet controller: Intel Corporation X722 Virtual Function (rev 09)
09:02.2 Ethernet controller: Intel Corporation X722 Virtual Function (rev 09)
09:02.3 Ethernet controller: Intel Corporation X722 Virtual Function (rev 09)
09:06.0 Ethernet controller: Intel Corporation X722 Virtual Function (rev 09)
09:06.1 Ethernet controller: Intel Corporation X722 Virtual Function (rev 09)
09:06.2 Ethernet controller: Intel Corporation X722 Virtual Function (rev 09)
09:06.3 Ethernet controller: Intel Corporation X722 Virtual Function (rev 09)
2f:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01)
2f:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01)
31:00.0 Ethernet controller: Intel Corporation Ethernet Controller XXV710 for 25GbE SFP28 (rev 02)
31:00.1 Ethernet controller: Intel Corporation Ethernet Controller XXV710 for 25GbE SFP28 (rev 02)
58:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01)
58:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01)
86:00.0 Ethernet controller: Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe
86:00.1 Ethernet controller: Broadcom Limited NetXtreme BCM5720 Gigabit Ethernet PCIe
af:00.0 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01)
af:00.1 Ethernet controller: Intel Corporation Ethernet Controller X710 for 10GbE SFP+ (rev 01)
[root@kvm-02 rc.d]#
三、HOST上给VF配置mac地址
[root@kvm-02 /]# ip link set eno1 vf 0 mac 00:A0:00:00:01:00
[root@kvm-02 /]# ip link set eno2 vf 0 mac 00:A0:00:00:02:00
[root@kvm-02 /]# ip link show eno1
3: eno1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master br1 portid 7cd30a5bdb98 state DOWN mode DEFAULT qlen 1000
link/ether 7c:d3:0a:5b:db:98 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 00:a0:00:00:01:00, spoof checking on, link-state auto, trust off
vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 3 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
[root@kvm-02 /]# ip link show eno2
5: eno2: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq master br2 portid 7cd30a5bdb99 state DOWN mode DEFAULT qlen 1000
link/ether 7c:d3:0a:5b:db:99 brd ff:ff:ff:ff:ff:ff
vf 0 MAC 00:a0:00:00:02:00, spoof checking on, link-state auto, trust off
vf 1 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 2 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
vf 3 MAC 00:00:00:00:00:00, spoof checking on, link-state auto, trust off
四、将生成的VF从HOST分离出来
先查看有哪些PCI
[root@kvm-02 rc.d]# virsh nodedev-list --tree | grep 09
| +- pci_0000_09_00_0
| +- pci_0000_09_00_1
| +- pci_0000_09_00_2
| +- pci_0000_09_00_3
| +- pci_0000_09_02_0
| +- pci_0000_09_02_1
| +- pci_0000_09_02_2
| +- pci_0000_09_02_3
| +- pci_0000_09_06_0
| +- pci_0000_09_06_1
| +- pci_0000_09_06_2
| +- pci_0000_09_06_3
+- pci_0000_05_09_0
+- pci_0000_05_09_1
+- pci_0000_05_09_2
+- pci_0000_05_09_3
+- pci_0000_05_09_4
+- pci_0000_05_09_5
+- pci_0000_05_09_6
+- pci_0000_05_09_7
+- pci_0000_2e_09_0
| | +- block_sdc_MTFDDAK480TBY_1AR1ZA_01PE061D7A09450LEN_1CC00A37
| | +- block_sdd_MTFDDAK480TBY_1AR1ZA_01PE061D7A09450LEN_1CFD6740
+- pci_0000_85_09_0
+- pci_0000_85_09_1
+- pci_0000_85_09_2
+- pci_0000_85_09_3
+- pci_0000_85_09_4
+- pci_0000_85_09_5
+- pci_0000_85_09_6
+- pci_0000_85_09_7
+- pci_0000_ae_09_0
[root@kvm-02 rc.d]# virsh nodedev-dettach pci_0000_09_02_0
已分离设备 pci_0000_09_02_0
[root@kvm-02 rc.d]# virsh nodedev-dettach pci_0000_09_06_0
已分离设备 pci_0000_09_06_0
五、将分离出的VF 加入到虚拟机里去，通过virsh edit命令给虚拟机的增加hostdev配置
<devices>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x000' bus='0x09' slot='0x02' function='0x0' />
</source>
</hostdev>
<hostdev mode='subsystem' type='pci' managed='yes'>
<source>
<address domain='0x000' bus='0x09' slot='0x06' function='0x0' />
</source>
</hostdev>
</devices>
六、启动虚拟机，在虚拟机中查看新生成的PCI
[root@vnf1-0 ~]# lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 03)
00:02.0 VGA compatible controller: Cirrus Logic GD 5446
00:03.0 Ethernet controller: Red Hat, Inc Virtio network device
00:04.0 Ethernet controller: Red Hat, Inc Virtio network device
00:05.0 Ethernet controller: Red Hat, Inc Virtio network device
00:06.0 Ethernet controller: Red Hat, Inc Virtio network device
00:07.0 Ethernet controller: Red Hat, Inc Virtio network device
00:08.0 Ethernet controller: Red Hat, Inc Virtio network device
00:09.0 Ethernet controller: Red Hat, Inc Virtio network device
00:0a.0 Ethernet controller: Red Hat, Inc Virtio network device
00:0b.0 Ethernet controller: Red Hat, Inc Virtio network device
00:0c.0 Ethernet controller: Red Hat, Inc Virtio network device
00:0d.0 Ethernet controller: Intel Corporation Device 37cd (rev 09)
00:0e.0 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #1 (rev 03)
00:0e.1 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #2 (rev 03)
00:0e.2 USB controller: Intel Corporation 82801I (ICH9 Family) USB UHCI Controller #3 (rev 03)
00:0e.7 USB controller: Intel Corporation 82801I (ICH9 Family) USB2 EHCI Controller #1 (rev 03)
00:0f.0 Communication controller: Red Hat, Inc Virtio console
00:10.0 SCSI storage controller: Red Hat, Inc Virtio block device
00:11.0 Unclassified device [00ff]: Red Hat, Inc Virtio memory balloon
00:12.0 Ethernet controller: Intel Corporation Device 37cd (rev 09)
[root@vnf1-0 ~]#
七、给新的PCI增加driver，让vpp能识别然后生成interface
modprobe uio
insmod /home/dpdk-stable-18.02.2/x86_64-native-linuxapp-gcc/kmod/igb_uio.ko
/home/dpdk-stable-18.02.2/usertools/dpdk-devbind.py --bind=igb_uio 0000:00:0d.0 0000:00:12.0
重启vpp后，进入vpp的CLI，通过show interface可以查看到生成的VF接口。
[root@vnf1-0 ~]# systemctl restart vpp
[root@vnf1-0 ~]# vppctl
vpp# show interface
Name Idx State Counter Count
VirtualFunctionEthernet0/12/0 2 down
VirtualFunctionEthernet0/d/0 1 down
local0 0 down
vpp#
然后就可以利用VF开始进行愉快的测试工作啦！
附，问题：
为了使永久生效。可以将生成VF的配置写到rc.d里头去
[root@kvm-02 net]# cd /etc/rc.d
[root@kvm-02 rc.d]# touch /var/lock/subsys/local
[root@kvm-02 rc.d]# echo 4 > /sys/class/net/eno1/device/sriov_numvfs
[root@kvm-02 rc.d]# echo 4 > /sys/class/net/eno2/device/sriov_numvfs
但是试了没有用，HOST重启后仍然是没有了VF，得重新配置。
vpp的网卡直通配置和SRIOV配置方法相关推荐
- PCIe规范的扩展：SR-IOV（Single Root I/O virtual）网卡直通技术
目录 Single Root I/O虚拟化(SR-IOV)概述 SR-IOV架构 Hyper-V可扩展交换机简介 SR-IOV物理功能(PF) 编写SR-IOV PF微型端口驱动程序概述 SR-IOV ...
- CentOS之——双网卡双IP双网关配置(双网卡配置一个上外网一个接局域网)
转载请注明出处:http://blog.csdn.net/l1028386804/article/details/77487639 一.配置讲解 1.配置DNS 修改对应网卡的DNS的配置文件 # v ...
- Linux 配置网卡、主机名(基础配置、网卡会话配置、网卡绑定配置)
目录 配置网卡基本信息 通过nmcli命令配置网卡 通过配置网卡文件配置网卡 通过nmtui命令配置网卡 通过nm-connection-editor命令配置网卡 网卡高级配置 配置网络会话 配置网卡 ...
- Linux网卡绑定（bonding）配置
Linux网卡绑定(bonding)配置 目录 1 网卡绑定(Bonding)概述... 4 2 Linux下bonding配置... 6 2.1 建立bonding网卡.. 6 2.2 配置开机自动 ...
- esxi6.7下pci网卡直通无法生效的解决办法
如题,解决办法见截图,我是新手,今天配置网卡直通无法生效的问题找到解决办法了,分享出来. 经测试我的螃蟹卡可以直通给虚拟机黑裙了
- linux下配置ip地址的方法
(1)Ifconfig命令 第一种使用ifconfig命令配置网卡的ip地址.此命令通常用来零时的测试用,计算机启动后 ip地址的配置将自动失效.具体用法如下.Ipconfig ethx ipa ...
- vCenter6.0配置二：配置HA群集
本文主要介绍HA群集的创建,主要有以下要求:1.每台宿主机具有冗余的网络连接,因此最好要有2块物理网卡,前文在介绍配置分布式交换机时已经满足.2.每台宿主机具有冗余的共享存储,因此最好要添加2块共享磁 ...
- linux添加静态网关route文件,详解Linux系统中配置静态路由的方法
主机只有一块网卡的时候,不会涉及到默认路由和静态路由的问题.两块网卡时,也可以一块配置有网关(默认路由走这个网卡),另外一块只配置IP和掩码确认(该主机同网络的可以互通).不过在稍大型的.有多网段的环 ...
- linux 配置路由时via,详解Linux系统中配置静态路由的方法
主机只有一块网卡的时候,不会涉及到默认路由和静态路由的问题.两块网卡时,也可以一块配置有网关(默认路由走这个网卡),另外一块只配置IP和掩码确认(该主机同网络的可以互通).不过在稍大型的.有多网段的环 ...
最新文章
- BZOJ1491: [NOI2007]社交网络(Floyd 最短路计数)
- HDU 1036 Average is not Fast Enough!：题目解答源码
- ES failed shard on node[XXX]: failed recovery, failure RecoveryFailedException XXX Too many openfile
- 数据挖掘算法之离散化和二元化
- python slicer_报表筛选：Excel Slicer与SQL Server Reporting Services（SSRS）参数
- 在windows下安装node-sass失败，提示\node-sass: Command failed，解决方案
- @Valid 和 @Validated的使用
- 音视频即时通讯解决方案
- 一文读懂内生性问题之两阶段最小二乘法TSLS
- 什么叫wipe，安卓用户如何去wipe？
- 网站关键词优化工具，网站关键词排名优化工具免费
- 最小生成树的第三种求法-Borůvka (Sollin) 算法
- 数字逻辑---头歌实训作业---逻辑函数及其描述工具（Logisim）
- DBO与DB_OWER
- Java白盒测试三角形函数_白盒测试实验报告-三角形形状-山东大学
- Docker之介绍与安装
- 一个非常好的大数据学习网站
- [益智]：两人过桥问题
- python中row是什么意思_row
- 有哪些巧妙说服对方的说话技巧