tpproxy-tcp透明代理
tcp transparent proxy
Background
最近有个需求，需要在路由器设备中截获数据包，从而实现中转。按照下面的拓补，说明。我们需要在主机h2上截获h1发往h3的TCP协议包。最先实现的版本是基于tun设备，数据包截获之后，采用UDP协议中转，类似openvpn的方式。
h1----s1----h2------h3
采用tun的方式，需要配置h1的默认网关。但是我们的需求比较特别，我们对h1是没有修改权限的。之前师弟实现了一个版本，采用tap的方式，将h2-eth0同tap设备进行桥接。这种解决方案，参考博客[1]。
采用桥接的模式，确实不用修改h1中路由表配置。但是我们又进一步的需求，h2截获了h1发送的tcp数据包，h2能够回复ack报文给h1。如果采用桥接的模式，截获的是以太网帧，回复ack，就需要移植一个用户态TCP协议栈，也看了几天，相中了mtcp。但是，感觉太麻烦。麻烦的事情，坚持走下去，就需要大量的时间去填坑。我希望这事可以快点。我写代码的能力有限，摘抄别人的代码很厉害。比如今天写的demo，连抄带写，不到500行。
之后，就查到了tcp透明代理[3,4,5]，可以通过修改h2的路由表，截获TCP数据包，中转应用层数据。
但是，这种模式，需要在h1中指定h2位默认网关。之前测试的时候，在h1指定了h2作为网关，h1就能ping通h3。h1指定h3或者一个不存在的一个ip作为网关，h1 ping h3就失败。我一直在猜，h1在发送icmp包之前和h2是不是存在什么握手协议。无奈，本科时候计算机网络学的太差，实在不知道这里有个什么握手协议。在照本宣科的课堂上，能学到鬼的东西。当时学习的教材还是国内的一本经典教材，不过也是从坦南鲍姆那本书中抄的。特殊时代的特殊产物。在知乎上有个很调皮的问题：谭浩强是c++之父吗？ 有个回答很有意思：不是，谭是c+++++之父。学过他的教材的同学就能理解这个梗：请问 i+++++i，i最后的取值是什么？据说谭根本不会编程，但却写了一系列编程语言教材。谭有他的贡献，在那个国内计算机教育起步的年代，他出的书，也算是播下火种。
后来，很偶然，在抓包的时候，发现h1在发送ip数据包前，会向外广播arp报文，获得下一跳的mac地址。获取了h2的mac地址，h1发送的数据包的目的mac就是h2-eth0的mac。h2才会处理数据包。后来就采用arp欺骗的方式[5]，在不更改h1的路由配置的情况下，h2可以伪造arp响应，将h2-eth0的mac地址返回给h1。
Code and experiment
The topology on mininet:
#!/usr/bin/python
from mininet.topo import Topo
from mininet.net import Mininet
from mininet.cli import CLI
from mininet.link import TCLink
import time
import datetime
import subprocess
import os,signal
import sys
#https://segmentfault.com/a/1190000009562333
# arpspoof -i h2-eth0 -t 10.0.1.1 10.0.1.3
#
# h1--s1----h2------h3
#
bottleneckbw=6
nonbottlebw=20;
buffer_size =bottleneckbw*1000*30*3/(1500*8)
net = Mininet( cleanup=True )
h1 = net.addHost('h1',ip='10.0.1.1')
h2 = net.addHost('h2',ip='10.0.1.2')
h3 = net.addHost('h3',ip='10.0.2.2')
s1 = net.addSwitch( 's1' )
c0 = net.addController('c0')
net.addLink(h1,s1,intfName1='h1-eth0',intfName2='s1-eth0',cls=TCLink , bw=nonbottlebw, delay='10ms', max_queue_size=10*buffer_size)
net.addLink(s1,h2,intfName1='s1-eth1',intfName2='h2-eth0',cls=TCLink , bw=nonbottlebw, delay='10ms', max_queue_size=10*buffer_size)
net.addLink(h2,h3,intfName1='h2-eth1',intfName2='h3-eth0',cls=TCLink , bw=bottleneckbw, delay='10ms', max_queue_size=buffer_size)
net.build()
h1.cmd("ifconfig h1-eth0 10.0.1.1/24")
h1.cmd("route add default gw 10.0.1.2 dev h1-eth0")
h1.cmd('sysctl net.ipv4.ip_forward=1')#tproxy
h2.cmd("iptables -t nat -N MY_TCP")
h2.cmd("iptables -t nat -A PREROUTING -j MY_TCP")
h2.cmd("iptables -t nat -A MY_TCP -p tcp -d 10.0.2.2 -j REDIRECT --to-ports 2223")h2.cmd("ifconfig h2-eth0 10.0.1.2/24")
h2.cmd("ifconfig h2-eth1 10.0.2.1/24")
h2.cmd("ip route add to 10.0.2.0/24 via 10.0.2.2")
h2.cmd("ip route add to 10.0.1.0/24 via 10.0.1.1")
h2.cmd('sysctl net.ipv4.ip_forward=1')h3.cmd("ifconfig h3-eth0 10.0.2.2/24")
h3.cmd("route add default gw 10.0.2.1 dev h3-eth0")
h3.cmd('sysctl net.ipv4.ip_forward=1')net.start()
time.sleep(1)
CLI(net)
net.stop()These commands can be also used on real host to enble tcp transparent proxy.
iptables -t nat -N MY_TCP
iptables -t nat -A PREROUTING -j MY_TCP
iptables -t nat -A MY_TCP -p tcp -d 10.0.2.2 -j REDIRECT --to-ports 2223
The proxy server listens on port 2223 and wokes on hosts. Iperf can be used to test the available bandwidth between h1 and h2.
iperf client(h1)--------|TpProxyLeft|TpProxyRight|(h2)-----------iperf server(h3)
In an early version (the code below), h2 will put packets in buffer as long as the socket fd is readable. Then TpProxyRight will relay packets to h3. But there is an obvious drawback: when the available bandwidth between h2 and h3 is less than the bandwidth between h1 and h2, the read packets can not be sent to h3 as fast as they come in, packets will be buffered by TpProxyRight. In extreme case, the buffer in h2 will be exhausted and the transparent proxy server wll collapse. To test the below code on the sampled topology, iperf client can probe bandwidth about 20Mbps when it sends packets to h3. Since the packets are intercepted by h2, the probed bandwidth (20Mbps) is the bandwidth between h1 and h2. So traffic control module shoud be implmented.
透明代理的v1版本。这里TpProxyLeft截获包，就立马发送给TpProxyRight。这种方案有很大缺点，没有流控。h1到h3的瓶颈链路带宽是6Mbps。采用iperf测试h1到h3的带宽，因为数据包被h2截获，测试出的带宽是h1到h2链路的带宽20Mbps。在读取速率高于数据发送速率的情况下，h2中的TpProxyRight会占用大量的内存资源，最终可能导致程序崩溃。
tpproxy_server.h
#pragma once
#include <string>
#include <atomic>
#include "base/epoll_api.h"
#include "base/socket_address.h"
#include "tcp/tcp_server.h"
#include "tcp/tcp_types.h"
namespace basic{
class TpProxyRight;
class TpProxyBase{
public:TpProxyBase(basic::BaseContext *context,int fd);virtual ~TpProxyBase(){}virtual void Notify(uint8_t sig){}void SendData(const char *pv,size_t size);void set_peer(TpProxyBase *peer) {peer_=peer;}
protected:void FlushBuffer();void OnReadEvent(int fd);void OnWriteEvent(int fd);void CheckCloseFd();void CloseFd();void DeleteSelf();basic::BaseContext* context_=nullptr;int fd_=-1;std::string fd_write_buffer_;std::atomic<bool> destroyed_{false};int send_bytes_=0;int recv_bytes_=0;TcpConnectionStatus status_=TCP_DISCONNECT;uint8_t signal_=0;TpProxyBase *peer_=nullptr;
};
class TpProxyLeft:public TpProxyBase,
public EpollCallbackInterface{
public:TpProxyLeft(basic::BaseContext *context,int fd);~TpProxyLeft();void Notify(uint8_t sig) override;// From EpollCallbackInterfacevoid OnRegistration(basic::EpollServer* eps, int fd, int event_mask) override{}void OnModification(int fd, int event_mask) override {}void OnEvent(int fd, basic::EpollEvent* event) override;void OnUnregistration(int fd, bool replaced) override {}void OnShutdown(basic::EpollServer* eps, int fd) override;std::string Name() const override {return "TpProxyLeft";}
};
class TpProxyRight:public TpProxyBase,
public EpollCallbackInterface{
public:TpProxyRight(basic::BaseContext *context,int fd);~TpProxyRight();void Notify(uint8_t sig) override;bool AsynConnect(SocketAddress &local,SocketAddress &remote);// From EpollCallbackInterfacevoid OnRegistration(basic::EpollServer* eps, int fd, int event_mask) override {}void OnModification(int fd, int event_mask) override {}void OnEvent(int fd, basic::EpollEvent* event) override;void OnUnregistration(int fd, bool replaced) override {}void OnShutdown(basic::EpollServer* eps, int fd) override;std::string Name() const override {return "TpProxyRight";}
private:struct sockaddr_storage src_addr_;struct sockaddr_storage dst_addr_;
};
class TpProxyBackend:public Backend{
public:TpProxyBackend(){}void CreateEndpoint(basic::BaseContext *context,int fd) override;
};
class TpProxyFactory: public SocketServerFactory{
public:~TpProxyFactory(){}PhysicalSocketServer* CreateSocketServer(BaseContext *context) override;
};
}
tpproxy_server.cc
#include <memory.h>
#include <unistd.h>
#include <error.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <netinet/ip.h>
#include <linux/netfilter_ipv4.h>
#include <iostream>
#include "tpproxy/tpproxy_server.h"
namespace basic{
const int kBufferSize=1500;
enum TPPROXY_SIGNAL: uint8_t{TPPROXY_MIN,TPPROXY_CONNECT_FAIL,TPPROXY_CONNECTED,TPPROXY_CLOSE,TPPROXY_MAX,
};
/*EPOLLHUP
Hang up happened on the associated file descriptor.
epoll_wait(2) will always wait for this event; it is not neces-
sary to set it in events.
*/
TpProxyBase::TpProxyBase(basic::BaseContext *context,int fd):context_(context),fd_(fd){}
void TpProxyBase::FlushBuffer(){if(fd_<0){return ;}int remain=fd_write_buffer_.size();const char *data=fd_write_buffer_.data();bool flushed=false;while(remain>0){int intend=std::min(kBufferSize,remain);int sent=write(fd_,data,intend);if(sent<=0){break;}send_bytes_+=sent;flushed=true;data+=sent;remain-=sent;}if(flushed){if(remain>0){std::string copy(data,remain);copy.swap(fd_write_buffer_);}else{std::string null_str;null_str.swap(fd_write_buffer_);} }
}
void TpProxyBase::SendData(const char *pv,size_t size){if(fd_<0){return ;}if(status_!=TCP_CONNECTED){size_t old_size=fd_write_buffer_.size();fd_write_buffer_.resize(old_size+size);memcpy(&fd_write_buffer_[old_size],pv,size);}if(status_==TCP_CONNECTED){FlushBuffer();size_t old_size=fd_write_buffer_.size();if(old_size>0){fd_write_buffer_.resize(old_size+size);memcpy(&fd_write_buffer_[old_size],pv,size);return;}if(old_size==0){size_t sent=write(fd_,pv,size);if(sent<size){send_bytes_+=sent;const char *data=pv+sent;size_t remain=size-sent;fd_write_buffer_.resize(old_size+remain);memcpy(&fd_write_buffer_[old_size],data,remain);context_->epoll_server()->ModifyCallback(fd_,EPOLLIN|EPOLLOUT|EPOLLET|EPOLLRDHUP|EPOLLERR);}} }
}
void TpProxyBase::OnReadEvent(int fd){char buffer[kBufferSize];while(true){size_t nbytes=read(fd,buffer,kBufferSize); if (nbytes == -1) {//if(errno == EWOULDBLOCK|| errno == EAGAIN){}break; }else if(nbytes==0){CloseFd();}else{recv_bytes_+=nbytes;if(peer_){peer_->SendData(buffer,nbytes);}} }
}
void TpProxyBase::OnWriteEvent(int fd){FlushBuffer();if(fd_write_buffer_.size()>0){context_->epoll_server()->ModifyCallback(fd_,EPOLLIN|EPOLLOUT|EPOLLET|EPOLLRDHUP|EPOLLERR); }else{context_->epoll_server()->ModifyCallback(fd_,EPOLLIN|EPOLLET|EPOLLRDHUP|EPOLLERR);}CheckCloseFd();
}
void TpProxyBase::CheckCloseFd(){if((TPPROXY_CLOSE==signal_)&&(fd_write_buffer_.size()==0)){CloseFd();}
}
void TpProxyBase::CloseFd(){if(fd_>0){context_->epoll_server()->UnregisterFD(fd_); close(fd_);fd_=-1;status_=TCP_DISCONNECT;}if(peer_){peer_->Notify(TPPROXY_CLOSE);peer_=nullptr;}DeleteSelf();
}
void TpProxyBase::DeleteSelf(){if(destroyed_){return;}destroyed_=true;context_->PostTask([this]{delete this;});
}TpProxyLeft::TpProxyLeft(basic::BaseContext *context,int fd):TpProxyBase(context,fd){context_->epoll_server()->RegisterFD(fd_,this,EPOLLIN|EPOLLET|EPOLLRDHUP| EPOLLERR);struct sockaddr_storage remote_addr;/*IpAddress ip_addr;ip_addr.FromString("10.0.2.2");SocketAddress socket_addr(ip_addr,3333);remote_addr=socket_addr.generic_address();*/socklen_t n=sizeof(remote_addr);int ret =getsockopt(fd_, SOL_IP, SO_ORIGINAL_DST, &remote_addr, &n);if(ret!=0){CloseFd();return;}int right_fd=socket(AF_INET, SOCK_STREAM, 0);if(0>right_fd){CloseFd();return; }SocketAddress local(IpAddress::Any4(),0);SocketAddress remote(remote_addr);std::cout<<remote.ToString()<<std::endl;peer_=new TpProxyRight(context,right_fd);bool success=((TpProxyRight*)peer_)->AsynConnect(local,remote);if(!success){CloseFd();std::cout<<"asyn failed"<<std::endl;return; }else{peer_->set_peer(this);}status_=TCP_CONNECTED;
}
TpProxyLeft::~TpProxyLeft(){std::cout<<"left dtor "<<recv_bytes_<<" "<<send_bytes_<<std::endl;
}
void TpProxyLeft::Notify(uint8_t sig){if(TPPROXY_CLOSE==sig||TPPROXY_CONNECT_FAIL==sig){peer_=nullptr;signal_=TPPROXY_CLOSE;CheckCloseFd();}
}
void TpProxyLeft::OnEvent(int fd, basic::EpollEvent* event){if(event->in_events & EPOLLIN){OnReadEvent(fd);}if(event->in_events&EPOLLOUT){OnWriteEvent(fd);}if(event->in_events &(EPOLLRDHUP|EPOLLHUP)){CloseFd(); }
}
void TpProxyLeft::OnShutdown(basic::EpollServer* eps, int fd){if(fd_>0){close(fd_);fd_=-1;}DeleteSelf();
}TpProxyRight::TpProxyRight(basic::BaseContext *context,int fd):TpProxyBase(context,fd){}
TpProxyRight::~TpProxyRight(){std::cout<<"right dtor "<<recv_bytes_<<" "<<send_bytes_<<std::endl;
}
void TpProxyRight::Notify(uint8_t sig){if(TPPROXY_CLOSE==sig){signal_=sig;peer_=nullptr;CheckCloseFd();}
}
bool TpProxyRight::AsynConnect(SocketAddress &local,SocketAddress &remote){src_addr_=local.generic_address();dst_addr_=remote.generic_address();int yes=1;bool success=false;size_t addr_size = sizeof(struct sockaddr_storage);if(bind(fd_, (struct sockaddr *)&src_addr_, addr_size)<0){CloseFd();return success;}if(setsockopt(fd_,SOL_SOCKET,SO_REUSEADDR,&yes,sizeof(int))!=0){CloseFd();return success; }context_->epoll_server()->RegisterFD(fd_, this,EPOLLIN|EPOLLOUT| EPOLLRDHUP | EPOLLERR | EPOLLET);if(connect(fd_,(struct sockaddr *)&dst_addr_,addr_size) == -1&& errno != EINPROGRESS){//connect doesn't work, are we running out of available ports ? if yes, destruct the socket if (errno == EAGAIN){CloseFd();return success; } }status_=TCP_CONNECTING;return true;
}
void TpProxyRight::OnEvent(int fd, basic::EpollEvent* event){if (event->in_events&(EPOLLERR|EPOLLRDHUP| EPOLLHUP)){CloseFd(); } if(event->in_events&EPOLLOUT){if(status_==TCP_CONNECTING){status_=TCP_CONNECTED;std::cout<<"right connected"<<std::endl;context_->epoll_server()->ModifyCallback(fd_,EPOLLIN|EPOLLRDHUP|EPOLLERR | EPOLLET);}OnWriteEvent(fd);}if(event->in_events&EPOLLIN){OnReadEvent(fd);}
}
void TpProxyRight::OnShutdown(basic::EpollServer* eps, int fd){if(fd_>0){close(fd_);fd_=-1;}DeleteSelf();
}
void TpProxyBackend::CreateEndpoint(basic::BaseContext *context,int fd){TpProxyLeft *endpoint=new TpProxyLeft(context,fd);UNUSED(endpoint);
}
PhysicalSocketServer* TpProxyFactory::CreateSocketServer(BaseContext *context){std::unique_ptr<TpProxyBackend> backend(new TpProxyBackend());return new PhysicalSocketServer(context,std::move(backend));
}
}The newest version of the transparent proxy code [7] on github has implemented the traffic control function. When the size of fd_write_buffer_ in TpProxyRight exceeds a predefined threshold (1500*10), TpProxyLeft will not read packets from tcp stack for temporally and the occupied buffer in tcp stack will feedback reduced receive window (Window Size in tcp header 1) to sender. The sender with reduce packet sending speed. With such mechanism, the probed bandwidth between h1 and h3 is closed to the bottleneck bnadwidth (6Mbps).
[1] 利用TAP设备实现透明传输
[2] mtcp
[3] socket的IP_TRANSPARENT选项实现代理
[4] tproxy-example
[5] Linux透明代理 —— 使用iptables实现TCP透明代理
[6] test arp spoof on mininet
[7] tcp transparent proxy
[8] TCP流量控制、拥塞控制
[9] Network Analysis: TCP Window Size
The size of the receive window, which specifies the number of window size units[c] that the sender of this segment is currently willing to receive ↩︎
tpproxy-tcp透明代理相关推荐
- 实现TCP透明代理（附下载）
本文所描述的TCP代理服务器工作于网络协议层次中的应用层,位于传输层之上.只要是以TCP的方式为客户提供服务的(包括我们的HTTP服务器,HTTP底层走的仍然是TCP),我们都可以在真正的TCP服务器 ...
- tcp透明代理 linux（c语言）
一个使用c编写的tcp透明代理 1.为啥要做 2.透明代理的实现 3.源码 4.编译 5.输出 6.引申 需要的系统配置 参考文章 1.为啥要做 一个需求,需要SMB的代理来监查上传内容进行管控.查遍 ...
- Linux下透明代理+Privoxy实现页面相关的广告植入
一. 前言 Linux系统下Privoxy实现广告植入的方式很简单.通常的做法,就是将HTTP流量(端口80,8080等)NAT到Privoxy,然后由Privoxy实现流量的过滤与广告的植入.这种方 ...
- Linux 实现透明代理（使用开源项目 tproxy-example）
一.透明代理概述 根据百度百科的资料:透明代理的意思是客户端根本不需要知道有代理服务器的存在,它改变你的 request fields(报文),并会传送真实IP,多用于路由器的NAT转发中. 透明代理 ...
- squid+iptables实现透明代理
NAT 网络地址转换(网络地址映射) 就是把数据包的源IP或者目标IP进行修改. 作用: 修改源IP,叫源地址映射,一般为了实现让私有网络的机器能够访问互联网 修改目标IP,叫目标地址映射,一般为了实 ...
- squid+iptalbes实现透明代理配置记录
首先描述下硬件环境cpu P4 2.93\2G\80G 单网卡;系统环境为CentOS 5.4 x86_64,所需的squid由yum安装,squid包的版本为squid-2.6.STABLE21-6 ...
- 使用squid配置透明代理并对上网行为进行控制
使用Squid配置透明代理 环境:CentOS 6.4 + squid-3.1.10-20.el6_5.3.x86_641.检查squid是否默认安装,没有安装先安装 rpm -qa squid 假如 ...
- Haproxy全透明代理
1. 系统环境搭建 操作系统Centos7 内核版本3.10 Centos7已自带TPROXY模块,不需要安装TPROXY 2. Haproxy下载,编译,安装,配置 下载地址 http://www. ...
- mysql 透明代理_透明代理MySQL_基于zbus的MySQL透明代理（100行）-云栖社区
我们上次讲到zbus网络通讯的核心API: Dispatcher -- 负责-NIO网络事件Selector引擎的管理,对Selector引擎负载均衡 IoAdaptor -- 网络事件的处理,服务器 ...
最新文章
- 用Visual Studio开发以太坊智能合约
- bat 指定jdk_微服务开发 bat 一键批量启动 jar（效率小技巧）
- 逻辑回归模型_联邦学习体系下——逻辑回归模型
- 零基础转行学习python是否还在纠结？这里告诉你答案！
- 推荐一款好用的截图软件Faststone capture
- 20162312 2016-2017-2《Java程序设计》课程总结
- Spring：连接池连接数据库报错Unknown system variable ‘tx_isolation‘
- 标准输入输出详解（C++）
- 关于hashcode和equals方法
- Common Digester学习
- ubuntu解决E: Prior errors apply to /var/cache/apt/archives/sqliteman-doc_1.2.2-0ubuntu6_all.deb的问题
- 团队展示——我说的都队
- 为什么未来是全栈工程师的世界？
- win10家庭版 mysql_MySQL下载安装详解（win10家庭版）
- Vue 引入 icon 图标
- 计算机用户分为哪4类,计算机的分类-通常将计算机分为哪几类？通常将计算机分为哪几类，各自的特点和用途 爱问知识人...
- 安信可EC系列模组接入OneNET物联网开放平台的多协议接入产品
- Activiti 5 提示：Default sequenceflow has a condition, which is not allowed
- 处女篇-我的程序员之路
- 计算机视觉与图像处理区别