HTTP Agent理解

http.Agent官方文档

附图，个人对http.Agent的理解：

image.png

一个 Agent是在client端用来管理链接的持久性和重用。对于一个host+port维持着一个请求队列，这些请求重复使用一个socket，直到这个队列空，这时，这个socket会被destroy或者放到pool里，在pool里时这个socket将会被再次重用(这两个行为取决于keepAlive的配置）

keepAlive
keepAliveMsecs
maxSockets
maxFreeSockets

在pool中的链接已经开启了tcp的Keep-Alive，然而在server端会有如下行为影响pool中的链接：

测试代码准备：

client.js


const http = require('http');const agent = new http.Agent({keepAlive: true,keepAliveMsecs: 1000,maxSockets: 4,maxFreeSockets: 2
});const test = () => {return new Promise((resolve, reject) => {const option = {protocol: 'http:',host: 'localhost',port: 9990,path: `/`,agent: agent,// agent: agent,headers: {"Connection": "keep-alive"},method: 'GET'};const req = http.request(option, function(res) {res.setEncoding('utf8');let body = '';res.on('data', (chunk) => {body += chunk;});res.on('end', () => {resolve(body)});});req.on('error', (e) => {console.error(`problem with request: ${e.message}`);console.log(e.stack)});req.end();})
};const sendReq = (count) => {let arr = [];for (let i=0;i<count;i++) arr.push(test())Promise.all(arr).then(function(){console.log('======end======')})
}

server.js

const http = require('http');let server = http.createServer(function(req, res) {console.log(req.connection.remotePort);res.end('200');}).listen(9990);server.keepAliveTimeout = 5000; // 这个值默认就是5s,可以直接赋值修改

server端主动关闭空闲链接：client收到通知后，当前socket会从pool中移除，下一次请求时会创建一个新的socket

Pooled connections have TCP Keep-Alive enabled for them, but servers may still close idle connections, in which case they will be removed from the pool and a new connection will be made when a new HTTP request is made for that host and port.

client.js补充

sendReq(1);  // 先发送一个reqsetTimeout(() => {sendReq(1)}, 10 * 1000); //隔10s后再次发送一次req

server.js输出如下：

 // console.log(req.connection.remotePort);53957 // 发送第一个请求的socket port54011 // 隔10s后发送第二个请求的socket port。port不同，说明第一个socket已经被关闭

wireshark抓包如下：

image.png

可以看到每隔1s发送向server端发送了一次TCP Keep-Alive探测。由于server端设置的keepAliveTimeout为5s(默认就是5s)，所以在5s后关闭了这个tcp链接，相应的，client端收到关闭的信号就会close到当前的socket，并从pool中移除这个socket

_http_agent.js

Agent.prototype.removeSocket = function removeSocket(s, options) {var name = this.getName(options);debug('removeSocket', name, 'writable:', s.writable);var sets = [this.sockets];// If the socket was destroyed, remove it from the free buffers too.if (!s.writable)sets.push(this.freeSockets);for (var sk = 0; sk < sets.length; sk++) {var sockets = sets[sk];if (sockets[name]) {var index = sockets[name].indexOf(s);if (index !== -1) {sockets[name].splice(index, 1);// Don't leakif (sockets[name].length === 0)delete sockets[name];}}}// 省略其他代码
};

server端拒绝多个请求共用一个tcp链接，在这种情况下，在每次请求时链接都会建立并且不能被pool。agent仍然会处理请求的发送，只是每个请求都会建立在一个新的tcp链接上

Servers may also refuse to allow multiple requests over the same connection, in which case the connection will have to be remade for every request and cannot be pooled. The Agent will still make the requests to that server, but each one will occur over a new connection.

client.js不变

server.js添加如下代码

    res.shouldKeepAlive = false; // 禁用shouldkeepAliveres.end('200');

wireshark抓包如下：

image.png

可以看到，请求结束后，server就会关闭socket

When a connection is closed by the client or the server, it is removed from the pool. Any unused sockets in the pool will be unrefed so as not to keep the Node.js process running when there are no outstanding requests. (see socket.unref()).

当想要保持一个http请求很长时间并不在pool中，可以调用“agentRemove”（这个时间取决于server端socket close的时间）

Sockets are removed from an agent when the socket emits either a 'close' event or an 'agentRemove' event. When intending to keep one HTTP request open for a long time without keeping it in the agent, something like the following may be done:

client.js

      // newreq.on('socket', (socket) => {socket.emit('agentRemove');});

server.js

server.keepAliveTimeout = 20000; // 为了清楚，服务端设置20s后再关闭

wireshark抓包如下：

image.png

可以看到，触发“agentRemove”后，当前socket并没有发送探测包，并且知道server端通知关闭才关闭。

当agent参数设置为false时，client将会为每一个http请求都创建一个链接。

node keep-alive还是很有必要开启的，尤其时作为中间层代理，当有如下模式时：高并发下优势更明显

browser浏览器 -> nginx -> node -> nginx -> java

当nginx和node都开启keep-alive时，性能测试如下：

image.png

参考资料mark：
http://tldp.org/HOWTO/TCP-Keepalive-HOWTO/overview.html

https://stackoverflow.com/questions/30365250/what-will-happen-if-i-use-socket-setkeepalive-in-node-js-server

https://stackoverflow.com/questions/19043355/how-to-use-request-js-node-js-module-pools

https://github.com/nodejs/node/issues/10774

NodeJS的底层通信

这篇文章很详细，赞一个
https://www.zhuxiaodong.net/2018/tcp-http-keepalive/

HTTP Agent理解相关推荐

Agent系列（一）什么是Agent
声明:此文章为原创,不得转载. 本文主要是介绍下什么是Agent , Agent有什么特性,并简要的分析了Agent技术的应用前景. 一.Agent定义通常,我们可以把Agent理解为"个 ...
场景理解的统一感知分析ECCV2018
目录前置内容-PPM 摘要 1.Introduction 1.1.Related work 2.统一感知解析的定义 2.1.Datasets 2.2.Metrics 3.Designing Netw ...
VLN阅读报告1：Vision-and-Language Navigation综述（2022ACL）
本博文是结合论文Vision-and-Language Navigation: A Survey of Tasks, Methods, and Future Directions,对VLN进行学习总结 ...
vnx 服务器映射,主机到VNX系统的常见注册方式
介绍通常情况下,在主机和VNX存储系统物理连接正常后,新安装的主机都要到存储系统上完成注册.注册过程中会将主机IP地址.主机名称和操作系统信息发送到存储系统,以方便存储系统对主机的日后管理.目前有三 ...
Unity的机器学习工具包ML-Agents
官方:Unity ML-Agents深度学习工具包|Unity中国官网 | Unity中国官网 Github下载链接:https://github.com/Unity-Technologies/ml- ...
VLN阅读报告6：SOON: Scenario Oriented Object Navigation with Graph-based Exploration
SOON: Scenario Oriented Object Navigation with Graph-based Exploration 一,引言二,SOON任务(Scenario Orient ...
Relational Deep Reinforcement Learning
Abstract 我们介绍了一种深度强化学习的方法,它通过结构化感知和关系推理提高了传统方法的效率.泛化能力和可解释性.它使用self-attention来迭代推理场景中实体之间的关系,并指导无模型策 ...
理解一个名词：用户代理(user agent)
文/牛海彬原文: The term user agent and the lack of understanding what a user agent is can also be a probl ...
读后感与机翻《理解工具:面向任务的对象建模、学习和识别》
以下是研究朱松纯FPICU概念中P(physics)的第一篇论文记录: 目录读后感: 作者干了什么事? 作者怎么做的? 效果怎么样? 局限性摘要 1 介绍 2 面向任务的对象表示 2.1 三维空间 ...

HTTP Agent理解

HTTP Agent理解相关推荐

最新文章

热门文章