requests.exceptions.ConnectionError: HTTPConnectionPool(host=‘****, port=80): Max retries exceeded w

python requests.get爬虫时，跑几个数据后，报错：requests.exceptions.ConnectionError: HTTPConnectionPool(host='****, port=80): Max retries exceeded with url: /beijing/file/4023E7D190674D26934AED5F4306DBC0/B76E1E3D9DF842D6ACBC63978C3A89FE/977FE5585248441E86067EFBF097E587/zz/ew3.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbc166b6490>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

具体如下：

Traceback (most recent call last):File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 156, in _new_connconn = connection.create_connection(File "/usr/lib/python3/dist-packages/urllib3/util/connection.py", line 61, in create_connectionfor res in socket.getaddrinfo(host, port, family, socket.SOCK_STREAM):File "/usr/lib/python3.8/socket.py", line 918, in getaddrinfofor res in _socket.getaddrinfo(host, port, family, type, proto, flags):
socket.gaierror: [Errno -3] Temporary failure in name resolutionDuring handling of the above exception, another exception occurred:Traceback (most recent call last):File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 665, in urlopenhttplib_response = self._make_request(File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 387, in _make_requestconn.request(method, url, **httplib_request_kw)File "/usr/lib/python3.8/http/client.py", line 1240, in requestself._send_request(method, url, body, headers, encode_chunked)File "/usr/lib/python3.8/http/client.py", line 1286, in _send_requestself.endheaders(body, encode_chunked=encode_chunked)File "/usr/lib/python3.8/http/client.py", line 1235, in endheadersself._send_output(message_body, encode_chunked=encode_chunked)File "/usr/lib/python3.8/http/client.py", line 1006, in _send_outputself.send(msg)File "/usr/lib/python3.8/http/client.py", line 946, in sendself.connect()File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 184, in connectconn = self._new_conn()File "/usr/lib/python3/dist-packages/urllib3/connection.py", line 168, in _new_connraise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPConnection object at 0x7fbc166b6490>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolutionDuring handling of the above exception, another exception occurred:Traceback (most recent call last):File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 439, in sendresp = conn.urlopen(File "/usr/lib/python3/dist-packages/urllib3/connectionpool.py", line 719, in urlopenretries = retries.increment(File "/usr/lib/python3/dist-packages/urllib3/util/retry.py", line 436, in incrementraise MaxRetryError(_pool, url, error or ResponseError(cause))
urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host='****', port=80): Max retries exceeded with url: /beijing/file/4023E7D190674D26934AED5F4306DBC0/B76E1E3D9DF842D6ACBC63978C3A89FE/977FE5585248441E86067EFBF097E587/zz/ew3.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbc166b6490>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))During handling of the above exception, another exception occurred:Traceback (most recent call last):File "/data/xll/code/dzy/ddjs/parsebj.py", line 143, in <module>r = s.get(url=url, headers=headers, timeout=60)File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 543, in getreturn self.request('GET', url, **kwargs)File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 530, in requestresp = self.send(prep, **send_kwargs)File "/usr/local/lib/python3.8/dist-packages/requests/sessions.py", line 643, in sendr = adapter.send(request, **kwargs)File "/usr/local/lib/python3.8/dist-packages/requests/adapters.py", line 516, in sendraise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPConnectionPool(host='****, port=80): Max retries exceeded with url: /beijing/file/4023E7D190674D26934AED5F4306DBC0/B76E1E3D9DF842D6ACBC63978C3A89FE/977FE5585248441E86067EFBF097E587/zz/ew3.jpg (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fbc166b6490>: Failed to establish a new connection: [Errno -3] Temporary failure in name resolution'))

在网上找到一些办法：参考https://blog.csdn.net/a1007720052/article/details/83383220

在请求头中加：'Connection': 'close'
设置请求间隔：time.sleep(5)
设置session:

s = requests.session()
s.keep_alive = False

都没有解决这个问题，后来发现原因处在：socket.gaierror: [Errno -3] Temporary failure in name resolution

参考：https://stackoverflow.com/questions/40238610/what-is-the-meaning-of-gaierror-errno-3-temporary-failure-in-name-resolutio

域名解析失败，修改域名为ip地址

r = requests.get(url=url.replace("<域名>", "<IP>"))

例如：

r = requests.get(url=url.replace("aaa.bb.com", "123.12.13.14"))

总结：改域名为ip

requests.exceptions.ConnectionError: HTTPConnectionPool(host=‘****, port=80): Max retries exceeded w相关推荐

python OSError: [Errno 24] Too many open files | HTTPConnectionPool(host=‘‘, port=80): Max retries e
对于问题:python OSError: [Errno 24] Too many open files 原因:超出了进程同一时间最多可开启的文件数. 解决方案P: 使用ulimit -n查看进程同一时 ...
HDFS读取问题：HTTPConnectionPool(host=, port=50075): Max retries exceeded
在使用python读写HDFS时,遇到以下问题: ConnectionError: HTTPConnectionPool(host='node1.******.com', port=50075): M ...
requests.exceptions.ConnectionError: HTTPConnectionPool(host=‘localhost‘, port=8123): Max retries ex
参考requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8123): Max retries ...
requests.exceptions.ConnectionError:HTTPSConnectionPool(host
requests.exceptions.ConnectionError: HTTPSConnectionPool(host= 应用场景(爬虫一) 使用requests.get()请求链接报错,具体不知 ...
已解决requests.exceptions.ConnectTimeout: HTTPConnectionPool(host=‘123.96.1.95‘, port=30090): Max retri
已解决requests.exceptions.ConnectTimeout: HTTPConnectionPool(host='123.96.1.95', port=30090): Max retri ...
解决requests.exceptions.SSLError: HTTPSConnectionPool(host=xxxxx‘, port=443): Max retries exceeded
问题描述使用scrapy框架时报错SSL,于是另起一个文件,使用requests,报错requests.exceptions.SSLError: HTTPSConnectionPool(host=' ...
HTTPConnectionPool(host=‘localhost‘, port=28333): Max retries exceeded with url: /events (Caused by
错误: 训练神经网络的时候报错如下: HTTPConnectionPool(host='localhost', port=28333): Max retries exceeded with url: ...
HTTPConnectionPool（host:XX）Max retries exceeded with url 解决方法
2019独角兽企业重金招聘Python工程师标准>>> HTTPConnectionPool(host:XX)Max retries exceeded with url 解决方法在 ...
python HTTPConnectionPool（host:XX）Max retries exceeded with url
python 出现HTTPConnectionPool(host:XX)Max retries exceeded with url问题在每次数据传输前客户端要和服务器建立TCP连接,为节省传输消耗, ...
跑cyclegan时遇到了ConnectionError: HTTPConnectionPool(host=‘localhost‘, port=8097): Max retries exceeded
搜了一下解决方法,如下图:

requests.exceptions.ConnectionError: HTTPConnectionPool(host=‘****, port=80): Max retries exceeded w

总结：改域名为ip

requests.exceptions.ConnectionError: HTTPConnectionPool(host=‘****, port=80): Max retries exceeded w相关推荐

最新文章

热门文章