已解决requests.exceptions.ConnectTimeout: HTTPConnectionPool(host=‘123.96.1.95’, port=30090): Max retries exceeded with url: http://cdict.qq.pinyin.cn/list?cate_id=461&sort1_id=436&sort2_id=461&page=4 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x00000204934C49A0>, ‘Connection to 123.96.1.95 timed out. (connect timeout=20)’))


文章目录

  • 报错代码
  • 报错翻译
  • 报错原因
  • 解决方法
  • 千人全栈VIP答疑群联系博主帮忙解决报错

报错代码

粉丝群一个小伙伴用requests.get爬取网页源码的时候报错(当时他心里瞬间凉了一大截,跑来找我求助,然后顺利帮助他解决了,顺便记录一下希望可以帮助到更多遇到这个bug不会解决的小伙伴),报错代码如下:

def get_url(url):"""发送请求获取响应返回网页源码"""# 如果访问失败无限循环访问五次try:r = requests.get(url, headers=headers)except:time.sleep(120)for i in range(5):r = requests.get(url, headers=headers, timeout=20)if r.status_code == '200':breaktime.sleep(300)html_str = r.content.decode('utf8')return html_str

报错信息如下

requests.exceptions.ConnectTimeout: HTTPConnectionPool(host=‘123.96.1.95’, port=30090): Max retries exceeded with url: http://cdict.qq.pinyin.cn/list?cate_id=461&sort1_id=436&sort2_id=461&page=4 (Caused by ConnectTimeoutError(<urllib3.connection.HTTPConnection object at 0x00000204934C49A0>, ‘Connection to 123.96.1.95 timed out. (connect timeout=20)’))

报错翻译

报错信息翻译

请求.例外情况。连接超时:http连接池(主机=‘123.96.1.95’,端口=30090):超过url的最大重试次数:http://cdict.qq.pinyin.cn/list?cate_id=461&sort1_id=436&sort2_id=461&page=4(由连接超时报错(<0x00000204934C49A0处的urllib3.连接池.HTTP连接池对象>,‘到123.96.1.95的连接超时(连接超时=20’))引起)

报错原因

若分别指定连接和读取的超时时间,服务器在指定时间没有应答,抛出 requests.exceptions.ConnectTimeout

  • timeout=([连接超时时间], [读取超时时间])
  • 连接:客户端连接服务器并并发送http请求服务器
  • 读取:客户端等待服务器发送第一个字节之前的时间

小伙伴按下面的四种解决方法即可解决!!!

解决方法

第一种解决方法:循环抓错,重复请求

def get_url(url):try:response = requests.get(url, headers=headers, timeout=10)  # 超时设置为10秒except:for i in range(4):  # 循环去请求网站response = requests.get(url, headers=headers, timeout=20)if response.status_code == 200:breakhtml_str = response.textreturn html_str

第二种解决方法:构建headers列表每次随机选一个

headers_list = [{'user-agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.0.0; SM-G955U Build/R16NW) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 10; SM-G981B) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.162 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (iPad; CPU OS 13_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/87.0.4280.77 Mobile/15E148 Safari/604.1'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.109 Safari/537.36 CrKey/1.54.248666'}, {'user-agent': 'Mozilla/5.0 (X11; Linux aarch64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.188 Safari/537.36 CrKey/1.54.250320'}, {'user-agent': 'Mozilla/5.0 (BB10; Touch) AppleWebKit/537.10+ (KHTML, like Gecko) Version/10.0.9.2372 Mobile Safari/537.10+'}, {'user-agent': 'Mozilla/5.0 (PlayBook; U; RIM Tablet OS 2.1.0; en-US) AppleWebKit/536.2+ (KHTML like Gecko) Version/7.2.1.0 Safari/536.2+'}, {'user-agent': 'Mozilla/5.0 (Linux; U; Android 4.3; en-us; SM-N900T Build/JSS15J) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30'}, {'user-agent': 'Mozilla/5.0 (Linux; U; Android 4.1; en-us; GT-N7100 Build/JRO03C) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30'}, {'user-agent': 'Mozilla/5.0 (Linux; U; Android 4.0; en-us; GT-I9300 Build/IMM76D) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 7.0; SM-G950U Build/NRD90M) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.84 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.0.0; SM-G965U Build/R16NW) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.111 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.1.0; SM-T837A) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.80 Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; U; en-us; KFAPWI Build/JDQ39) AppleWebKit/535.19 (KHTML, like Gecko) Silk/3.13 Safari/535.19 Silk-Accelerated=true'}, {'user-agent': 'Mozilla/5.0 (Linux; U; Android 4.4.2; en-us; LGMS323 Build/KOT49I.MS32310c) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Windows Phone 10.0; Android 4.2.1; Microsoft; Lumia 550) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2486.0 Mobile Safari/537.36 Edge/14.14263'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 6.0.1; Moto G (4)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 10 Build/MOB31T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 4.4.2; Nexus 4 Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.0.0; Nexus 5X Build/OPR4.170623.006) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 7.1.1; Nexus 6 Build/N6F26U) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.0.0; Nexus 6P Build/OPP3.170518.006) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 7 Build/MOB30X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (compatible; MSIE 10.0; Windows Phone 8.0; Trident/6.0; IEMobile/10.0; ARM; Touch; NOKIA; Lumia 520)'}, {'user-agent': 'Mozilla/5.0 (MeeGo; NokiaN9) AppleWebKit/534.13 (KHTML, like Gecko) NokiaBrowser/8.5.0 Mobile Safari/534.13'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 9; Pixel 3 Build/PQ1A.181105.017.A1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.158 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 10; Pixel 4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 11; Pixel 3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.181 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.0.0; Pixel 2 XL Build/OPD1.170816.004) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_1 like Mac OS X) AppleWebKit/603.1.30 (KHTML, like Gecko) Version/10.0 Mobile/14E304 Safari/602.1'}, {'user-agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1'}, {'user-agent': 'Mozilla/5.0 (iPad; CPU OS 11_0 like Mac OS X) AppleWebKit/604.1.34 (KHTML, like Gecko) Version/11.0 Mobile/15A5341f Safari/604.1'}
]headers = random.choice(headers_list)

第三种方法解决方法:放慢请求速度

time.sleep(10)

第四种方法解决方法:代码设置代理(这里的代理ip需要自己购买,网上的免费代理都是不行的)

proxies = {'http': '127.0.0.1:1212','https': '127.0.0.1:1212'}
r = requests.get(url, headers=headers, proxies=proxies, timeout=20)

以上是此问题报错原因的解决方法,欢迎评论区留言讨论是否能解决,如果有用欢迎点赞收藏文章谢谢支持,博主才有动力持续记录遇到的问题!!!

千人全栈VIP答疑群联系博主帮忙解决报错

由于博主时间精力有限,每天私信人数太多,没办法每个粉丝都及时回复,所以优先回复VIP粉丝,可以通过订阅限时9.9付费专栏《100天精通Python从入门到就业》进入千人全栈VIP答疑群,获得优先解答机会(代码指导、远程服务),白嫖80G学习资料大礼包,专栏订阅地址:https://blog.csdn.net/yuan2019035055/category_11466020.html

  • 优点:作者优先解答机会(代码指导、远程服务),群里大佬众多可以抱团取暖(大厂内推机会),此专栏文章是专门针对零基础和需要进阶提升的同学所准备的一套完整教学,从0到100的不断进阶深入,后续还有实战项目,轻松应对面试!

  • 专栏福利:简历指导、招聘内推、每周送实体书、80G全栈学习视频、300本IT电子书:Python、Java、前端、大数据、数据库、算法、爬虫、数据分析、机器学习、面试题库等等

  • 注意:如果希望得到及时回复,订阅专栏后私信博主进千人VIP答疑群

免费资料获取,更多粉丝福利,关注下方公众号获取

已解决requests.exceptions.ConnectTimeout: HTTPConnectionPool(host=‘123.96.1.95‘, port=30090): Max retri相关推荐

  1. requests.exceptions.ConnectionError: HTTPConnectionPool(host=‘localhost‘, port=8123): Max retries ex

    参考requests.exceptions.ConnectionError: HTTPConnectionPool(host='localhost', port=8123): Max retries ...

  2. 解决requests.exceptions.SSLError: HTTPSConnectionPool(host=xxxxx‘, port=443): Max retries exceeded

    问题描述 使用scrapy框架时报错SSL,于是另起一个文件,使用requests,报错requests.exceptions.SSLError: HTTPSConnectionPool(host=' ...

  3. HTTPConnectionPool(host=‘127.0.0.1‘, port=44245) 解决方案

    Error: 1. 访问网络时 requests.exceptions.ProxyError: HTTPConnectionPool(host='127.0.0.1', port=44245): Ma ...

  4. requests.exceptions.ConnectionError:HTTPSConnectionPool(host

    requests.exceptions.ConnectionError: HTTPSConnectionPool(host= 应用场景(爬虫一) 使用requests.get()请求链接报错,具体不知 ...

  5. 解决python爬虫requests.exceptions.SSLError: HTTPSConnectionPool(host=‘XXXXXXX‘, port=443)问题

    报错如下 Traceback (most recent call last):File "C:/Users/17653/Desktop/测试.py", line 23, in &l ...

  6. 关于requests.exceptions.SSLError: HTTPSConnectionPool(host='XXX', port=443)问题

    在运行一下代码的时候,会进行报错 requests.exceptions.SSLError: HTTPSConnectionPool(host='XXX', port=443) 解决方法如下: 1.先 ...

  7. requests.exceptions.SSLError: HTTPSConnectionPool(host='XXX', port=443)问题

    具体报错是这样的: requests.exceptions.SSLError: HTTPSConnectionPool(host='某某某网站', port=443):Max retries exce ...

  8. urllib3.exceptions.MaxRetryError: HTTPConnectionPool(host=‘localhost‘, port=8097): Max retries excee

    问题描述: 使用nohup - > log.txt &命令训练深度学习模型时,输出日志报错Connection to remote host was lost- 相关命令可见我的另一篇博 ...

  9. Python代理IP异常:requests.exceptions.ProxyError: HTTPSConnectionPool(host=‘xxxx‘, port=443)

    背景 这是我在使用Python爬取网页信息的时候出现的异常. 由于很多要爬取的网站会把"IP限制"作为反爬虫手段,所以我偶尔会使用"代理IP服务器"去处理相关需 ...

最新文章

  1. 每日一皮:天气转凉了,你的长袖穿起来了吗?
  2. Bzoj1312 / POJ3155 Neerc2006 Hard Life
  3. python-自动发邮件
  4. 无极绳连续牵引绞车_可视化无极绳绞车保护控制装置
  5. Spring Cloud –基本设置
  6. JavaScript开发工具--Aptana
  7. Cache related website
  8. 判断页面环境是否在小程序的webview中
  9. P1955 [NOI2015]程序自动分析
  10. EM算法原理解释及公式推导
  11. PyTorch常用5个抽样函数
  12. 如何实现转圈圈动画效果
  13. 【每日一题】字符串转换成整数
  14. unity找到指定名称的一个物体的子物体,多个子物体有相同的名称
  15. 用element-ui el-select 实现拼音码搜搜功能ts版
  16. 华为荣耀5c的语言是英文,原来华为手机自带翻译功能,千万别浪费了!
  17. 【_ 記 】vue中的slot属性和slot样式自定义
  18. Hive Sql中均值、方差、标准差、皮尔逊相关系数、偏度、峰度统计字段
  19. 工业锅炉计算机控制的研究与设计,工业锅炉的节能与微机控制
  20. 得大数据者 得新工业革命先机

热门文章

  1. 二维码的扫描和生成二维码
  2. javascript(JS) 0基础快速入门 (二)(this指向问题)
  3. C - 有符号数和无符号数扩展
  4. elasticsearch中对于空字符串的过滤操作
  5. 关闭Linux防火墙
  6. Web全栈~06.CSS选择器
  7. WinMain:应用程序入口点【WinMain: The Application Entry Point】
  8. 程序员的种种崩溃瞬间
  9. spring系列——BeanNameGenerator接口(bean名字生成器)
  10. [汉化主题] Knowhow v1.1.16 – 响应式论坛知识库WordPress主题