报错信息

粉丝群里面的一个小伙伴想用python爬取网页但是发生了报错（当时他心里瞬间凉了一大截，跑来找我求助，然后顺利帮助他解决了，顺便记录一下希望可以帮助到更多遇到这个bug不会解决的小伙伴），报错代码如下：：
：

ProxyError：HTTPConnectionPool（host ='127.0.0.1'，port = 39149）

报错翻译

报错内容翻译：

代理错误：HTTP连接池（host =‘127.0.0.1’，port = 39149）：URL超过了最大重试次数：http : //www.baidu.com/（由代理错误（‘无法连接到代理服务器。’，NewConnectionError（‘：无法建立新的连接：[Errno 111]连接被拒绝’，）））

报错原因

报错原因：

1.、http的连接数超过最大限制，默认的情况下连接是Keep-alive的，所以这就导致了服务器保持了太多连接而不能再新建连接。
2、ip被封
3、程序请求速度过快。

方面三种解决方法小伙伴们任选其一即可！！！

解决方法

第一种解决方法：循环抓错，重复请求：

def get_url(url):try:response = requests.get(url, headers=headers, timeout=10)  # 超时设置为10秒except:for i in range(4):  # 循环去请求网站response = requests.get(url, headers=headers, timeout=20)if response.status_code == 200:breakhtml_str = response.textreturn html_str

第二种解决方法：构建headers列表每次随机选一个：

headers_list = [{'user-agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.0.0; SM-G955U Build/R16NW) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 10; SM-G981B) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/80.0.3987.162 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (iPad; CPU OS 13_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/87.0.4280.77 Mobile/15E148 Safari/604.1'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.109 Safari/537.36 CrKey/1.54.248666'}, {'user-agent': 'Mozilla/5.0 (X11; Linux aarch64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.188 Safari/537.36 CrKey/1.54.250320'}, {'user-agent': 'Mozilla/5.0 (BB10; Touch) AppleWebKit/537.10+ (KHTML, like Gecko) Version/10.0.9.2372 Mobile Safari/537.10+'}, {'user-agent': 'Mozilla/5.0 (PlayBook; U; RIM Tablet OS 2.1.0; en-US) AppleWebKit/536.2+ (KHTML like Gecko) Version/7.2.1.0 Safari/536.2+'}, {'user-agent': 'Mozilla/5.0 (Linux; U; Android 4.3; en-us; SM-N900T Build/JSS15J) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30'}, {'user-agent': 'Mozilla/5.0 (Linux; U; Android 4.1; en-us; GT-N7100 Build/JRO03C) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30'}, {'user-agent': 'Mozilla/5.0 (Linux; U; Android 4.0; en-us; GT-I9300 Build/IMM76D) AppleWebKit/534.30 (KHTML, like Gecko) Version/4.0 Mobile Safari/534.30'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 7.0; SM-G950U Build/NRD90M) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/62.0.3202.84 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.0.0; SM-G965U Build/R16NW) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.111 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.1.0; SM-T837A) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.80 Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; U; en-us; KFAPWI Build/JDQ39) AppleWebKit/535.19 (KHTML, like Gecko) Silk/3.13 Safari/535.19 Silk-Accelerated=true'}, {'user-agent': 'Mozilla/5.0 (Linux; U; Android 4.4.2; en-us; LGMS323 Build/KOT49I.MS32310c) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Windows Phone 10.0; Android 4.2.1; Microsoft; Lumia 550) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2486.0 Mobile Safari/537.36 Edge/14.14263'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 6.0.1; Moto G (4)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 10 Build/MOB31T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 4.4.2; Nexus 4 Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.0.0; Nexus 5X Build/OPR4.170623.006) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 7.1.1; Nexus 6 Build/N6F26U) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.0.0; Nexus 6P Build/OPP3.170518.006) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 6.0.1; Nexus 7 Build/MOB30X) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (compatible; MSIE 10.0; Windows Phone 8.0; Trident/6.0; IEMobile/10.0; ARM; Touch; NOKIA; Lumia 520)'}, {'user-agent': 'Mozilla/5.0 (MeeGo; NokiaN9) AppleWebKit/534.13 (KHTML, like Gecko) NokiaBrowser/8.5.0 Mobile Safari/534.13'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 9; Pixel 3 Build/PQ1A.181105.017.A1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/66.0.3359.158 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 10; Pixel 4) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/81.0.4044.138 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 11; Pixel 3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/88.0.4324.181 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 5.0; SM-G900P Build/LRX21T) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.0; Pixel 2 Build/OPD3.170816.012) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (Linux; Android 8.0.0; Pixel 2 XL Build/OPD1.170816.004) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/102.0.0.0 Mobile Safari/537.36'}, {'user-agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 10_3_1 like Mac OS X) AppleWebKit/603.1.30 (KHTML, like Gecko) Version/10.0 Mobile/14E304 Safari/602.1'}, {'user-agent': 'Mozilla/5.0 (iPhone; CPU iPhone OS 13_2_3 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/13.0.3 Mobile/15E148 Safari/604.1'}, {'user-agent': 'Mozilla/5.0 (iPad; CPU OS 11_0 like Mac OS X) AppleWebKit/604.1.34 (KHTML, like Gecko) Version/11.0 Mobile/15A5341f Safari/604.1'}
]headers = random.choice(headers_list)

第三种方法解决方法：放慢请求速度

time.sleep(10)

帮忙解决

本文已收录于：《告别Bug》专栏

本专栏用于记录学习和工作中遇到的各种疑难Bug问题，以及粉丝群里小伙伴提出的各种问题，文章形式：报错代码 + 报错翻译 + 报错原因 + 解决方法，包括程序安装、运行程序过程中等等问题，订阅专栏+关注博主后如遇到其他问题可私聊帮忙解决！！！

已解决（Python爬虫requests报错）requests.exceptions.ProxyError: HTTPSConnectionPool相关推荐

python安装报错类型_解决Python安装cryptography报错问题
解决Python安装cryptography报错问题错误一: gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wst ...
（已解决）wamp + Apache报错：出现403Forbidden　You don't have permission to access / on this server.
文章目录 `解决办法:` `*** `附件 · 延伸阅读 · 参考: 解决办法: 注意: 案例环境:win7 x64 或 win10 仅仅只需要修改apache配置文件httpd.conf 即可. 上 ...
python打开文件报错无效序列_解决Python 写文件报错TypeError的问题
处理上传的文件: f1 = request.FILES['pic'] fname = '%s/%s' % (settings.MEDIA_ROOT, f1.name) with open(fname, ...
python连接数据库mysql失败_解决python连接mysql报错问题
解决python连接mysql报错问题最近跑python脚本时遇到一个问题,当sql语句中有中文时,执行python脚本报以下错误: Traceback (most recent call last ...
python运行启动报错解决方法_解决python运行启动报错问题
解决python运行启动报错问题问题一: python启动报错api-ms-win-crt-process-l1-1-0.dll丢失解决: 下载api-ms-win-crt-process-l1- ...
已解决（MongoDB安装报错）Service ‘MongoDB Server (MongoDB)’ (MongoDB) failed tostart. Verify that you have su
成功解决(MongoDB安装报错):Service 'MongoDB Server (MongoDB)' (MongoDB) failed tostart. Verify that you have ...
【python爬虫】报错module requests has no attribute get
欢迎移步我的个人博客错误原因 python文件名命名为了requests,与代码中的库名重了导致报错. 解决方法类名.包名都不能与该包.类的代码中所用到的库名重复.起名字时候避免这种情况即可.
【已解决】导入tensorflow报错/python已停止工作/The kernel appears to have died
目录背景问题1:in future version of numpy, it will be understaood as(type,(1,)) 原因解决问题2:python已停止工作/The ...
解决Python打开文件报错UnicodeDecodeError: 'gbk' codec can't decode byte
用Python打开文件时报错: UnicodeDecodeError: 'gbk' codec can't decode byte 0x80 in position 10: illegal multi ...
解决python toad包报错joblib.externals.loky.process_executor.TerminatedWorkerError
当运行官方的toad.quality()时,无法得到结果,并抛出报错解决方法首先检查toad版本:我的是toad == 0.0.64,python3.6版本只需更换python版本为3.8,然后 ...

已解决（Python爬虫requests报错）requests.exceptions.ProxyError: HTTPSConnectionPool

文章目录