Python爬虫：中国结算，关于新开股票账户数等参数数据的爬取
2024-04-08 02:55:24
1数据来源：http://www.chinaclear.cn/zdjs/tjyb1/center_tjbg.shtml
2获取内容 ：09到至今的主要指标概览数据内容
3可参考代码，直接用就行
import requests
import re
import datetimedef get_month_range(start_day,end_day):months = (end_day.year - start_day.year)*12 + end_day.month - start_day.monthmonth_range = ['%s年%s月'%(start_day.year + mon//12,str(mon%12+1).zfill(2)) for mon in range(start_day.month-1,start_day.month + months)]return month_rangedef spider(date_list):for i in date_list:date = int(i.replace('年','').replace('月',''))if date >=200904 and date < 201001 :Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.textget_data = r'<tr style=.*?>.*?<td width="277" .*?><font .*?>.*?</font>.*?<p .*?><span .*?>(.*?)</span></p>.*?</font></td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[3]end_investors = data[2]registered_securities_number = data[4]registered_securities_totalparvalue = data[5]registered_securities_totalmarketvalue = data[6]non_restricted_market_value = data[7]total_number_of_transfers = data[9]total_amount_of_transfer = data[10]total_settlement = data[11]net_settlement = data[12]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date == 201001:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.textget_data = r'<tr style=.*?>.*?<td .*?>.*?<p align="right" .*?><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)month = data[0]new_investors = data[3]end_investors = data[2]registered_securities_number = data[4]registered_securities_totalparvalue = data[5]registered_securities_totalmarketvalue = data[6]non_restricted_market_value = data[7]total_number_of_transfers = data[9]total_amount_of_transfer = data[10]total_settlement = data[11]net_settlement = data[12]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date >201001 and date <=201311:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.textget_data = r'<tr style="height:13.5pt">.*?<td .*?>.*?<p .*?><span .*?>.*?</span></p>.*?</td>.*?<td .*?>.*?<p .*? align="right"><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[3]end_investors = data[2]registered_securities_number = data[4]registered_securities_totalparvalue = data[5]registered_securities_totalmarketvalue = data[6]non_restricted_market_value = data[7]total_number_of_transfers = data[9]total_amount_of_transfer = data[10]total_settlement = data[11]net_settlement = data[12]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date == 201312:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style="height: 13.5pt;">.*?<td width="19%" .*?>.*?<p align="right" .*?><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[3]end_investors = data[2]registered_securities_number = data[4]registered_securities_totalparvalue = data[5].replace('<span> </span>','')registered_securities_totalmarketvalue = data[6]non_restricted_market_value = data[7].replace('<span> </span>','')total_number_of_transfers = data[9]total_amount_of_transfer = data[10]total_settlement = data[11]net_settlement = data[12]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date == 201401:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td width="19%" .*?>.*?<p .*? align="right"><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[3]end_investors = data[2]registered_securities_number = data[4]registered_securities_totalparvalue = data[5].replace('<span> </span>','')registered_securities_totalmarketvalue = data[6]non_restricted_market_value = data[7].replace('<span> </span>','')total_number_of_transfers = data[9]total_amount_of_transfer = data[10]total_settlement = data[11]net_settlement = data[12]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date > 201401 and date < 201410:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td width="20%" .*?>.*?<p .*?><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[3]end_investors = data[2]registered_securities_number = data[4]registered_securities_totalparvalue = data[5].replace('<span> </span>', '')registered_securities_totalmarketvalue = data[6]non_restricted_market_value = data[7].replace('<span> </span>', '')total_number_of_transfers = data[9]total_amount_of_transfer = data[10]total_settlement = data[11]net_settlement = data[12]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date >= 201410 and date <= 201412:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td width="20%" .*?>.*?<p .*?><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[4]end_investors = data[2]registered_securities_number = data[5]registered_securities_totalparvalue = data[6]registered_securities_totalmarketvalue = data[7]non_restricted_market_value = data[8]total_number_of_transfers = data[10]total_amount_of_transfer = data[11]total_settlement = data[12]net_settlement = data[13]print(month, new_investors, end_investors, registered_securities_number, registered_securities_totalparvalue,registered_securities_totalmarketvalue, non_restricted_market_value, total_number_of_transfers,total_amount_of_transfer, total_settlement, net_settlement)elif date >= 201501 and date <201503:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data1 = r'<tr style=.*?>.*?<td .*?>.*?<p .*? align=.*?><span .*?>.*?</span></p>.*?</td>.*?<td .*?>.*?<p .*? align=.*?><span .*?>(.*?)</span></p>.*?</td>.*?<td .*?>.*?<p .*? align=.*?><span .*?>.*?</span></p>.*?</td>.*?</tr>'pattern1 = re.compile(get_data1, re.I | re.S | re.M)data1 = pattern1.findall(html)get_data = r'<tr style=.*?>.*?<td .*?>.*?<p .*? align="left"><span .*?>.*?</span></p>.*?</td>.*?.*?<td .*?>.*?<p .*? align="right"><span .*?>(.*?)</span></p>.*?</td>.*?.*?<td .*?>.*?<p .*? align="right"><span .*?>.*?</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)month = data1[0]new_investors = data[3]end_investors = data[1]registered_securities_number = data[4]registered_securities_totalparvalue = data[5]registered_securities_totalmarketvalue = data[6]non_restricted_market_value = data[7]total_number_of_transfers = data[9]total_amount_of_transfer = data[10]total_settlement = data[11]net_settlement = data[12]print(month, new_investors, end_investors, registered_securities_number, registered_securities_totalparvalue,registered_securities_totalmarketvalue, non_restricted_market_value, total_number_of_transfers,total_amount_of_transfer, total_settlement, net_settlement)elif date == 201503:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td .*?>.*?<p .*?><span .*?>.*?</span></p>.*?</td>.*?<td .*?>.*?<p .*? align=.*?><span .*?>(.*?)</span></p>.*?</td>.*?<td .*?>.*?<p .*? align=.*?><span .*?>.*?</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)month = data[0]new_investors = data[3]end_investors = data[1]registered_securities_number = data[4]registered_securities_totalparvalue = data[5]registered_securities_totalmarketvalue = data[6]non_restricted_market_value = data[7]total_number_of_transfers = data[9]total_amount_of_transfer = data[10]total_settlement = data[11]net_settlement = data[12]print(month, new_investors, end_investors, registered_securities_number, registered_securities_totalparvalue,registered_securities_totalmarketvalue, non_restricted_market_value, total_number_of_transfers,total_amount_of_transfer, total_settlement, net_settlement)elif date > 201503 and date <=201506:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td width="100" .*?>.*?<p .*?><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[1]end_investors = data[2]registered_securities_number = data[5]registered_securities_totalparvalue = data[6]registered_securities_totalmarketvalue = data[7]non_restricted_market_value = data[8]total_number_of_transfers = data[10]total_amount_of_transfer = data[11]total_settlement = data[12]net_settlement = data[13]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date == 201507:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td width="158" .*?>.*?<p .*?><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[1]end_investors = data[2]registered_securities_number = data[3]registered_securities_totalparvalue = data[4]registered_securities_totalmarketvalue = data[5]non_restricted_market_value = data[6]total_number_of_transfers = data[8]total_amount_of_transfer = data[9]total_settlement = data[10]net_settlement = data[11]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date == 201508:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td width="100" .*?>.*?<p .*?><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[1]end_investors = data[2]registered_securities_number = data[3]registered_securities_totalparvalue = data[4]registered_securities_totalmarketvalue = data[5]non_restricted_market_value = data[6]total_number_of_transfers = data[8]total_amount_of_transfer = data[9]total_settlement = data[10]net_settlement = data[11]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date == 201509:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td width="158" .*?>.*?<p .*?><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[1]end_investors = data[2]registered_securities_number = data[3]registered_securities_totalparvalue = data[4]registered_securities_totalmarketvalue = data[5]non_restricted_market_value = data[6]total_number_of_transfers = data[8]total_amount_of_transfer = data[9]total_settlement = data[10]net_settlement = data[11]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date > 201509 and date <= 201511:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td width="26%" .*?>.*?<p .*?><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[1]end_investors = data[2]registered_securities_number = data[3]registered_securities_totalparvalue = data[4]registered_securities_totalmarketvalue = data[5]non_restricted_market_value = data[6]total_number_of_transfers = data[8]total_amount_of_transfer = data[9]total_settlement = data[10]net_settlement = data[11]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date == 201512:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td width="27%" .*?>.*?<p .*?><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[1]end_investors = data[2]registered_securities_number = data[3]registered_securities_totalparvalue = data[4]registered_securities_totalmarketvalue = data[5]non_restricted_market_value = data[6]total_number_of_transfers = data[8]total_amount_of_transfer = data[9]total_settlement = data[10]net_settlement = data[11]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date == 201601:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td width="29%" .*?>.*?<p .*?><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[1]end_investors = data[2]registered_securities_number = data[3]registered_securities_totalparvalue = data[4]registered_securities_totalmarketvalue = data[5]non_restricted_market_value = data[6]total_number_of_transfers = data[8]total_amount_of_transfer = data[9]total_settlement = data[10]net_settlement = data[11]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date > 201601 and date <=201607:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td width="26%" .*?>.*?<p .*?><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[1]end_investors = data[2]registered_securities_number = data[3]registered_securities_totalparvalue = data[4]registered_securities_totalmarketvalue = data[5]non_restricted_market_value = data[6]total_number_of_transfers = data[8]total_amount_of_transfer = data[9]total_settlement = data[10]net_settlement = data[11]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date ==201608:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td .*? width="142" noWrap="">.*?<p .*?><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[1]end_investors = data[2]registered_securities_number = data[3].replace('<span> </span>','').strip()registered_securities_totalparvalue = data[4].replace('<span> </span>','').strip()registered_securities_totalmarketvalue = data[5].replace('<span> </span>','').strip()non_restricted_market_value = data[6].replace('<span> </span>','').strip()total_number_of_transfers = data[8]total_amount_of_transfer = data[9]total_settlement = data[10]net_settlement = data[11]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date >= 201609 and date<=201610:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td width="26%" .*?>.*?<p .*?><span .*?>(.*?)</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[1]end_investors = data[2]registered_securities_number = data[3].replace(' ','').replace('<span>','').replace('</span>','').strip()registered_securities_totalparvalue = data[4].replace(' ','').replace('<span>','').replace('</span>','').strip()registered_securities_totalmarketvalue = data[5].replace(' ','').replace('<span>','').replace('</span>','').strip()non_restricted_market_value = data[6].replace(' ','').replace('<span>','').replace('</span>','').strip()total_number_of_transfers = data[8]total_amount_of_transfer = data[9]total_settlement = data[10]net_settlement = data[11]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date == 201611:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td width="26%" .*?><span style=.*?><font .*?>.*?</font></span>.*?<p align="right" .*?><span style=.*?><font .*?>(.*?)</font></span></p>.*?<span style=.*?><font .*?>.*?</font></span></td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[1]end_investors = data[2]registered_securities_number = data[3].replace(' ','').replace('<span style="font-size: 10.5pt;">','').replace('</span>','').strip()registered_securities_totalparvalue = data[4].replace(' ','').replace('<span style="font-size: 10.5pt;">','').replace('</span>','').strip()registered_securities_totalmarketvalue = data[5].replace(' ','').replace('<span style="font-size: 10.5pt;">','').replace('</span>','').strip()non_restricted_market_value = data[6]total_number_of_transfers = data[8]total_amount_of_transfer = data[9]total_settlement = data[10]net_settlement = data[11]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date == 201612:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr style=.*?>.*?<td width="26%" .*?><span style=.*?>.*?</span>.*?<p align="right" .*?><span .*?>(.*?)</span></p>.*?<span style=.*?>.*?</span></td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[1]end_investors = data[2].replace('<span>','').strip()registered_securities_number = data[3].replace('<span>','').replace(' ','').replace('<span style="font-size: 10.5pt;">','').replace('</span>','').strip()registered_securities_totalparvalue = data[4].replace('<span>','').replace(' ','').replace('<span style="font-size: 10.5pt;">','').replace('</span>','').strip()registered_securities_totalmarketvalue = data[5].replace('<span>','').replace(' ','').replace('<span style="font-size: 10.5pt;">','').replace('</span>','').strip()non_restricted_market_value = data[6]total_number_of_transfers = data[8]total_amount_of_transfer = data[9]total_settlement = data[10]net_settlement = data[11]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date == 201701:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = '<tbody>.*?<tr style=.*?>.*?<td .*?>.*?</td>.*?<td .*?>.*?<p .*?><span style=.*?>(.*?)</span></p>.*?</td>.*?<td .*?>.*?</td>.*?</tr>.*?<tr style=.*?>.*?<td .*?>.*?</td>.*?<td .*?>.*?<p .*?><span style=.*?>(.*?)</span></p>.*?</td>.*?<td .*?>.*?</td>.*?</tr>.*?</tbody>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)month = data[0][0]new_investors = data[0][1].replace('<span>', '').replace('</span>', '').replace(' ', '').strip()get_data_one = '<tr .*?>.*?<td .*?>.*?<p .*?>.*?<span .*?>.*?</span>.*?</p>.*?</td>.*?<td .*?>.*?<p .*?>.*?<span style=.*?>.*?<span>.*?</span>(.*?)</span>.*?</p>.*?</td>.*?</tr>'pattern_one = re.compile(get_data_one, re.I | re.S | re.M)data_one = pattern_one.findall(html)[1:-1]# print(data_one)end_investors = data_one[0]registered_securities_number = data_one[1]registered_securities_totalparvalue = data_one[2]registered_securities_totalmarketvalue = data_one[3]non_restricted_market_value = data_one[4]total_number_of_transfers = data_one[5]total_amount_of_transfer = data_one[6]total_settlement = data_one[6]get_data_second ='<td .*?>.*?<p .*?>.*?<span .*?>.*?<span>.*?</span>.*?<span>.*?</span>(.*?)</span>.*?</p>.*?</td>'pattern_second = re.compile(get_data_second , re.I | re.S | re.M)net_settlement = pattern_second.findall(html)[-2]print(month,new_investors,end_investors,registered_securities_number,registered_securities_totalparvalue,registered_securities_totalmarketvalue,non_restricted_market_value,total_number_of_transfers,total_amount_of_transfer,total_settlement,net_settlement)elif date > 201701 and date <= 201705:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url,headers=Headers, data=data)response.encoding= 'utf-8'html = response.text# print(html)get_data = '<tbody>.*?<tr style=.*?>.*?<td .*?>.*?</td>.*?<td .*?>.*?<p .*?><span style=.*?>(.*?)</span></p>.*?</td>.*?<td .*?>.*?</td>.*?</tr>.*?<tr style=.*?>.*?<td .*?>.*?</td>.*?<td .*?>.*?<p .*?><span style=.*?>(.*?)</span></p>.*?</td>.*?<td .*?>.*?</td>.*?</tr>.*?</tbody>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)month = data[0][0]get_data1 = '<tr .*?>.*?<td .*?>.*?<p .*?>.*?<span .*?>.*?</span>.*?</p>.*?</td>.*?<td .*?>.*?<p .*?>.*?<span style=.*?>.*?<span>.*?</span>(.*?)</span>.*?</p>.*?</td>.*?</tr>'pattern1 = re.compile(get_data1, re.I | re.S | re.M)data1 = pattern1.findall(html)[1:]new_investors = data1[0]end_investors = data1[1]registered_securities_number = data1[2]registered_securities_totalparvalue = data1[3]registered_securities_totalmarketvalue = data1[4]non_restricted_market_value = data1[5]total_number_of_transfers = data1[6]total_amount_of_transfer = data1[7]total_settlement = data1[8]net_settlement = data1[9]print(month,new_investors,end_investors,registered_securities_number,registered_securities_totalparvalue,registered_securities_totalmarketvalue,non_restricted_market_value,total_number_of_transfers,total_amount_of_transfer,total_settlement,net_settlement)elif date >= 201706 and date <201709:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.textget_data = '<tr style=.*?>.*?<td .*?>.*?<p .*?><span style=.*?>.*?</span></p>.*?</td>.*?<td .*?>.*?<p .*?><span style=.*?>(.*?)</span></p>.*?</td>.*?<td .*?>.*?<p .*?><span style=.*?>.*?</span></p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)month = data[0]new_investors = data[1]end_investors = data[2]registered_securities_number = data[3].replace('<span>', '').replace('</span>', '').replace(' ', '').strip()registered_securities_totalparvalue = data[4].replace('<span>', '').replace('</span>', '').replace(' ', '').strip()registered_securities_totalmarketvalue = data[5].replace('<span>', '').replace('</span>', '').replace(' ', '').strip()non_restricted_market_value = data[6].replace('<span>', '').replace('</span>', '').replace(' ', '').strip()total_number_of_transfers = data[7]total_amount_of_transfer = data[8]total_settlement = data[9]net_settlement = data[10]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)elif date >= 201709:Headers = {'Accept': 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Encoding': 'gzip, deflate','Accept-Language': 'zh-CN,zh;q=0.9','Cache-Control': 'max-age=0','Connection': 'keep-alive','Content-Length': '122','Content-Type': 'application/x-www-form-urlencoded','Cookie': 'JSESSIONID=00005q0oN93pCb5mAK5eZQGAa7t:1amj63rte','Host': 'www.chinaclear.cn','Origin': 'http://www.chinaclear.cn','Referer': 'http://www.chinaclear.cn/cms-search/monthview.action?action=china','Upgrade-Insecure-Requests': '1','User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/86.0.4240.193 Safari/537.36'}data = {'riqi': '{0}'.format(i),'channelFidStr': 'e990411f19544e46be84333c25b63de6','channelIdStr': 'bd095cc08f744c089b159a3bb744b9d0'}url = 'http://www.chinaclear.cn/cms-search/monthview.action?action=china'response = requests.post(url, headers=Headers, data=data)response.encoding = 'utf-8'html = response.text# print(html)get_data = r'<tr .*?>.*?<td .*?>.*?<p .*?>.*?<span .*?>.*?</span>.*?</p>.*?</td>.*?<td .*?>.*?<p .*?>.*?<span .*?>(.*?)</span>.*?</p>.*?</td>.*?</tr>'pattern = re.compile(get_data, re.I | re.S | re.M)data = pattern.findall(html)# print(data)month = data[0]new_investors = data[1]end_investors = data[2]registered_securities_number = data[3].replace('<span> ','')registered_securities_totalparvalue = data[4].strip().replace('<span style="font-size:10.5pt;font-family:宋体;color:#424242;">','').replace('<span style="font-size: 10.5pt; font-family: 宋体; color: rgb(66, 66, 66);">','').replace('<span style="font-size:10.5pt;font-family: 宋体;color:#424242;">','').replace(' <span style="font-size: 9.0pt;font-family:宋体;color:#424242;">','')registered_securities_totalmarketvalue = data[5]non_restricted_market_value = data[6]total_number_of_transfers = data[7]total_amount_of_transfer = data[8]total_settlement = data[9]net_settlement = data[10]print(month, new_investors, end_investors, registered_securities_number,registered_securities_totalparvalue, registered_securities_totalmarketvalue,non_restricted_market_value, total_number_of_transfers, total_amount_of_transfer, total_settlement,net_settlement)else:passif __name__ == '__main__':date_list = get_month_range(datetime.date(2005, 1, 31),datetime.date(2020,9,1))spider(date_list)Python爬虫：中国结算，关于新开股票账户数等参数数据的爬取相关推荐
- python爬虫！ 网站维护人员：真的求求你们了，不要再来爬取了，受不了了！！
话说 在很久很久以前 小明不小心发现了一个叫做 学习python的正确姿势 从此一发不可收拾 看到什么网站都想爬取 有一天 小明发现了一个小黄网 里面的小姐姐 一个比一个诱人 看了一会这个小黄网 小明 ...
- Python爬虫入门教程【4】：美空网未登录图片爬取
美空网未登录图片----简介 上一篇写的时间有点长了,接下来继续把美空网的爬虫写完,这套教程中编写的爬虫在实际的工作中可能并不能给你增加多少有价值的技术点,因为它只是一套入门的教程,老鸟你自动绕过就可 ...
- Python爬虫自学之第（③）篇——实战：requests+BeautifulSoup实现静态爬取
题外话: <Pi Network 免费挖矿国外热门项目 一个π币大约值3元到10元>相信过去BTC的人,信不信未来的PI,了解一下,唯一一个高度与之持平的项目 前篇全片都是生硬的理论使用, ...
- python爬取网页json数据_python爬取json数据库
手把手教你使用Python抓取QQ音乐数据(第一弹) [一.项目目标] 获取 QQ 音乐指定歌手单曲排行指定页数的歌曲的歌名.专辑名.播放链接. 由浅入深,层层递进,非常适合刚入门的同学练手. [二. ...
- Python 爬虫 中国行政区划信息爬取 （初学者）
Python 爬虫 中国行政区划信息爬取 (初学者) 背景 环境准备 代码片段 1.定义地址信息对象 2.地址解析对象 2.1 获取web信息 2.2 web信息解析 2.3 区划信息提取 2.4 省 ...
- 使用Python爬虫获取上交所和深交所所有股票的名称和交易信息
使用Python爬虫获取上交所和深交所所有股票的名称和交易信息 功能描述 目标 获取上交所和深交所所有股票的名称和交易信息 输出 保存到文件中 技术路线 requests-bs4-re 候选数据网站的 ...
- Python 爬虫中国知网论文过程中遇到的坑及解决办法
假期,老师给布置了 Python 爬虫中国知网论文的任务,目前实现了登录和搜索功能,先写一下遇到的坑和解决办法吧. Python 爬虫中国知网论文过程中遇到的坑及解决办法 一. selenium 模块 ...
- python爬虫之股票数据定向爬取
python爬虫之股票数据定向爬取 功能描述 目标:获取上交所和深交所所有股票的名称和交易的信息 输出:保存到文件中 技术路线:requests-bs4-re 前期分析 选取原则:股票的信息静态存在H ...
- 股票数据定向爬取（可运行） Python网络爬虫与信息提取（北京理工大学—嵩天）
注意:由于东方财富网与百度股票网站发生变动,所以选择了与原先百度股票类似的股城网作为信息爬取的目标网站.(感谢文章:(4条消息) Python爬虫入门实例八之股票数据定向爬取并保存(优化版)_Mr.Q ...
最新文章
- sql join 与where的区别
- python怎么改界面颜色_Python的matplotlib绘图如何修改背景颜色的实现
- k8s 为何成为大厂标配？
- 2017年国内开源镜像站点汇总 1
- oracle 统计文本字符串,oracle人员权限，字符串转列，统计管理详解-Oracle
- 机器学习实战(MachineLearinginAction) 第三章 决策树
- 浮动和清除（闭合）浮动
- 讨老婆之前要记熟的十句话
- Mybatis简单入门及配置文件标签详情
- 【信号与系统】三大变换公式表 | 傅里叶变换 | 拉普拉斯变换 | Z变换
- Atitit r2017 r4 doc list on home ntpc.docx 驱动器 D 中的卷是 p2soft 卷的序列号是 9AD0-D3C8 D:\ati ext\r2017
- word文档单独删除或修改首页或某一页的页眉页脚及其下划线
- java使用爬虫工具jsoup实现抓取网页的内容及图片并写入到word文档中
- 点阵字体显示系列之二：汉字显示
- COVID应对小tips
- 17种数据可视化图表，有哪些适用场景？
- 文本编辑工具 | Editplus_v5.5 +汉化包，用于java、C/C++的语言工具
- 如何完美解决Sqoop导入导出MySQL数据错位问题
- 取带runas的一些优秀小工具介绍
- 基于路网和GeoPandas的高斯两步移动搜索法可达性分析