Python爬取百度指数搜索结果，查看你想了解的热点信息吧

今天无意间搜索问题的时候跳转到了百度指数这里，索性就打开来看看，下面是首页截图：

这里你可以自己输入自己想要查询的人物、事件等等，anything，只要是你感兴趣的都可以，有一种感觉就是你认为是热点的就是热点。。。。

闲话不多说了，这里直接进入实践，先看代码，完整的实现如下：

#!usr/bin/env python
#encoding:utf-8'''
__Author__:沂水寒城
功能： 爬取百度指数结果搜索入口地址：http://index.baidu.com/v2/index.html#/'''import os
import sys
import json
import time
import random
import requests
import datetimeif sys.version_info < (3, 0):reload(sys)sys.setdefaultencoding("utf-8")month_day_dict = {"01": 31,"02": 28,"03": 31,"04": 30,"05": 31,"06": 30,"07": 31,"08": 31,"09": 30,"10": 31,"11": 30,"12": 31,}def generateMonthDays(month_day_dict,year='2017',month='03'):'''生成指定年份、月份中的所有日期'''day_num=month_day_dict[month]day_date_list=[]for i in range(1,day_num+1):one=str(i)if len(one)==1:one='0'+one day_date_list.append(year+'-'+month+'-'+one)return day_date_listdef generatePeriodDays(start="2019-11-01", end="2019-11-31"):"""生成指定时间段内所有的 天 日期"""start_year, start_mon = start.split("-")[0].strip(), start.split("-")[1].strip()all_day_list = generateMonthDays(month_day_dict, year=start_year, month=start_mon)end_year, end_mon = start.split("-")[0].strip(), start.split("-")[1].strip()all_day_list += generateMonthDays(month_day_dict, year=end_year, month=end_mon)day_list = [one for one in all_day_list if one >= start and one <= end]return sorted(list(set(day_list)))def reqData(start="2019-11-01", end="2019-12-03", kw="海康威视"):'''网络请求数据'''date_list = generatePeriodDays(start = start, end = end)date_str = ','.join(date_list)search_url = urlTemplate.format(date_str, kw)data = requests.get(search_url, timeout=5)content = data.json()return contentdef dataParser(content):'''数据解析'''data_list= content['data']["追我吧"]result = []key_list = ['date', 'title', 'url', 'source', 'same_news']for one_dict in data_list:T = one_dict['date']news_list = one_dict['news']for one_son_dict in news_list:one_tmp_list = []for i in range(len(key_list)):try:one_tmp_list.append(one_son_dict[key_list[i]])except:one_tmp_list.append(None)result.append(one_tmp_list)result = sorted(result, key = lambda e:e[0])result.insert(0, key_list)for one in result:print oneif __name__ == '__main__':content = reqData(start="2019-11-01", end="2019-12-03", kw="追我吧")dataParser(content)

最近《追我吧》这个节目组还是上了好久的热点头条的，这里我就搜索“追我吧”，结果如下：

{"status":0,"data":{"追我吧":[{"date":"2019-11-24","news":[{"date":"2019-11-24","title":"《追我吧<\/em>》这期大咖云集,跑男家族相聚一起,阵容再变引起热议","url":"https:\/\/new.qq.com\/omn\/20191124\/20191124A00PJP.html?pc","source":"腾讯新闻","same_news":2}]},{"date":"2019-11-25","news":[{"date":"2019-11-25","title":"《追我吧<\/em>》VS跑男家族团魂"开战" 喜提收视三连冠","url":"http:\/\/news.gxnews.com.cn\/staticpages\/20191125\/newgx5ddb716d-19060415.shtml","source":"广西新闻网","same_news":5},{"date":"2019-11-25","title":"《追我吧<\/em>》VS跑男家族团魂“开战”引爆多平台喜提收视三连冠","url":"http:\/\/www.dzwww.com\/yule\/zy\/201911\/t20191125_19393325.htm","source":"大众网","same_news":3},{"date":"2019-11-25","title":"《追我吧<\/em>》下期放大招!请来两大流量男团,粉丝都开心坏了","url":"https:\/\/new.qq.com\/omn\/20191125\/20191125A057NX.html?pc","source":"腾讯新闻","same_news":1}]},{"date":"2019-11-26","news":[{"date":"2019-11-26","title":"跑男团神武3战队双双踢馆追我吧<\/em> “踢”出周末最高收视","url":"http:\/\/www.cet.com.cn\/xwsd\/2426554.shtml","source":"中国经济新闻网","same_news":6},{"date":"2019-11-26","title":"《追我吧<\/em>》VS跑男家族团魂“开战” 引爆多平台喜提收视三连冠","url":"http:\/\/ent.ynet.com\/2019\/11\/26\/2226817t1254.html","source":"北青网","same_news":2},{"date":"2019-11-26","title":"《追我吧<\/em>》190长腿追跑者 中国山地车攀爬第一人张京坤","url":"http:\/\/www.cnr.cn\/rdzx\/cxxhl\/zxxx\/20191126\/t20191126_524873496.shtml","source":"中国广播网","same_news":1}]},{"date":"2019-11-27","news":[{"date":"2019-11-27","title":"高以翔去世是什么原因?追我吧<\/em>凌晨录制极限项目被质疑安全性","url":"http:\/\/www.mnw.cn\/news\/ent\/2224569.html","source":"闽南网","same_news":5},{"date":"2019-11-27","title":"钟楚曦回应追我吧<\/em>难度和强度:连吃3天速效救心丸","url":"http:\/\/ent.sina.com.cn\/tv\/zy\/2019-11-27\/doc-iihnzahi3716875.shtml","source":"新浪","same_news":4},{"date":"2019-11-27","title":"高以翔出事的《追我吧<\/em>》,李小鹏喊累邹市明要吸氧 高强度节目如此...","url":"http:\/\/www.dzwww.com\/yule\/nd\/201911\/t20191127_19400574.htm","source":"大众网","same_news":4}]},{"date":"2019-11-28","news":[{"date":"2019-11-28","title":"追我吧<\/em>结束录制 陈伟霆张继科离开剧组","url":"https:\/\/news.china.com\/socialgd\/10000169\/20191128\/37471028.html","source":"中华网","same_news":10},{"date":"2019-11-28","title":"探访高以翔节目录制地:《追我吧<\/em>》停录,部分设施被拆除","url":"http:\/\/news.sina.com.cn\/c\/2019-11-28\/doc-iihnzhfz2240506.shtml","source":"新浪新闻","same_news":8},{"date":"2019-11-28","title":"浙江卫视终于发声:为高以翔猝死负责!徐峥痛斥!《追我吧<\/em>》节目组有...","url":"http:\/\/finance.ifeng.com\/c\/7ryAnVtT1A7","source":"凤凰网","same_news":6}]},{"date":"2019-11-29","news":[{"date":"2019-11-29","title":"网曝《追我吧<\/em>》负责人因高以翔事件被全部开除","url":"http:\/\/ent.sina.com.cn\/tv\/zy\/2019-11-29\/doc-iihnzhfz2423924.shtml","source":"新浪","same_news":14},{"date":"2019-11-29","title":"浙江卫视宣布高以翔参录综艺《追我吧<\/em>》今晚停播","url":"http:\/\/ent.163.com\/19\/1129\/15\/EV5K41ET00038FO9.html","source":"网易","same_news":6},{"date":"2019-11-29","title":"《追我吧<\/em>》正常播出?知情人士:非新一期节目","url":"http:\/\/ent.sina.com.cn\/tv\/zy\/2019-11-29\/doc-iihnzahi4134718.shtml","source":"新浪","same_news":4}]},{"date":"2019-11-30","news":[{"date":"2019-11-30","title":"疑《追我吧<\/em>》工作人员发声:带节奏的请消停一点","url":"http:\/\/ent.163.com\/19\/1130\/07\/EV7CIOE600038FO9.html","source":"网易","same_news":3},{"date":"2019-11-30","title":"邹市明参加追我吧<\/em>腿部失去知觉,粉丝:轩轩爸爸命真大","url":"http:\/\/m.sohu.com\/a\/357415896_403336","source":"搜狐","same_news":2},{"date":"2019-11-30","title":"凭借《奔跑吧》走红的宋雨琦人气不再,参加《追我吧<\/em>》如今遭停播","url":"https:\/\/new.qq.com\/omn\/20191130\/20191130A0JQF900","source":"腾讯新闻","same_news":1}]}]},"message":""}

接下来搜索一下“海康威视”，结果如下：

{"status": 0,"data": {"海康威视": [{"date": "2019-11-20","news": [{"date": "2019-11-20","title": "海康威视</em>高级副总经理蒋海青辞职 年薪为248.62万元","url": "http://finance.eastmoney.com/a/201911201296670767.html","source": "东方财富网","same_news": 3},{"date": "2019-11-20","title": "...高新技术企业创新能力百强榜发布,士兰微、新华三、海康威视</em>等...","url": "https://laoyaoba.com/html/share/news?source","source": "集微网","same_news": 1},{"date": "2019-11-20","title": "海康威视</em>与国网杭州供电公司签订战略合作协议","url": "http://www.afzhan.com/news/detail/78811.html","source": "中国安防展览网","same_news": 1}]},{"date": "2019-11-22","news": [{"date": "2019-11-22","title": "海康威视</em>:融资净偿还8784.07万元,两市排名第三(11-21)","url": "http://stock.eastmoney.com/a/201911221299836442.html","source": "东方财富网","same_news": 2},{"date": "2019-11-22","title": "后安防时代,海康威视</em>的信心与底气在哪? ","url": "http://www.techsir.com/a/201911/59129.html","source": "Techsir","same_news": 2}]},{"date": "2019-11-25","news": [{"date": "2019-11-25","title": "海康威视</em>闪耀2019首届四川教育装备博览会 ","url": "http://www.ceiea.com/html/201911/201911251324011299.shtml","source": "中国教育..","same_news": 1},{"date": "2019-11-25","title": "湖南广电、网易、海康威视</em>都来了!浙传这场招聘会让2000多人达成...","url": "http://edu.zjol.com.cn/jyjsb/gx/201911/t20191125_11397473.shtml","source": "浙江在线","same_news": 1},{"date": "2019-11-25","title": "海康威视</em>龚虹嘉的财富密码:十年暴赚两万倍后,为何被查?","url": "http://finance.sina.com.cn/stock/relnews/cn/2019-11-25/doc-iihnzahi3264395.shtml","source": "新浪","same_news": 1}]},{"date": "2019-11-29","news": [{"date": "2019-11-29","title": "龚虹嘉退胡扬忠进 海康威视</em>玩“对敲”?","url": "http://finance.sina.com.cn/stock/s/2019-11-29/doc-iihnzhfz2363840.shtml","source": "新浪","same_news": 3},{"date": "2019-11-29","title": "海康威视</em>:融资净偿还110.94万元,融资余额16.31亿元(11-28)","url": "http://stock.eastmoney.com/a/201911291307749335.html","source": "东方财富网","same_news": 1},{"date": "2019-11-29","title": "投资者提问:标普:将海康威视</em>移出观察名单,因供应链恢复。海康是被...","url": "http://finance.sina.com.cn/stock/relnews/dongmiqa/2019-11-29/doc-iihnzhfz2562007.shtml?source","source": "新浪","same_news": 1}]},{"date": "2019-12-02","news": [{"date": "2019-12-02","title": "海康威视</em>:关于部分国有股权无偿划转完成的公告","url": "http://finance.sina.com.cn/roll/2019-12-02/doc-iihnzhfz3131565.shtml","source": "新浪","same_news": 3},{"date": "2019-12-02","title": "海康威视</em>如何构建智慧农业的基石?","url": "http://www.afzhan.com/news/detail/79064.html","source": "中国安防展览网","same_news": 1},{"date": "2019-12-02","title": "[公司]海康威视</em>:实控人完成划转公司0.22%国有股权","url": "http://www.p5w.net/kuaixun/201912/t20191202_2358966.htm","source": "全景网","same_news": 1}]}]},"message": ""
}

格式化解析结果如下：

['date', 'title', 'url', 'source', 'same_news']
[u'2019-11-20', u'海康威视</em>高级副总经理蒋海青辞职 年薪为248.62万元', u'http://finance.eastmoney.com/a/201911201296670767.html', u'东方财富网', 3]
[u'2019-11-20', u'...高新技术企业创新能力百强榜发布,士兰微、新华三、海康威视</em>等...', u'https://laoyaoba.com/html/share/news?source', u'集微网', 1]
[u'2019-11-20', u'海康威视</em>与国网杭州供电公司签订战略合作协议', u'http://www.afzhan.com/news/detail/78811.html', u'中国安防展览网', 1]
[u'2019-11-22', u'海康威视</em>:融资净偿还8784.07万元,两市排名第三(11-21)', u'http://stock.eastmoney.com/a/201911221299836442.html', u'东方财富网', 2]
[u'2019-11-22', u'后安防时代,海康威视</em>的信心与底气在哪? ', u'http://www.techsir.com/a/201911/59129.html', u'techsir', 2]
[u'2019-11-25', u'海康威视</em>闪耀2019首届四川教育装备博览会 ', u'http://www.ceiea.com/html/201911/201911251324011299.shtml', u'中国教育..', 1]
[u'2019-11-25', u'湖南广电、网易、海康威视</em>都来了!浙传这场招聘会让2000多人达成...', u'http://edu.zjol.com.cn/jyjsb/gx/201911/t20191125_11397473.shtml', u'浙江在线', 1]
[u'2019-11-25', u'海康威视</em>龚虹嘉的财富密码:十年暴赚两万倍后,为何被查?', u'http://finance.sina.com.cn/stock/relnews/cn/2019-11-25/doc-iihnzahi3264395.shtml', u'新浪', 1]
[u'2019-11-29', u'龚虹嘉退胡扬忠进 海康威视</em>玩“对敲”?', u'http://finance.sina.com.cn/stock/s/2019-11-29/doc-iihnzhfz2363840.shtml', u'新浪', 3]
[u'2019-11-29', u'海康威视</em>:融资净偿还110.94万元,融资余额16.31亿元(11-28)', u'http://stock.eastmoney.com/a/201911291307749335.html', u'东方财富网', 1]
[u'2019-11-29', u'投资者提问:标普:将海康威视</em>移出观察名单,因供应链恢复。海康是被...', u'http://finance.sina.com.cn/stock/relnews/dongmiqa/2019-11-29/doc-iihnzhfz2562007.shtml?source', u'新浪', 1]
[u'2019-12-02', u'海康威视</em>:关于部分国有股权无偿划转完成的公告', u'http://finance.sina.com.cn/roll/2019-12-02/doc-iihnzhfz3131565.shtml', u'新浪', 3]
[u'2019-12-02', u'海康威视</em>如何构建智慧农业的基石?', u'http://www.afzhan.com/news/detail/79064.html', u'中国安防展览网', 1]
[u'2019-12-02', u'[公司]海康威视</em>:实控人完成划转公司0.22%国有股权', u'http://www.p5w.net/kuaixun/201912/t20191202_2358966.htm', u'全景网', 1]

简单的小实践。

Python爬取百度指数搜索结果，查看你想了解的热点信息吧相关推荐

Python爬取百度图片搜索结果
爬取百度图片搜索的图片,我们先需要分析其访问 URL,我们在搜索页面,比如搜索 "abc" ,打开 F12 调试,下拉结果页面页,查看网络请求,在其中我们可以找到这样一个请求 ht ...
利用Python爬取百度指数中需求图谱的关键词
文章目录需求背景 0.获取cookies 一.使用datetime计算查询的日期二.爬取需求图谱关键词三.扔进csv里总结已更新!!! 之前有小伙伴在评论里反应代码有点问题,今天看了下,报错 ...
python爬取百度指数
def baidu(keyword):"""百度指数"""headers = {'User-Agent': 'Mozilla/5.0 (Ma ...
python爬取百度百科搜索结果_用Python抓取百度搜索结果,python,爬取,的
前言前几天爬的今天整理了一下发现就两个需要注意的点一是记得用带cookie的方式去访问,也就是实例化requests.session() 二是转化一下爬取到的url,访问爬到的url得到返回的Lo ...
Python 爬取百度搜索风云榜新闻并自动推送到邮箱
本文将使用Python爬取百度新闻搜索指数排名前50的新闻,并通过服务器运行,每天定时发送到指定邮箱. 先上代码: # -*- coding:utf-8 -*- import requests,os, ...
python爬取百度搜索_使用Python + requests爬取百度搜索页面
想学一下怎样用python爬取百度搜索页面,因为是第一次接触爬虫,遇到一些问题,把解决过程与大家分享一下 1.使用requests爬取网页首先爬取百度主页www.baidu.com import r ...
Python爬取百度翻译及有道翻译
Python爬取百度翻译及网易有道翻译百度翻译一.简介明确翻译链接,百度翻译链接:https://fanyi.baidu.com/,但是该链接不能为我们提供翻译的内容,此时需要在chrome浏览 ...
【Python】python爬取百度云网盘资源-源码
今天测试用了一下python爬取百度云网盘资源. 标签: <无> 代码片段 [代码][Python]代码 import urllib import urllib.request impor ...
python爬取百度云网盘资源-源码
今天测试用了一下python爬取百度云网盘资源. 代码片段 import urllib import urllib.request import webbrowser import re def yu ...

Python爬取百度指数搜索结果，查看你想了解的热点信息吧

Python爬取百度指数搜索结果，查看你想了解的热点信息吧相关推荐

最新文章

热门文章