python实现艺龙爬虫

有反爬措施暂时只能爬取一页

import requests
import re
import pandas as pd
import timedef get_data(page):""""""url = "http://hotel.elong.com/ajax/list/asyncsearch"data = {"code": "7809114","listRequest.areaID": "","listRequest.bookingChannel": 5,"listRequest.cardNo": "192928","listRequest.checkInDate": "2018-03-19 00:00:00","listRequest.checkOutDate": "2018-03-20 00:00:00","listRequest.cityID": "0401","listRequest.cityName": "重庆市","listRequest.customLevel": "11","listRequest.distance": "20","listRequest.endLat": 0,"listRequest.endLng": 0,"listRequest.facilityIds": "","listRequest.highPrice": 0,"listRequest.hotelBrandIDs": "","listRequest.isAdvanceSave": "false","listRequest.isAfterCouponPrice": "true","listRequest.isCoupon": "false","listRequest.isDebug": "false","listRequest.isLimitTime": "false","listRequest.isLogin": "false","listRequest.isMobileOnly": "true","listRequest.isNeed5Discount": "true","listRequest.isNeedNotContractedHotel": "false","listRequest.isNeedSimilarPrice": "false","listRequest.isReturnNoRoomHotel": "true","listRequest.isStaySave": "false","listRequest.isTrace": "false","listRequest.isUnionSite": "false","listRequest.keywords": "","listRequest.keywordsType": 0,"listRequest.language": "cn","listRequest.listType": 0,"listRequest.lowPrice": 0,"listRequest.orderFromID": "50793","listRequest.pageIndex": page,"listRequest.pageSize": 20,"listRequest.payMethod": 0,"listRequest.personOfRoom": 0,"listRequest.poiId": 0,"listRequest.promotionChannelCode": "0000","listRequest.proxyID": "ZD","listRequest.rankType": 0,"listRequest.returnFilterItem": "true","listRequest.sellChannel": 1,"listRequest.seoHotelStar": 0,"listRequest.sortDirection": 1,"listRequest.sortMethod": 1,"listRequest.starLevels": "","listRequest.startLat": 0,"listRequest.startLng": 0,"listRequest.taRecommend": "false","listRequest.themeIds": "","listRequest.ctripToken": "815b07a9-3f97-4ae0-965c-e8d9d3b9a57f","listRequest.elongToken": "jeww06u3-7967-4a47-9e59-91d212f31e82",}header = {"Accept": "application/json, text/javascript, */*; q=0.01","Accept-Encoding": "gzip, deflate","Accept-Language": "zh-CN,zh;q=0.9","Cache-Control": "no-cache","Connection": "keep-alive","Content-Length": "1623","Content-Type": "application/x-www-form-urlencoded; charset=UTF-8","Host": "hotel.elong.com","Origin": "http://hotel.elong.com","Pragma": "no-cache","Referer": "http://hotel.elong.com/chongqing/","User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.186 Safari/537.36","X-Requested-With": "XMLHttpRequest"}response = requests.post(url, data=data, headers=header)html = response.json()hotel_name = re.findall('target="_blank" title="(.*?)"><span class="icon_nmb">', html['value']['hotelListHtml'])hotel_price = re.findall('<span class="h_pri_num ">(.*?)</span>', html['value']['hotelListHtml'])hotel_address = re.findall('data-hoteladdress="(.*?)" >', html['value']['hotelListHtml'])# 返回酒店名称，酒店价格，酒店地址return hotel_name, hotel_price, hotel_address
if __name__ == '__main__':hotel_name = []hotel_price = []hotel_address = []for i in range(10):hotel_name_, hotel_price_, hotel_address_ = get_data(i)hotel_name.extend(hotel_name_)hotel_price.extend(hotel_price_)hotel_address.extend(hotel_address_)time.sleep(1)print("已完成第"+str(i)+"页爬取")dataframe = pd.DataFrame({'酒店名称': hotel_name, '酒店价格': hotel_price, '酒店地址': hotel_address})dataframe.to_csv("hotel.csv", index=False, sep=',', encoding="utf_8_sig")

python实现艺龙爬虫相关推荐

Python爬虫实战爬取同城艺龙酒店信息
1.爬虫说明同城艺龙的反爬做的是非常好的,本博主在与同城艺龙进行了一整天的殊死搏斗才将其完全的爬下来,本博主是以无锡为例,将无锡的所有酒店的相关信息都爬了下来,共3399条酒店数据,当 ...
python爬虫爬取艺龙国际酒店信息
项目需要用到数据,在网上找了好久的数据都没有结果,就自己写了爬虫.没咋写过程序,为了简单都没有用函数.中间遇见了不少问题,首先爬虫只能爬取前十条信息.为了解决这个问题,上网搜了一些信息.网上说动态网页 ...
Python旅游景点推荐系统爬虫酒店旅游新闻
技术栈:python语言 django框架 vue框架 scrapy爬虫框架系统功能: 景点推荐.景点详情.旅游路线.旅游时节. 周边景点.周边酒店.评论.景点.站内旅游新闻. 旅游酒店.酒店详情. ...
基于python的影评数据分析_基于Python聚焦型网络爬虫的影评获取技术
龙源期刊网 http://www.qikan.com.cn 基于 Python 聚焦型网络爬虫的影评获取技术作者:郭向向郑嘉慧苗学芹来源:<时代金融> 2019 年第 11 期 ...
从Zabbix到Prometheus，同程艺龙数据库监控系统的实践
来自:DBAplus社群作者介绍闫晓宇,同程艺龙数据库技术专家,具有多年互联网行业DB运维经验,在游戏.O2O及电商行业从事过DBA运维工作.2016年加入同程艺龙,目前在团队负责数据库架构设计及 ...
python爬虫小说代码示例-中文编程，用python编写小说网站爬虫
原标题:中文编程,用python编写小说网站爬虫作者:乘风龙王原文:https://zhuanlan.zhihu.com/p/51309019 为保持源码格式, 转载时使用了截图. 原文中的源码块 ...
python初学者web还是爬虫-还在纠结学爬虫还是数据分析，不如看看这篇文章
原标题:还在纠结学爬虫还是数据分析,不如看看这篇文章身为职场人,收集上万条表格数据做商业分析,裁剪上千张图片,发送数百封邮件...这些都是经常会遇到的场景.我一直期待能有个工具解放我,直到我遇到了P ...
python爬虫简单实例-Python 利用Python编写简单网络爬虫实例3
利用Python编写简单网络爬虫实例3 by:授客 QQ:1033553122 实验环境 python版本:3.3.5(2.7下报错实验目的获取目标网站"http://bbs.51tes ...
python中国大学排名爬虫写明详细步骤-python中国大学排名爬虫
python 中国大学排名爬虫首先,给一个最好大学网URL:http://www.zuihaodaxue.cn/zuihaodaxuepaiming2016.html, 点击这里进入 . 功能描述 ...

python实现艺龙爬虫

python实现艺龙爬虫相关推荐

最新文章

热门文章