xpath 爬取某网站图片

效果图请自行脑补，此处不做展示（狗头保命）

from lxml import etree
import requests
import osif __name__ == '__main__':url = "http://pic.netbian.com/4kmeinv/"headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/""85.0.4183.83 Safari/537.36"}response = requests.get(url=url, headers=headers)response.encoding = response.apparent_encodingpage_text = response.texttree = etree.HTML(page_text)li_list = tree.xpath("//div[@class='slist']/ul/li")if not os.path.exists("./picLibs"):os.mkdir("./picLibs")for li in li_list:img_src = "http://pic.netbian.com" + li.xpath("./a/img/@src")[0]img_name = li.xpath("./a/img/@alt")[0] + ".jpg"img_data = requests.get(url=img_src, headers=headers).contentimg_path = "picLibs/" + img_namewith open(img_path, "wb") as fp:fp.write(img_data)print(img_name + "下载成功")for x in range(2, 172):url_too = f"http://pic.netbian.com/4kmeinv/index_{x}.html"response_too = requests.get(url=url_too, headers=headers)response_too.encoding = response_too.apparent_encodingpage_text = response_too.texttree_too = etree.HTML(page_text)li_list = tree_too.xpath("//div[@class='slist']/ul/li")for li_too in li_list:new_img_src = f"http://pic.netbian.com/4kmeinv/index_{x}.html" + li_too.xpath("./a/img/@src")[0]img_name = li_too.xpath("./a/img/@alt")[0] + ".jpg"img_data = requests.get(url=new_img_src, headers=headers).contentimg_path = "picLibs/" + img_namewith open(img_path, "wb") as fp:fp.write(img_data)print(img_name + "下载成功")

优化后（可能并没有优化）

from lxml import etree
import requests
import osdef get(url, headers):response = requests.get(url=url, headers=headers)response.encoding = response.apparent_encodingreturn response.textdef parse(url, headers):response = get(url=url, headers=headers)tree = etree.HTML(response)li_list = tree.xpath("//div[@class='slist']/ul/li")return li_listdef save(url):for li in li_list:img_src = url + li.xpath("./a/img/@src")[0]img_name = li.xpath("./a/img/@alt")[0] + ".jpg"img_data = requests.get(url=img_src, headers=headers).contentimg_path = "小姐姐图片/" + img_namewith open(img_path, "wb") as fp:fp.write(img_data)print(img_name + "下载成功")if __name__ == '__main__':url = "http://pic.netbian.com/4kmeinv/"headers = {"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/""85.0.4183.83 Safari/537.36"}li_list = parse(url=url, headers=headers)if not os.path.exists("./小姐姐图片"):os.mkdir("./小姐姐图片")save(url)for x in range(2, 172):url = f"http://pic.netbian.com/4kmeinv/index_{x}.html"li_list = parse(url=url, headers=headers)save(url)

xpath 爬取某网站图片相关推荐

python爬虫真假网址,python爬取福利网站图片完整代码,懂得人都懂
网址需要自己替换懂的人都懂512*2,主要学习简单的爬虫,别乱用,否则后果自负! [Python] 纯文本查看复制代码 import requests,bs4,re,os,threadingclas ...
爬虫基础练习: 基于 java + Jsoup + xpath 爬取51job网站
最基本的网页爬虫练习爬取51jb网站,并将数据写入Excel中需要导入jsoup包和POI相关包 JSoup简介 jsoup是一款Java的HTML解析器,主要用来对HTML解析, 可通过DOM, ...
python最新官网图片_python爬取福利网站图片完整代码
存起来自己学习... import requests,bs4,re,os,threading class MeiNvTu: def __init__(self): self.url_main='ht ...
代理的基本原理及用Xpath爬取代理网站IP列表测试并存入数据库
前言在网络爬虫中,有些网站会设置反爬虫措施,服务器会检测某个IP在单位时间内的请求次数,如果超过了这个阈值,就会直接拒绝服务,返回一些错误信息,例如 403 Forbidden,"您的IP ...
python数据解析——xpath爬取文字和图片
xpath解析:最常用且最便捷高效的一种解析方式,通用性. xpath解析原理: 1.实例化一个etree的对象,且需要将被解析的页面源码数据加载到该对象中. 2.调用etree对象中的xpath方法 ...
python爬虫爬取某网站图片
学习分享 | 今天刚学完爬虫,就随便写了一个爬虫代码爬取某网站的图片网站就是这个图片网站,我选的是1080p格式,4k的要会员,我反正是还不会导入的包如下 import requests from ...
python批量访问网页保存结果_Python爬虫(批量爬取某网站图片)
1.需要用到的库有: Requests re os time 如果没有安装的请自己安装一下,pycharm中打开终端输入命令就可以安装 2.IDE : pycharm 3.python 版本: 3.8 ...
python爬取pexels网站图片
#coding:utf-8from lxml import etree import requests import urllib.request import progressbar header= ...
python 动态相册,python3爬取动态网站图片
思路: 1.图片放在XXX标签中 2.利用fiddler抓包获取存放图片信息的js文件url 3.利用requests库获取html内容,然后获取其中图片id 4.利用fiddler抓取下载图片地址, ...

xpath 爬取某网站图片

xpath 爬取某网站图片相关推荐

最新文章

热门文章