Python小工具-电影天堂爬取电影下载链接

import requests
import bs4# 获取单独的url
def movie_info(url):'''内容标签：<div id="Zoom">下载链接标签：--》a标签属性：thunderrestitle'''# url = 'http://www.dytt8.net/html/gndy/dyzz/20180118/56127.html'headers = {'User-Agent': "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 ""(KHTML, like Gecko) Chrome/22.0.1207.1 Safari/537.1", 'Referer': 'http://www.dytt8.net/html/gndy/dyzz/list_23_1.html', }html = requests.get(url, headers=headers)html.encoding = 'unicode'Soup = bs4.BeautifulSoup(html.content, 'html.parser')all_info = Soup.find('div', class_='co_content8').texttitle, movie_time, score, thunderrestitle = '', '', '', ''try:title = Soup.find('div', class_='bd3r').find('div', class_='co_area2').find('div', class_='title_all').textexcept:passtry:time_tag = all_info.index('发布时间')movie_time = all_info[time_tag:time_tag+15]except:passtry:score_tag = all_info.index('豆瓣评分')score = all_info[score_tag:score_tag + 12]except:passtry:thunderrestitle_tag = all_info.index('下载地址')try:thunderrestitle_tag_last = all_info.index('磁力链下载')except:thunderrestitle_tag_last = all_info.index('下载地址2')# thunderrestitle_tag_last = all_info.index('磁力链下载') if all_info.index('磁力链下载') else all_info.index('下载地址2')# thunderrestitle_tag_last = all_info.index('下载地址2')thunderrestitle = all_info[thunderrestitle_tag + 9:thunderrestitle_tag_last - 5]except:pass# print(title, movie_time, score, thunderrestitle)# print(all_info)# return {'title':title,'movie_time':movie_time,'score':score,'thunderrestitle':thunderrestitle}with open('电影天堂爬取下载链接.txt','a',encoding='utf8') as f:try:# f.write(str(title+movie_time+score+thunderrestitle))f.write(str(title + '/' + movie_time + '/' + score + thunderrestitle + '\n'))except:pass# print(all_info)def get_url(page=2500):all_url = 'http://www.dytt8.net/html/gndy/dyzz/index.html'headers = {'User-Agent': "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 ""(KHTML, like Gecko) Chrome/22.0.1207.1 Safari/537.1", 'Referer': 'http://www.dytt8.net/', }html = requests.get(all_url, headers=headers)html.encoding = 'unicode'Soup = bs4.BeautifulSoup(html.content, 'html.parser')all_info = Soup.find('select', attrs={"name":'sldd'}).findAll('option')urls = []n = 0for i in all_info:if n < page:# print(i.attrs['value'])url = 'http://www.dytt8.net/html/gndy/dyzz/' + i.attrs['value']# print(url)urls.append(url)n += 1else:breakreturn urlsdef get_movie_url(movie_url):# movie_url = 'http://www.dytt8.net/html/gndy/dyzz/list_23_1.html'headers = {'User-Agent': "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 ""(KHTML, like Gecko) Chrome/22.0.1207.1 Safari/537.1", 'Referer': 'http://www.dytt8.net/html/gndy/dyzz/list_23_1.html', }html = requests.get(movie_url, headers=headers)html.encoding = 'unicode'Soup = bs4.BeautifulSoup(html.content, 'html.parser')all_info = Soup.find('div', class_='co_content8').find('ul').findAll('table')movie_urls = []for i in all_info:url = 'http://www.dytt8.net' + i.find('a').attrs['href']print(i.find('a').attrs['href'])movie_urls.append(url)return movie_urls# movie_info('http://www.dytt8.net/html/gndy/dyzz/20180121/56158.html')
if __name__ == '__main__':urls = get_url(1)for movie_url in urls:movie_urls = get_movie_url(movie_url)for url in movie_urls:movie_info(url)# 下面这个页面爬取的是乱码，不知道为什么。
# url = 'http://www.dytt8.net/html/gndy/dyzz/20180130/56216.html'
#
# html = requests.get(url)
# html.encoding = 'unicode'
#
# Soup = bs4.BeautifulSoup(html.content, 'html.parser')
# all_info = Soup.find('div', class_='co_content8').text
# print(all_info)

Python小工具-电影天堂爬取电影下载链接相关推荐

python程序爬电影_Python爬取电影天堂最新发布影片消息
从今天开始我会把我学习python爬虫的一些心得体会和代码发布在我现在的博客,好记性不如烂笔头,以便以后的我进行复习. 虽然我现在的爬虫还很幼小,希望有一天她能长得非常非常的强大. --------- ...
Python爬虫-从pixabay网站爬取并下载图片
Python爬虫--Python爬虫爬取Demo图片数据 GitHub:https://github.com/kevinten10/Lifecat-Python pixabay是优秀的图片搜索网站:h ...
Python爬虫（二）——爬取电影天堂，保存下载地址
首先我们开始要分析一下,下载种子我们需要哪几步: 获取所有电影页的访问地址获取电影页源码提取出下载地址将下载地址保存首先第一步,我们来分析一下电影天堂网站的结构,发现他跟我们的古诗文网还是非常 ...
python正则表达式【电影天堂爬取】
正则表达式 import re result = re.match("\w{4,20}@163\.com$","hello@163.com") if resul ...
python教程app下载地址_Python爬取APP下载链接的实现方法
首先是准备工作 Python 2.7.11:下载python 其中python2和python3目前同步发行,我这里使用的是python2作为环境.Pycharm是一款比较高效的Python IDE, ...
【爬虫】从B站或央视网爬取视频下载链接
import requests from bs4 import BeautifulSoup import re import senddetail import sys import pandas a ...
爬虫(9)实战爬取电影天堂的1000+最新电影
文章来因: 客官们,久等了,在家上网课,上的无聊,想看个电影放松一下,但是却不知道看啥电影,想起最近学习的爬虫,于是找电影天堂爬个电影信息,不就知道看那个电影了,上菜菜单文章来因: 实战内容:直接 ...
python3爬虫：爬取电影天堂电影信息
python3爬虫:爬取电影天堂电影信息 #爬取电影天堂电影信息 #爬取电影天堂电影信息 #爬取电影天堂电影信息 from lxml import etree import requestsBASE_ ...
爬虫入门案例——爬取电影信息
综述:先介绍开发环境,在介绍从豆瓣电影上面爬取电影的电影名称,电影信息,电影剧照等信息,最终效果是以电影名称为文件夹名,文件夹包含一个txt文件和许多张剧照.如下图所示1是电影名,2是电影信息,3是各 ...

Python小工具-电影天堂爬取电影下载链接

Python小工具-电影天堂爬取电影下载链接相关推荐

最新文章

热门文章