第一次使用scrapy爬豆瓣top250 报错 AttributeError Requset has no attribute dont

首次学习scrapy框架来爬取豆瓣Top250,在写完代码执行时,报了AttributeError: ‘Request’ object has no attribute ‘meta’ 还有AttributeError: ‘Request’ object has no attribute ‘dont_filter’ .

执行的代码

# -*- coding: utf-8 -*-
from requests import Request
from scrapy.spiders import Spider
from scrapyspider.items import DoubanMovieItemclass DoubanMovieTop250Spider(Spider):name = "douban250"start_urls = ["https://movie.douban.com/top250"]headers ={'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36'}def start_requests(self):# url = "https://movie.douban.com/top250"url = 'https://movie.douban.com/top250'yield Request(url, headers = self.headers)def parse(self, response):item = DoubanMovieItem()movies = response.xpath('//ol[@class="grid_view"]/li')for movie in movies:item['ranking'] = movie.xpath('.//div[@class="pic"]/em/text()').extract()[0]item['movie_name'] = movie.xpath('.//div[@class="hd"]/a/span[1]/text()').extract()[0]item['score'] = movie.xpath('.//div[@class="star"]/span[@class="rating_num"]/text()').extract()[0]item['score_number'] = movie.xpath('.//div[@class="star"]/span/text()').re(u'(\d+)人评价')[0]yield item

跑完代码cmd出现下面的错误情况:

后来经过一番查找才发现原来是因为自己导包错误的问题…

from requests import Request
应该改为
from scrapy import Request

修改完的代码:

from scrapy import Request
from scrapy.spiders import Spider
from scrapyspider.items import DoubanMovieItemclass DoubanMovieTop250Spider(Spider):name = "douban250"start_urls = ["https://movie.douban.com/top250"]headers ={'User-Agent': 'Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/53.0.2785.143 Safari/537.36'}def start_requests(self):# url = "https://movie.douban.com/top250"url = 'https://movie.douban.com/top250'yield Request(url, headers = self.headers)def parse(self, response):item = DoubanMovieItem()movies = response.xpath('//ol[@class="grid_view"]/li')for movie in movies:item['ranking'] = movie.xpath('.//div[@class="pic"]/em/text()').extract()[0]item['movie_name'] = movie.xpath('.//div[@class="hd"]/a/span[1]/text()').extract()[0]item['score'] = movie.xpath('.//div[@class="star"]/span[@class="rating_num"]/text()').extract()[0]item['score_number'] = movie.xpath('.//div[@class="star"]/span/text()').re(u'(\d+)人评价')[0]yield item

执行完的成功,cmd是这样的:

第一次用爬虫框架爬取数据马马虎虎… 记录一下

第一次使用scrapy爬豆瓣top250 报错 AttributeError Requset has no attribute dont_filter 和 meta相关推荐

Cesi运行报错AttributeError: can‘t set attribute
环境说明: Centos 7 + Python3.6 + Cesi 2.7.1 按照官网说明运行,服务启动正常,在登录界面输入密码后提示密码错误. 打开浏览器开发者工具,查看对应的登录POST请求,提 ...
近期爬虫学习体会以及爬豆瓣Top250源码实战
近期爬虫学习体会以及爬豆瓣Top250源码实战我是在B站https://www.bilibili.com/video/BV12E411A7ZQ?p=25里学习的,至今已经可以手写爬豆瓣Top250代 ...
Scrapy爬虫报错AttributeError: ‘NoneType‘ object has no attribute ‘write‘
前言一.报错 AttributeError: 'NoneType' object has no attribute 'write' 二.报错原因 1.piplines文件中的方法不能自定义的呢开始 ...
python 报错 AttributeError: module ‘time‘ has no attribute ‘clock 解决方法
源码如下: #引入所需要的时间库 import datetime import time#程序计时器,启动计时器 start = time.clock()#中间是放置需要测试运行时间的程序代码#计算启 ...
第一：Python+Allure运行报错AttributeError: module ‘allure‘ has no attribute ‘severity_level‘
1.很多查询结果都是先卸载pytest-allure-adaptor,然后再安装allure-pytest pip3 uninstall pytest-allure-adaptor 2.pip3 in ...
python中引入包的时候报错AttributeError: module ‘sys‘ has no attribute ‘setdefaultencoding‘解决方法？
python中引入包的时候报错AttributeError: module 'sys' has no attribute 'setdefaultencoding'解决方法? 参考文章: (1)pyth ...
关于在《python编程从入门到实践》书中练习“外星人大战”报错“AttributeError: ‘AlienInvasion‘ object has no attribute ‘blit‘”
关于在<python编程从入门到实践>书中练习"外星人大战"报错"AttributeError: 'AlienInvasion' object has no ...
解决 Pycharm 使用高版本 python（3.9）报错 ‘HTMLParser‘ object has no attribute ‘unescape‘ 之修改pycharm
由于Python出现3.10版本,此方法修改后仍会导致python版本识别错误,所以不再推荐此方法. 感谢本篇博文参考了以下内容: 程序员的一天--pycharm 切换 python3.9 报错 ' ...
python操作html的object_关于pycharm 切换 python3.9 报错 ‘HTMLParser‘ object has no attribute ‘unescape‘ 的问题...
有的bug,莫名其妙就好了- python3.9 报错 "AttributeError: 'HTMLParser' object has no attribute 'unescape'&qu ...

第一次使用scrapy爬豆瓣top250 报错 AttributeError Requset has no attribute dont_filter 和 meta

第一次使用scrapy爬豆瓣top250 报错 AttributeError Requset has no attribute dont_filter 和 meta相关推荐

最新文章

热门文章