scrapy.request

https://doc.scrapy.org/en/latest/topics/request-response.html#request-objects

参数	说明	describe
url?(string)	需要请求的url	the URL of this request
callback?(callable)	回调函数	the function that will be called with the response of this request (once its downloaded) as its first parameter. For more information see?Passing additional data to callback functions?below. If a Request doesn’t specify a callback, the spider’s?parse()?method will be used. Note that if exceptions are raised during processing, errback is called instead.
method?(string)	请求的方法，默认是GET	the HTTP method of this request. Defaults to?’GET’.
meta?(dict)	这个属性的值可以在response中得到	the initial values for the?Request.meta?attribute. If given, the dict passed in this parameter will be shallow copied.
body?(str?or?unicode)	请求体	the request body. If a?unicode?is passed, then it’s encoded tostr?using the?encoding?passed (which defaults to?utf-8). If?body?is not given, an empty string is stored. Regardless of the type of this argument, the final value stored will be a?str?(never?unicode?or?None).
headers?(dict)	请求头	the headers of this request. The dict values can be strings (for single valued headers) or lists (for multi-valued headers). If?None?is passed as value, the HTTP header will not be sent at all.
cookies?(dict?or?list)	请求的cookie	the request cookies. These can be sent in two forms.
encoding?(string)	此请求的编码（默认为’utf-8’）	the encoding of this request (defaults to?’utf-8’). This encoding will be used to percent-encode the URL and to convert the body to?str?(if given as?unicode).
priority?(int)	这个请求的优先级，默认是0	the priority of this request (defaults to?0). The priority is used by the scheduler to define the order used to process requests. Requests with a higher priority value will execute earlier. Negative values are allowed in order to indicate relatively low-priority.
dont_filter?(boolean)	默认False，是否请求同一个页面	indicates that this request should not be filtered by the scheduler. This is used when you want to perform an identical request multiple times, to ignore the duplicates filter. Use it with care, or you will get into crawling loops. Default to?False.
errback?(callable)	如果出现错误则调用函数	a function that will be called if any exception was raised while processing the request. This includes pages that failed with 404 HTTP errors and such. It receives a?Twisted Failure?instance as first parameter. For more information, see?Using errbacks to catch exceptions in request processing?below.
flags?(list)	给请求贴的标签	Flags sent to the request, can be used for logging or similar purposes.

scrapy.request相关推荐

Scrapy框架的学习(5.scarpy实现翻页爬虫，以及scrapy.Request的相关参数介绍)
1. 创建爬虫项目 : scrapy startporject tencent 然后进入到项目中: cd tencent 创建爬虫:scrapy genspider tencent_spide ...
Scrapy - Request 和 Response（请求和响应）
Requests and Responses:http://doc.scrapy.org/en/latest/topics/request-response.html Requests and Res ...
scrapy-实现下一页请求， scrapy.Request
scrapy-实现下一页请求, scrapy.Request # -*- coding: utf-8 -*- import scrapyclass HrSpider(scrapy.Spider):na ...
scrapy Request
1.概念 1.构造请求,scrapy.Request(url,callback) callback:指定解析函数名称,表示该请求返回的响应使用哪一个函数进行解析 2.把请求交给引擎:yield scr ...
scrapy.Request使用meta传递数据，以及deepcopy的使用
scrapy.Request(url[,callback,method="GET",headers,body,cookies,meta,dont_filter=False]) 参数 ...
scrapy.Request callback不执行
上一篇博客介绍了Scrapy Post Request payload数据的问题,本篇记录一下scrapy.Request callback不执行的问题. 1.大家都说的解决方式,虽然没有解决我的问题 ...
request设置请求头_收藏 Scrapy框架各组件详细设置
今天说一下Scrapy框架各组件的详细设置. 关于Scrapy Scrapy是纯Python语言实现的爬虫框架,简单.易用.拓展性高是其主要特点.这里不过多介绍Scrapy的基本知识点,主要针对其高拓 ...
python 爬虫 scrapy 和 requsts 哪个快_Python爬虫：Scrapy研读之Request/Reponse
本帖最后由 shenzhenwan10 于 2016-6-17 07:52 编辑 1,引言在前一篇Scrapy:python3下的第一次运行测试中,我们以官网的tutorial为例,成功的运行了S ...
Scrapy中Request的回调函数不执行
一. 举例 def parse(self, response):...yield Request(url=parse.urljoin(response.url, title_herf), meta=m ...

scrapy.request

scrapy.request相关推荐

最新文章

热门文章