scrapy.spiders.crawl

module documentation

(source)

This modules implements the CrawlSpider which is the recommended spider to use for scraping typical web sites that requires crawling pages. See documentation in docs/topics/spiders.rst

Class	`CrawlSpider`	Undocumented
Class	`Rule`	Undocumented
Function	`_get_method`	Undocumented
Function	`_identity`	Undocumented
Function	`_identity_process_request`	Undocumented
Variable	`_default_link_extractor`	Undocumented

def _get_method(method, spider): (source) ¶

Undocumented

def _identity(x): (source) ¶

Undocumented

def _identity_process_request(request, response): (source) ¶

Undocumented

_default_link_extractor = (source) ¶

Undocumented