scrapy.commands.parse.Command

class documentation

class Command(BaseRunSpiderCommand): (source)

View In Hierarchy

Undocumented

Method add_items Undocumented
Method add_options Populate option parse with options available for this command
Method add_requests Undocumented
Method get_callback_from_rules Undocumented
Method prepare_request Undocumented
Method print_items Undocumented
Method print_requests Undocumented
Method print_results Undocumented
Method process_options Undocumented
Method process_request_cb_kwargs Undocumented
Method process_request_meta Undocumented
Method run Entry point for running commands
Method run_callback Undocumented
Method scraped_data Undocumented
Method set_spidercls Undocumented
Method short_desc A short description of the command
Method start_parsing Undocumented
Method syntax Command syntax (preferably one-line). Do not include command name.
Class Variable first_response Undocumented
Class Variable items Undocumented
Class Variable requests Undocumented
Class Variable requires_project Undocumented
Class Variable spider Undocumented
Instance Variable pcrawler Undocumented
Instance Variable spidercls Undocumented
Property max_level Undocumented
Method _get_items_and_requests Undocumented

Inherited from ScrapyCommand (via BaseRunSpiderCommand):

Method __init__ Undocumented
Method help An extensive help for the command. It will be shown when using the "help" command. It can contain newlines since no post-formatting will be applied to its contents.
Method long_desc A long description of the command. Return short description when not available. It cannot contain newlines since contents will be formatted by optparser which removes newlines and wraps text.
Method set_crawler Undocumented
Class Variable crawler_process Undocumented
Class Variable default_settings Undocumented
Class Variable exitcode Undocumented
Instance Variable settings Undocumented
Instance Variable _crawler Undocumented
def add_items(self, lvl, new_items): (source)

Undocumented

def add_options(self, parser): (source)
overrides scrapy.commands.BaseRunSpiderCommand.add_options

Populate option parse with options available for this command

def add_requests(self, lvl, new_reqs): (source)

Undocumented

def get_callback_from_rules(self, spider, response): (source)

Undocumented

def prepare_request(self, spider, request, opts): (source)

Undocumented

def print_items(self, lvl=None, colour=True): (source)

Undocumented

def print_requests(self, lvl=None, colour=True): (source)

Undocumented

def print_results(self, opts): (source)

Undocumented

def process_options(self, args, opts): (source)
overrides scrapy.commands.BaseRunSpiderCommand.process_options

Undocumented

def process_request_cb_kwargs(self, opts): (source)

Undocumented

def process_request_meta(self, opts): (source)

Undocumented

def run(self, args, opts): (source)
overrides scrapy.commands.ScrapyCommand.run

Entry point for running commands

def run_callback(self, response, callback, cb_kwargs=None): (source)

Undocumented

def scraped_data(self, args): (source)

Undocumented

def set_spidercls(self, url, opts): (source)

Undocumented

def short_desc(self): (source)
overrides scrapy.commands.ScrapyCommand.short_desc

A short description of the command

def start_parsing(self, url, opts): (source)

Undocumented

def syntax(self): (source)
overrides scrapy.commands.ScrapyCommand.syntax

Command syntax (preferably one-line). Do not include command name.

first_response = (source)

Undocumented

items: Dict[int, list] = (source)

Undocumented

requests: Dict[int, list] = (source)

Undocumented

requires_project: bool = (source)
overrides scrapy.commands.ScrapyCommand.requires_project

Undocumented

spider = (source)

Undocumented

pcrawler = (source)

Undocumented

spidercls = (source)

Undocumented

@property
max_level = (source)

Undocumented

def _get_items_and_requests(self, spider_output, opts, depth, spider, callback): (source)

Undocumented