scrapy.extensions.feedexport.FeedExporter

class documentation

class FeedExporter: (source)

View In Hierarchy

Undocumented

Class Method from_crawler Undocumented
Method __init__ Undocumented
Method close_spider Undocumented
Method item_scraped Undocumented
Method open_spider Undocumented
Instance Variable crawler Undocumented
Instance Variable exporters Undocumented
Instance Variable feeds Undocumented
Instance Variable filters Undocumented
Instance Variable settings Undocumented
Instance Variable slots Undocumented
Instance Variable storages Undocumented
Method _close_slot Undocumented
Method _exporter_supported Undocumented
Method _get_exporter Undocumented
Method _get_instance Undocumented
Method _get_storage Fork of create_instance specific to feed storage classes
Method _get_uri_params Undocumented
Method _handle_store_error Undocumented
Method _handle_store_success Undocumented
Method _load_components Undocumented
Method _load_filter Undocumented
Method _settings_are_valid If FEED_EXPORT_BATCH_ITEM_COUNT setting or FEEDS.batch_item_count is specified uri has to contain %(batch_time)s or %(batch_id)d to distinguish different files of partial output
Method _start_new_batch Redirect the output data stream to a new file. Execute multiple times if FEED_EXPORT_BATCH_ITEM_COUNT setting or FEEDS.batch_item_count is specified :param batch_id: sequence number of current batch :param uri: uri of the new batch to start :param feed_options: dict with parameters of feed :param spider: user spider :param uri_template: template of uri which contains %(batch_time)s or %(batch_id)d to create new uri...
Method _storage_supported Undocumented
@classmethod
def from_crawler(cls, crawler): (source)

Undocumented

def __init__(self, crawler): (source)

Undocumented

def close_spider(self, spider): (source)

Undocumented

def item_scraped(self, item, spider): (source)

Undocumented

def open_spider(self, spider): (source)

Undocumented

crawler = (source)

Undocumented

exporters = (source)

Undocumented

feeds: dict = (source)

Undocumented

filters: dict = (source)

Undocumented

settings = (source)

Undocumented

slots = (source)

Undocumented

storages = (source)

Undocumented

def _close_slot(self, slot, spider): (source)

Undocumented

def _exporter_supported(self, format): (source)

Undocumented

def _get_exporter(self, file, format, *args, **kwargs): (source)

Undocumented

def _get_instance(self, objcls, *args, **kwargs): (source)

Undocumented

def _get_storage(self, uri, feed_options): (source)

Fork of create_instance specific to feed storage classes It supports not passing the *feed_options* parameters to classes that do not support it, and issuing a deprecation warning instead.

def _get_uri_params(self, spider: Spider, uri_params_function: Optional[Union[str, Callable[[dict, Spider], dict]]], slot: Optional[_FeedSlot] = None) -> dict: (source)

Undocumented

def _handle_store_error(self, f, logmsg, spider, slot_type): (source)

Undocumented

def _handle_store_success(self, f, logmsg, spider, slot_type): (source)

Undocumented

def _load_components(self, setting_prefix): (source)

Undocumented

def _load_filter(self, feed_options): (source)

Undocumented

def _settings_are_valid(self): (source)

If FEED_EXPORT_BATCH_ITEM_COUNT setting or FEEDS.batch_item_count is specified uri has to contain %(batch_time)s or %(batch_id)d to distinguish different files of partial output

def _start_new_batch(self, batch_id, uri, feed_options, spider, uri_template): (source)

Redirect the output data stream to a new file. Execute multiple times if FEED_EXPORT_BATCH_ITEM_COUNT setting or FEEDS.batch_item_count is specified :param batch_id: sequence number of current batch :param uri: uri of the new batch to start :param feed_options: dict with parameters of feed :param spider: user spider :param uri_template: template of uri which contains %(batch_time)s or %(batch_id)d to create new uri

def _storage_supported(self, uri, feed_options): (source)

Undocumented