scrapy.pipelines.media.MediaPipeline

class documentation

class MediaPipeline: (source)

Known subclasses: scrapy.pipelines.files.FilesPipeline

View In Hierarchy

Undocumented

Class SpiderInfo Undocumented
Class Method from_crawler Undocumented
Method __init__ Undocumented
Method file_path Returns the path where downloaded media should be stored
Method get_media_requests Returns the media requests to download
Method item_completed Called per item when all media requests has been processed
Method media_downloaded Handler for success downloads
Method media_failed Handler for failed downloads
Method media_to_download Check request before starting download
Method open_spider Undocumented
Method process_item Undocumented
Constant LOG_FAILED_RESULTS Undocumented
Instance Variable allow_redirects Undocumented
Instance Variable download_func Undocumented
Instance Variable handle_httpstatus_list Undocumented
Instance Variable spiderinfo Undocumented
Method _cache_result_and_execute_waiters Undocumented
Method _check_media_to_download Undocumented
Method _check_signature Undocumented
Method _compatible Wrapper for overridable methods to allow backwards compatibility
Method _handle_statuses Undocumented
Method _key_for_pipe >>> MediaPipeline()._key_for_pipe("IMAGES") 'IMAGES' >>> class MyPipe(MediaPipeline): ... pass >>> MyPipe()._key_for_pipe("IMAGES", base_class_name="MediaPipeline") 'MYPIPE_IMAGES'
Method _make_compatible Make overridable methods of MediaPipeline and subclasses backwards compatible
Method _modify_media_request Undocumented
Method _process_request Undocumented
Instance Variable _expects_item Undocumented
@classmethod
def from_crawler(cls, crawler): (source)

Undocumented

def __init__(self, download_func=None, settings=None): (source)
overridden in scrapy.pipelines.files.FilesPipeline

Undocumented

def file_path(self, request, response=None, info=None, *, item=None): (source)
overridden in scrapy.pipelines.files.FilesPipeline

Returns the path where downloaded media should be stored

def get_media_requests(self, item, info): (source)
overridden in scrapy.pipelines.files.FilesPipeline

Returns the media requests to download

def item_completed(self, results, item, info): (source)
overridden in scrapy.pipelines.files.FilesPipeline

Called per item when all media requests has been processed

def media_downloaded(self, response, request, info, *, item=None): (source)
overridden in scrapy.pipelines.files.FilesPipeline

Handler for success downloads

def media_failed(self, failure, request, info): (source)
overridden in scrapy.pipelines.files.FilesPipeline

Handler for failed downloads

def media_to_download(self, request, info, *, item=None): (source)
overridden in scrapy.pipelines.files.FilesPipeline

Check request before starting download

def open_spider(self, spider): (source)

Undocumented

def process_item(self, item, spider): (source)

Undocumented

LOG_FAILED_RESULTS: bool = (source)

Undocumented

Value
True
allow_redirects = (source)

Undocumented

download_func = (source)

Undocumented

handle_httpstatus_list = (source)

Undocumented

spiderinfo = (source)

Undocumented

def _cache_result_and_execute_waiters(self, result, fp, info): (source)

Undocumented

def _check_media_to_download(self, result, request, info, item): (source)

Undocumented

def _check_signature(self, func): (source)

Undocumented

def _compatible(self, func): (source)

Wrapper for overridable methods to allow backwards compatibility

def _handle_statuses(self, allow_redirects): (source)

Undocumented

def _key_for_pipe(self, key, base_class_name=None, settings=None): (source)

>>> MediaPipeline()._key_for_pipe("IMAGES") 'IMAGES' >>> class MyPipe(MediaPipeline): ... pass >>> MyPipe()._key_for_pipe("IMAGES", base_class_name="MediaPipeline") 'MYPIPE_IMAGES'

def _make_compatible(self): (source)

Make overridable methods of MediaPipeline and subclasses backwards compatible

def _modify_media_request(self, request): (source)

Undocumented

def _process_request(self, request, info, item): (source)

Undocumented

_expects_item: dict = (source)

Undocumented