scrapy.pqueues

module documentation
(source)

Undocumented

Class DownloaderAwarePriorityQueue PriorityQueue which takes Downloader activity into account: domains (slots) with the least amount of active downloads are dequeued first.
Class DownloaderInterface No class docstring; 0/1 instance variable, 1/4 method documented
Class ScrapyPriorityQueue A priority queue implemented using multiple internal queues (typically, FIFO queues). It uses one internal queue for each priority value. The internal queue must implement the following methods:
Variable logger Undocumented
Function _path_safe Return a filesystem-safe version of a string ``text``
logger = (source)

Undocumented

def _path_safe(text): (source)

Return a filesystem-safe version of a string ``text`` >>> _path_safe('simple.org').startswith('simple.org') True >>> _path_safe('dash-underscore_.org').startswith('dash-underscore_.org') True >>> _path_safe('some@symbol?').startswith('some_symbol_') True