scrapy.dupefilters.RFPDupeFilter

class documentation

class RFPDupeFilter(BaseDupeFilter): (source)

View In Hierarchy

Request Fingerprint duplicates filter

Class Method from_crawler Undocumented
Class Method from_settings Undocumented
Method __init__ Undocumented
Method close Undocumented
Method log Log that a request has been filtered
Method request_fingerprint Undocumented
Method request_seen Undocumented
Instance Variable debug Undocumented
Instance Variable file Undocumented
Instance Variable fingerprinter Undocumented
Instance Variable fingerprints Undocumented
Instance Variable logdupes Undocumented
Instance Variable logger Undocumented

Inherited from BaseDupeFilter:

Method open Undocumented
@classmethod
def from_crawler(cls, crawler): (source)

Undocumented

@classmethod
def from_settings(cls: Type[RFPDupeFilterTV], settings: BaseSettings, *, fingerprinter=None) -> RFPDupeFilterTV: (source)
overrides scrapy.dupefilters.BaseDupeFilter.from_settings

Undocumented

def __init__(self, path: Optional[str] = None, debug: bool = False, *, fingerprinter=None): (source)

Undocumented

def close(self, reason: str): (source)
overrides scrapy.dupefilters.BaseDupeFilter.close

Undocumented

def log(self, request: Request, spider: Spider): (source)
overrides scrapy.dupefilters.BaseDupeFilter.log

Log that a request has been filtered

def request_fingerprint(self, request: Request) -> str: (source)

Undocumented

def request_seen(self, request: Request) -> bool: (source)
overrides scrapy.dupefilters.BaseDupeFilter.request_seen

Undocumented

debug = (source)

Undocumented

file = (source)

Undocumented

fingerprinter = (source)

Undocumented

fingerprints: Set[str] = (source)

Undocumented

logdupes: bool = (source)

Undocumented

logger = (source)

Undocumented