scrapy.utils.test

module documentation
(source)

This module contains some assorted functions used in tests

Function assert_gcs_environ Undocumented
Function assert_samelines Asserts text1 and text2 have the same lines, ignoring differences in line endings between platforms
Function get_crawler Return an unconfigured Crawler object. If settings_dict is given, it will be used to populate the crawler settings with a project level priority.
Function get_from_asyncio_queue Undocumented
Function get_ftp_content_and_delete Undocumented
Function get_gcs_content_and_delete Undocumented
Function get_pythonpath Return a PYTHONPATH suitable to use in processes so that they find this installation of Scrapy
Function get_testenv Return a OS environment dict suitable to fork processes that need to import this installation of Scrapy, instead of a system installed one.
Function get_web_client_agent_req Undocumented
Function mock_google_cloud_storage Creates autospec mocks for google-cloud-storage Client, Bucket and Blob classes and set their proper return values.
Function skip_if_no_boto Undocumented
def assert_gcs_environ(): (source)

Undocumented

def assert_samelines(testcase, text1, text2, msg=None): (source)

Asserts text1 and text2 have the same lines, ignoring differences in line endings between platforms

def get_crawler(spidercls=None, settings_dict=None, prevent_warnings=True): (source)

Return an unconfigured Crawler object. If settings_dict is given, it will be used to populate the crawler settings with a project level priority.

def get_from_asyncio_queue(value): (source)

Undocumented

def get_ftp_content_and_delete(path, host, port, username, password, use_active_mode=False): (source)

Undocumented

def get_gcs_content_and_delete(bucket, path): (source)

Undocumented

def get_pythonpath() -> str: (source)

Return a PYTHONPATH suitable to use in processes so that they find this installation of Scrapy

def get_testenv(): (source)

Return a OS environment dict suitable to fork processes that need to import this installation of Scrapy, instead of a system installed one.

def get_web_client_agent_req(url): (source)

Undocumented

def mock_google_cloud_storage(): (source)

Creates autospec mocks for google-cloud-storage Client, Bucket and Blob classes and set their proper return values.

def skip_if_no_boto(): (source)

Undocumented