scrapy.core.http2.protocol.H2ClientProtocol

class documentation

class H2ClientProtocol(Protocol, TimeoutMixin): (source)

Implements interfaces: twisted.internet.interfaces.IHandshakeListener

View In Hierarchy

Undocumented

Method __init__ Arguments: uri -- URI of the base url to which HTTP/2 Connection will be made. uri is used to verify that incoming client requests have correct base URL. settings -- Scrapy project settings conn_lost_deferred -- Deferred fires with the reason: Failure to notify that connection was lost...
Method connection_terminated Undocumented
Method connectionLost Called by Twisted when the transport connection is lost. No need to write anything to transport here.
Method connectionMade Called by Twisted when the connection is established. We can start sending some data now: we should open with the connection preamble.
Method data_received Undocumented
Method dataReceived Undocumented
Method handshakeCompleted Close the connection if it's not made via the expected protocol
Method pop_stream Perform cleanup when a stream is closed
Method request Undocumented
Method response_received Undocumented
Method settings_acknowledged Undocumented
Method stream_ended Undocumented
Method stream_reset Undocumented
Method timeoutConnection Called when the connection times out. We lose the connection with TimeoutError
Method window_updated Undocumented
Constant IDLE_TIMEOUT Undocumented
Instance Variable conn Undocumented
Instance Variable metadata Undocumented
Instance Variable streams Undocumented
Property allowed_max_concurrent_streams We keep total two streams for client (sending data) and server side (receiving data) for a single request. To be safe we choose the minimum. Since this value can change in event RemoteSettingsChanged we make variable a property.
Property h2_connected Boolean to keep track of the connection status. This is used while initiating pending streams to make sure that we initiate stream only during active HTTP/2 Connection
Method _check_received_data Checks for edge cases where the connection to remote fails without raising an appropriate H2Error
Method _handle_events Private method which acts as a bridge between the events received from the HTTP/2 data and IH2EventsHandler
Method _lose_connection_with_error Helper function to lose the connection with the error sent as a reason
Method _new_stream Instantiates a new Stream object
Method _send_pending_requests Initiate all pending requests from the deque following FIFO We make sure that at any time {allowed_max_concurrent_streams} streams are active.
Method _write_to_transport Write data to the underlying transport connection from the HTTP2 connection instance if any
Instance Variable _conn_lost_deferred Undocumented
Instance Variable _conn_lost_errors Undocumented
Instance Variable _pending_request_stream_pool Undocumented
Instance Variable _stream_id_generator Undocumented
def __init__(self, uri: URI, settings: Settings, conn_lost_deferred: Deferred): (source)

Arguments: uri -- URI of the base url to which HTTP/2 Connection will be made. uri is used to verify that incoming client requests have correct base URL. settings -- Scrapy project settings conn_lost_deferred -- Deferred fires with the reason: Failure to notify that connection was lost

def connection_terminated(self, event: ConnectionTerminated): (source)

Undocumented

def connectionLost(self, reason: Failure = connectionDone): (source)

Called by Twisted when the transport connection is lost. No need to write anything to transport here.

def connectionMade(self): (source)

Called by Twisted when the connection is established. We can start sending some data now: we should open with the connection preamble.

def data_received(self, event: DataReceived): (source)

Undocumented

def dataReceived(self, data: bytes): (source)

Undocumented

def handshakeCompleted(self): (source)

Close the connection if it's not made via the expected protocol

def pop_stream(self, stream_id: int) -> Stream: (source)

Perform cleanup when a stream is closed

def request(self, request: Request, spider: Spider) -> Deferred: (source)

Undocumented

def response_received(self, event: ResponseReceived): (source)

Undocumented

def settings_acknowledged(self, event: SettingsAcknowledged): (source)

Undocumented

def stream_ended(self, event: StreamEnded): (source)

Undocumented

def stream_reset(self, event: StreamReset): (source)

Undocumented

def timeoutConnection(self): (source)

Called when the connection times out. We lose the connection with TimeoutError

def window_updated(self, event: WindowUpdated): (source)

Undocumented

IDLE_TIMEOUT: int = (source)

Undocumented

Value
240
conn = (source)

Undocumented

metadata: Dict = (source)

Undocumented

streams: Dict[int, Stream] = (source)

Undocumented

@property
allowed_max_concurrent_streams: int = (source)

We keep total two streams for client (sending data) and server side (receiving data) for a single request. To be safe we choose the minimum. Since this value can change in event RemoteSettingsChanged we make variable a property.

@property
h2_connected: bool = (source)

Boolean to keep track of the connection status. This is used while initiating pending streams to make sure that we initiate stream only during active HTTP/2 Connection

def _check_received_data(self, data: bytes): (source)

Checks for edge cases where the connection to remote fails without raising an appropriate H2Error Arguments: data -- Data received from the remote

def _handle_events(self, events: List[Event]): (source)

Private method which acts as a bridge between the events received from the HTTP/2 data and IH2EventsHandler Arguments: events -- A list of events that the remote peer triggered by sending data

def _lose_connection_with_error(self, errors: List[BaseException]): (source)

Helper function to lose the connection with the error sent as a reason

def _new_stream(self, request: Request, spider: Spider) -> Stream: (source)

Instantiates a new Stream object

def _send_pending_requests(self): (source)

Initiate all pending requests from the deque following FIFO We make sure that at any time {allowed_max_concurrent_streams} streams are active.

def _write_to_transport(self): (source)

Write data to the underlying transport connection from the HTTP2 connection instance if any

_conn_lost_deferred = (source)

Undocumented

_conn_lost_errors: List[BaseException] = (source)

Undocumented

_pending_request_stream_pool: deque = (source)

Undocumented

_stream_id_generator = (source)

Undocumented