Interface

This section of the documentation covers the interface portions of hyper.

Primary HTTP Interface

class hyper.HTTPConnection(host, port=None, secure=None, window_manager=None, enable_push=False, ssl_context=None, proxy_host=None, proxy_port=None, **kwargs)

An object representing a single HTTP connection to a server.

This object behaves similarly to the Python standard library’s HTTPConnection object, with a few critical differences.

Most of the standard library’s arguments to the constructor are not supported by hyper. Most optional parameters apply to either HTTP/1.1 or HTTP/2.

Parameters:
  • host – The host to connect to. This may be an IP address or a hostname, and optionally may include a port: for example, 'http2bin.org', 'http2bin.org:443' or '127.0.0.1'.
  • port – (optional) The port to connect to. If not provided and one also isn’t provided in the host parameter, defaults to 80.
  • secure – (optional) Whether the request should use TLS. Defaults to False for most requests, but to True for any request issued to port 443.
  • window_manager – (optional) The class to use to manage flow control windows. This needs to be a subclass of the BaseFlowControlManager. If not provided, FlowControlManager will be used.
  • enable_push – (optional) Whether the server is allowed to push resources to the client (see get_pushes()).
  • ssl_context – (optional) A class with custom certificate settings. If not provided then hyper’s default SSLContext is used instead.
  • proxy_host – (optional) The proxy to connect to. This can be an IP address or a host name and may include a port.
  • proxy_port – (optional) The proxy port to connect to. If not provided and one also isn’t provided in the proxy parameter, defaults to 8080.
get_response(*args, **kwargs)

Returns a response object.

request(method, url, body=None, headers=None)

This will send a request to the server using the HTTP request method method and the selector url. If the body argument is present, it should be string or bytes object of data to send after the headers are finished. Strings are encoded as UTF-8. To use other encodings, pass a bytes object. The Content-Length header is set to the length of the body field.

Parameters:
  • method – The request method, e.g. 'GET'.
  • url – The URL to contact, e.g. '/path/segment'.
  • body – (optional) The request body to send. Must be a bytestring or a file-like object.
  • headers – (optional) The headers to send on the request.
Returns:

A stream ID for the request, or None if the request is made over HTTP/1.1.

HTTP/2

class hyper.HTTP20Connection(host, port=None, secure=None, window_manager=None, enable_push=False, ssl_context=None, proxy_host=None, proxy_port=None, force_proto=None, **kwargs)

An object representing a single HTTP/2 connection to a server.

This object behaves similarly to the Python standard library’s HTTPConnection object, with a few critical differences.

Most of the standard library’s arguments to the constructor are irrelevant for HTTP/2 or not supported by hyper.

Parameters:
  • host – The host to connect to. This may be an IP address or a hostname, and optionally may include a port: for example, 'http2bin.org', 'http2bin.org:443' or '127.0.0.1'.
  • port – (optional) The port to connect to. If not provided and one also isn’t provided in the host parameter, defaults to 443.
  • secure – (optional) Whether the request should use TLS. Defaults to False for most requests, but to True for any request issued to port 443.
  • window_manager – (optional) The class to use to manage flow control windows. This needs to be a subclass of the BaseFlowControlManager. If not provided, FlowControlManager will be used.
  • enable_push – (optional) Whether the server is allowed to push resources to the client (see get_pushes()).
  • ssl_context – (optional) A class with custom certificate settings. If not provided then hyper’s default SSLContext is used instead.
  • proxy_host – (optional) The proxy to connect to. This can be an IP address or a host name and may include a port.
  • proxy_port – (optional) The proxy port to connect to. If not provided and one also isn’t provided in the proxy parameter, defaults to 8080.
close(error_code=None)

Close the connection to the server.

This method is thread-safe.

Parameters:error_code – (optional) The error code to reset all streams with.
Returns:Nothing.
connect()

Connect to the server specified when the object was created. This is a no-op if we’re already connected.

This method is thread-safe. It may be called from multiple threads, and is a noop for all threads apart from the first.

Returns:Nothing.
endheaders(message_body=None, final=False, stream_id=None)

Sends the prepared headers to the server. If the message_body argument is provided it will also be sent to the server as the body of the request, and the stream will immediately be closed. If the final argument is set to True, the stream will also immediately be closed: otherwise, the stream will be left open and subsequent calls to send() will be required.

Parameters:
  • message_body – (optional) The body to send. May not be provided assuming that send() will be called.
  • final – (optional) If the message_body parameter is provided, should be set to True if no further data will be provided via calls to send().
  • stream_id – (optional) The stream ID of the request to finish sending the headers on.
Returns:

Nothing.

get_pushes(stream_id=None, capture_all=False)

Returns a generator that yields push promises from the server. Note that this method is not idempotent: promises returned in one call will not be returned in subsequent calls. Iterating through generators returned by multiple calls to this method simultaneously results in undefined behavior.

Parameters:
  • stream_id – (optional) The stream ID of the request for which to get push promises.
  • capture_all – (optional) If False, the generator will yield all buffered push promises without blocking. If True, the generator will first yield all buffered push promises, then yield additional ones as they arrive, and terminate when the original stream closes.
Returns:

A generator of HTTP20Push objects corresponding to the streams pushed by the server.

get_response(stream_id=None)

Should be called after a request is sent to get a response from the server. If sending multiple parallel requests, pass the stream ID of the request whose response you want. Returns a HTTP20Response instance. If you pass no stream_id, you will receive the oldest HTTPResponse still outstanding.

This method is thread-safe.

Parameters:stream_id – (optional) The stream ID of the request for which to get a response.
Returns:A HTTP20Response object.
network_buffer_size = None

The size of the in-memory buffer used to store data from the network. This is used as a performance optimisation. Increase buffer size to improve performance: decrease it to conserve memory. Defaults to 64kB.

putheader(header, argument, stream_id=None, replace=False)

Sends an HTTP header to the server, with name header and value argument.

Unlike the httplib version of this function, this version does not actually send anything when called. Instead, it queues the headers up to be sent when you call endheaders().

This method ensures that headers conform to the HTTP/2 specification. In particular, it strips out the Connection header, as that header is no longer valid in HTTP/2. This is to make it easy to write code that runs correctly in both HTTP/1.1 and HTTP/2.

Parameters:
  • header – The name of the header.
  • argument – The value of the header.
  • stream_id – (optional) The stream ID of the request to add the header to.
Returns:

Nothing.

putrequest(method, selector, **kwargs)

This should be the first call for sending a given HTTP request to a server. It returns a stream ID for the given connection that should be passed to all subsequent request building calls.

This method is thread-safe. It can be called from multiple threads, and each thread should receive a unique stream ID.

Parameters:
  • method – The request method, e.g. 'GET'.
  • selector – The path selector.
Returns:

A stream ID for the request.

request(method, url, body=None, headers=None)

This will send a request to the server using the HTTP request method method and the selector url. If the body argument is present, it should be string or bytes object of data to send after the headers are finished. Strings are encoded as UTF-8. To use other encodings, pass a bytes object. The Content-Length header is set to the length of the body field.

This method is thread-safe.

Parameters:
  • method – The request method, e.g. 'GET'.
  • url – The URL to contact, e.g. '/path/segment'.
  • body – (optional) The request body to send. Must be a bytestring or a file-like object.
  • headers – (optional) The headers to send on the request.
Returns:

A stream ID for the request.

send(data, final=False, stream_id=None)

Sends some data to the server. This data will be sent immediately (excluding the normal HTTP/2 flow control rules). If this is the last data that will be sent as part of this request, the final argument should be set to True. This will cause the stream to be closed.

Parameters:
  • data – The data to send.
  • final – (optional) Whether this is the last bit of data to be sent on this request.
  • stream_id – (optional) The stream ID of the request to send the data on.
Returns:

Nothing.

class hyper.HTTP20Response(headers, stream)

An HTTP20Response wraps the HTTP/2 response from the server. It provides access to the response headers and the entity body. The response is an iterable object and can be used in a with statement (though due to the persistent connections used in HTTP/2 this has no effect, and is done soley for compatibility).

close()

Close the response. In effect this closes the backing HTTP/2 stream.

Returns:Nothing.
fileno()

Return the fileno of the underlying socket. This function is currently not implemented.

headers = None

The response headers. These are determined upon creation, assigned once, and never assigned again.

read(amt=None, decode_content=True)

Reads the response body, or up to the next amt bytes.

Parameters:
  • amt – (optional) The amount of data to read. If not provided, all the data will be read from the response.
  • decode_content – (optional) If True, will transparently decode the response data.
Returns:

The read data. Note that if decode_content is set to True, the actual amount of data returned may be different to the amount requested.

read_chunked(decode_content=True)

Reads chunked transfer encoded bodies. This method returns a generator: each iteration of which yields one data frame unless the frames contain compressed data and decode_content is True, in which case it yields whatever the decompressor provides for each chunk.

Warning

This may yield the empty string, without that being the end of the body!

reason = None

The reason phrase returned by the server. This is not used in HTTP/2, and so is always the empty string.

status = None

The status code returned by the server.

trailers

Trailers on the HTTP message, if any.

Warning

Note that this property requires that the stream is totally exhausted. This means that, if you have not completely read from the stream, all stream data will be read into memory.

class hyper.HTTP20Push(request_headers, stream)

Represents a request-response pair sent by the server through the server push mechanism.

authority = None

The authority of the simulated request (usually host:port)

cancel()

Cancel the pushed response and close the stream.

Returns:Nothing.
get_response()

Get the pushed response provided by the server.

Returns:A HTTP20Response object representing the pushed response.
method = None

The method of the simulated request (must be safe and cacheable, e.g. GET)

path = None

The path of the simulated request

request_headers = None

The headers the server attached to the simulated request.

scheme = None

The scheme of the simulated request

HTTP/1.1

class hyper.HTTP11Connection(host, port=None, secure=None, ssl_context=None, proxy_host=None, proxy_port=None, **kwargs)

An object representing a single HTTP/1.1 connection to a server.

Parameters:
  • host – The host to connect to. This may be an IP address or a hostname, and optionally may include a port: for example, 'twitter.com', 'twitter.com:443' or '127.0.0.1'.
  • port – (optional) The port to connect to. If not provided and one also isn’t provided in the host parameter, defaults to 80.
  • secure – (optional) Whether the request should use TLS. Defaults to False for most requests, but to True for any request issued to port 443.
  • ssl_context – (optional) A class with custom certificate settings. If not provided then hyper’s default SSLContext is used instead.
  • proxy_host – (optional) The proxy to connect to. This can be an IP address or a host name and may include a port.
  • proxy_port – (optional) The proxy port to connect to. If not provided and one also isn’t provided in the proxy parameter, defaults to 8080.
close()

Closes the connection. This closes the socket and then abandons the reference to it. After calling this method, any outstanding Response objects will throw exceptions if attempts are made to read their bodies.

In some cases this method will automatically be called.

Warning

This method should absolutely only be called when you are certain the connection object is no longer needed.

connect()

Connect to the server specified when the object was created. This is a no-op if we’re already connected.

Returns:Nothing.
get_response()

Returns a response object.

This is an early beta, so the response object is pretty stupid. That’s ok, we’ll fix it later.

network_buffer_size = None

The size of the in-memory buffer used to store data from the network. This is used as a performance optimisation. Increase buffer size to improve performance: decrease it to conserve memory. Defaults to 64kB.

parser = None

The object used to perform HTTP/1.1 parsing. Needs to conform to the standard hyper parsing interface.

request(method, url, body=None, headers=None)

This will send a request to the server using the HTTP request method method and the selector url. If the body argument is present, it should be string or bytes object of data to send after the headers are finished. Strings are encoded as UTF-8. To use other encodings, pass a bytes object. The Content-Length header is set to the length of the body field.

Parameters:
  • method – The request method, e.g. 'GET'.
  • url – The URL to contact, e.g. '/path/segment'.
  • body – (optional) The request body to send. Must be a bytestring, an iterable of bytestring, or a file-like object.
  • headers – (optional) The headers to send on the request.
Returns:

Nothing.

class hyper.HTTP11Response(code, reason, headers, sock, connection=None)

An HTTP11Response wraps the HTTP/1.1 response from the server. It provides access to the response headers and the entity body. The response is an iterable object and can be used in a with statement.

close(socket_close=False)

Close the response. This causes the Response to lose access to the backing socket. In some cases, it can also cause the backing connection to be torn down.

Parameters:socket_close – Whether to close the backing socket.
Returns:Nothing.
headers = None

The response headers. These are determined upon creation, assigned once, and never assigned again.

read(amt=None, decode_content=True)

Reads the response body, or up to the next amt bytes.

Parameters:
  • amt – (optional) The amount of data to read. If not provided, all the data will be read from the response.
  • decode_content – (optional) If True, will transparently decode the response data.
Returns:

The read data. Note that if decode_content is set to True, the actual amount of data returned may be different to the amount requested.

read_chunked(decode_content=True)

Reads chunked transfer encoded bodies. This method returns a generator: each iteration of which yields one chunk unless the chunks are compressed, in which case it yields whatever the decompressor provides for each chunk.

Warning

This may yield the empty string, without that being the end of the body!

reason = None

The reason phrase returned by the server.

status = None

The status code returned by the server.

trailers = None

The response trailers. These are always intially None.

Headers

class hyper.common.headers.HTTPHeaderMap(*args, **kwargs)

A structure that contains HTTP headers.

HTTP headers are a curious beast. At the surface level they look roughly like a name-value set, but in practice they have many variations that make them tricky:

  • duplicate keys are allowed
  • keys are compared case-insensitively
  • duplicate keys are isomorphic to comma-separated values, except when they aren’t!
  • they logically contain a form of ordering

This data structure is an attempt to preserve all of that information while being as user-friendly as possible. It retains all of the mapping convenience methods (allowing by-name indexing), while avoiding using a dictionary for storage.

When iterated over, this structure returns headers in ‘canonical form’. This form is a tuple, where the first entry is the header name (in lower-case), and the second entry is a list of header values (in original case).

The mapping always emits both names and values in the form of bytestrings: never unicode strings. It can accept names and values in unicode form, and will automatically be encoded to bytestrings using UTF-8. The reason for what appears to be a user-unfriendly decision here is primarily to allow the broadest-possible compatibility (to make it possible to send headers in unusual encodings) while ensuring that users are never confused about what type of data they will receive.

Warning

Note that this data structure makes none of the performance guarantees of a dictionary. Lookup and deletion is not an O(1) operation. Inserting a new value is O(1), all other operations are O(n), including replacing a header entirely.

clear() → None. Remove all items from D.
get(name, default=None)

Unlike the dict get, this returns a list of items in the order they were added.

items()

This mapping iterates like the list of tuples it is.

iter_raw()

Allows iterating over the headers in ‘raw’ form: that is, the form in which they were added to the structure. This iteration is in order, and can be used to rebuild the original headers (e.g. to determine exactly what a server sent).

keys()

Returns an iterable of the header keys in the mapping. This explicitly does not filter duplicates, ensuring that it’s the same length as len().

merge(other)

Merge another header set or any other dict-like into this one.

pop(k[, d]) → v, remove specified key and return the corresponding value.

If key is not found, d is returned if given, otherwise KeyError is raised.

popitem() → (k, v), remove and return some (key, value) pair

as a 2-tuple; but raise KeyError if D is empty.

replace(key, value)

Replace existing header with new value. If header doesn’t exist this method work like __setitem__. Replacing leads to deletion of all existing headers with the same name.

setdefault(k[, d]) → D.get(k,d), also set D[k]=d if k not in D
update([E, ]**F) → None. Update D from mapping/iterable E and F.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

values()

This is an almost nonsensical query on a header dictionary, but we satisfy it in the exact same way we satisfy ‘keys’.

SSLContext

tls.init_context(cert_path=None, cert=None, cert_password=None)

Create a new SSLContext that is correctly set up for an HTTP/2 connection. This SSL context object can be customized and passed as a parameter to the HTTPConnection class. Provide your own certificate file in case you don’t want to use hyper’s default certificate. The path to the certificate can be absolute or relative to your working directory.

Parameters:
  • cert_path – (optional) The path to the certificate file of “certification authority” (CA) certificates
  • cert – (optional) if string, path to ssl client cert file (.pem). If tuple, (‘cert’, ‘key’) pair. The certfile string must be the path to a single file in PEM format containing the certificate as well as any number of CA certificates needed to establish the certificate’s authenticity. The keyfile string, if present, must point to a file containing the private key in. Otherwise the private key will be taken from certfile as well.
  • cert_password – (optional) The password argument may be a function to call to get the password for decrypting the private key. It will only be called if the private key is encrypted and a password is necessary. It will be called with no arguments, and it should return a string, bytes, or bytearray. If the return value is a string it will be encoded as UTF-8 before using it to decrypt the key. Alternatively a string, bytes, or bytearray value may be supplied directly as the password argument. It will be ignored if the private key is not encrypted and no password is needed.
Returns:

An SSLContext correctly set up for HTTP/2.

Requests Transport Adapter

class hyper.contrib.HTTP20Adapter(*args, **kwargs)

A Requests Transport Adapter that uses hyper to send requests over HTTP/2. This implements some degree of connection pooling to maximise the HTTP/2 gain.

add_headers(request, **kwargs)

Add any headers needed by the connection. As of v2.0 this does nothing by default, but is left for overriding by users that subclass the HTTPAdapter.

This should not be called from user code, and is only exposed for use when subclassing the HTTPAdapter.

Parameters:
  • request – The PreparedRequest to add headers to.
  • kwargs – The keyword arguments from the call to send().
build_response(request, resp)

Builds a Requests’ response object. This emulates most of the logic of the standard fuction but deals with the lack of the .headers property on the HTTP20Response object.

Additionally, this function builds in a number of features that are purely for HTTPie. This is to allow maximum compatibility with what urllib3 does, so that HTTPie doesn’t fall over when it uses us.

cert_verify(conn, url, verify, cert)

Verify a SSL certificate. This method should not be called from user code, and is only exposed for use when subclassing the HTTPAdapter.

Parameters:
  • conn – The urllib3 connection object associated with the cert.
  • url – The requested URL.
  • verify – Whether we should actually verify the certificate.
  • cert – The SSL certificate to verify.
close()

Disposes of any internal state.

Currently, this closes the PoolManager and any active ProxyManager, which closes any pooled connections.

connections = None

A mapping between HTTP netlocs and HTTP20Connection objects.

get_connection(host, port, scheme, cert=None)

Gets an appropriate HTTP/2 connection object based on host/port/scheme/cert tuples.

init_poolmanager(connections, maxsize, block=False, **pool_kwargs)

Initializes a urllib3 PoolManager.

This method should not be called from user code, and is only exposed for use when subclassing the HTTPAdapter.

Parameters:
  • connections – The number of urllib3 connection pools to cache.
  • maxsize – The maximum number of connections to save in the pool.
  • block – Block when no free connections are available.
  • pool_kwargs – Extra keyword arguments used to initialize the Pool Manager.
proxy_headers(proxy)

Returns a dictionary of the headers to add to any request sent through a proxy. This works with urllib3 magic to ensure that they are correctly sent to the proxy, rather than in a tunnelled request if CONNECT is being used.

This should not be called from user code, and is only exposed for use when subclassing the HTTPAdapter.

Parameters:proxies – The url of the proxy being used for this request.
proxy_manager_for(proxy, **proxy_kwargs)

Return urllib3 ProxyManager for the given proxy.

This method should not be called from user code, and is only exposed for use when subclassing the HTTPAdapter.

Parameters:
  • proxy – The proxy to return a urllib3 ProxyManager for.
  • proxy_kwargs – Extra keyword arguments used to configure the Proxy Manager.
Returns:

ProxyManager

request_url(request, proxies)

Obtain the url to use when making the final request.

If the message is being sent through a HTTP proxy, the full URL has to be used. Otherwise, we should only use the path portion of the URL.

This should not be called from user code, and is only exposed for use when subclassing the HTTPAdapter.

Parameters:
  • request – The PreparedRequest being sent.
  • proxies – A dictionary of schemes or schemes and hosts to proxy URLs.
send(request, stream=False, cert=None, **kwargs)

Sends a HTTP message to the server.

Flow Control

class hyper.http20.window.BaseFlowControlManager(initial_window_size, document_size=None)

The abstract base class for flow control managers.

This class defines the interface for pluggable flow-control managers. A flow-control manager defines a flow-control policy, which basically boils down to deciding when to increase the flow control window.

This decision can be based on a number of factors:

  • the initial window size,
  • the size of the document being retrieved,
  • the size of the received data frames,
  • any other information the manager can obtain

A flow-control manager may be defined at the connection level or at the stream level. If no stream-level flow-control manager is defined, an instance of the connection-level flow control manager is used.

A class that inherits from this one must not adjust the member variables defined in this class. They are updated and set by methods on this class.

blocked()

Called whenever the remote endpoint reports that it is blocked behind the flow control window.

When this method is called the remote endpoint is signaling that it has more data to send and that the transport layer is capable of transmitting it, but that the HTTP/2 flow control window prevents it being sent.

This method should return the size by which the window should be incremented, which may be zero. This method should not adjust any of the member variables of this class.

Returns:The amount to increase the receive window by. Return zero if the window should not be increased.
document_size = None

The size of the document being retrieved, in bytes. This is retrieved from the Content-Length header, if provided. Note that the total number of bytes that will be received may be larger than this value due to HTTP/2 padding. It should not be assumed that simply because the the document size is smaller than the initial window size that there will never be a need to increase the window size.

increase_window_size(frame_size)

Determine whether or not to emit a WINDOWUPDATE frame.

This method should be overridden to determine, based on the state of the system and the size of the received frame, whether or not a WindowUpdate frame should be sent for the stream.

This method should not adjust any of the member variables of this class.

Note that this method is called before the window size is decremented as a result of the frame being handled.

Parameters:frame_size – The size of the received frame. Note that this may be zero. When this parameter is zero, it’s possible that a WINDOWUPDATE frame may want to be emitted anyway. A zero-length frame size is usually associated with a change in the size of the receive window due to a SETTINGS frame.
Returns:The amount to increase the receive window by. Return zero if the window should not be increased.
initial_window_size = None

The initial size of the connection window in bytes. This is set at creation time.

window_size = None

The current size of the connection window. Any methods overridden by the user must not adjust this value.

class hyper.http20.window.FlowControlManager(initial_window_size, document_size=None)

hyper‘s default flow control manager.

This implements hyper’s flow control algorithms. This algorithm attempts to reduce the number of WINDOWUPDATE frames we send without blocking the remote endpoint behind the flow control window.

This algorithm will become more complicated over time. In the current form, the algorithm is very simple:

  • When the flow control window gets less than 1/4 of the maximum size, increment back to the maximum.
  • Otherwise, if the flow control window gets to less than 1kB, increment back to the maximum.

Exceptions

class hyper.http20.exceptions.HTTP20Error

The base class for all of hyper‘s HTTP/2-related exceptions.

class hyper.http20.exceptions.HPACKEncodingError

An error has been encountered while performing HPACK encoding.

class hyper.http20.exceptions.HPACKDecodingError

An error has been encountered while performing HPACK decoding.

class hyper.http20.exceptions.ConnectionError

The remote party signalled an error affecting the entire HTTP/2 connection, and the connection has been closed.