gem-hunter

Event-driven, streaming HTTP client and server implementation for ReactPHP.

Development version: This branch contains the code for the upcoming v3 release. For the code of the current stable v1 release, check out the 1.x branch.

The upcoming v3 release will be the way forward for this package. However, we will still actively support v1 for those not yet on the latest version. See also installation instructions for more details.

This HTTP library provides re-usable implementations for an HTTP client and server based on ReactPHP's Socket and EventLoop components. Its client component allows you to send any number of async HTTP/HTTPS requests concurrently. Its server component allows you to build plaintext HTTP and secure HTTPS servers that accept incoming HTTP requests from HTTP clients (such as web browsers). This library provides async, streaming means for all of this, so you can handle multiple concurrent HTTP requests without blocking.

Table of contents

• Quickstart example
• Client Usage
  • Request methods
  • Promises
  • Cancellation
  • Timeouts
  • Authentication
  • Redirects
  • Blocking
  • Concurrency
  • Streaming response
  • Streaming request
  • HTTP proxy
  • SOCKS proxy
  • SSH proxy
  • Unix domain sockets
• Server Usage
  • HttpServer
  • listen()
  • Server Request
    • Request parameters
    • Query parameters
    • Request body
    • Streaming incoming request
    • Request method
    • Cookie parameters
    • Invalid request
  • Server Response
    • Deferred response
    • Streaming outgoing response
    • Response length
    • Invalid response
    • Default response headers
  • Middleware
    • Custom middleware
    • Third-Party Middleware
• API
  • Browser
    • get()
    • post()
    • head()
    • patch()
    • put()
    • delete()
    • request()
    • requestStreaming()
    • withTimeout()
    • withFollowRedirects()
    • withRejectErrorResponse()
    • withBase()
    • withProtocolVersion()
    • withResponseBuffer()
    • withHeader()
    • withoutHeader()
  • React\Http\Message
    • Response
      • html()
      • json()
      • plaintext()
      • xml()
    • Request
    • ServerRequest
    • Uri
    • ResponseException
  • React\Http\Middleware
    • StreamingRequestMiddleware
    • LimitConcurrentRequestsMiddleware
    • RequestBodyBufferMiddleware
    • RequestBodyParserMiddleware
• Install
• Tests
• License

Once installed, you can use the following code to access an HTTP web server and send some simple HTTP GET requests:

<?php

require __DIR__ . '/vendor/autoload.php';

$client = new React\Http\Browser();

$client->get('http://www.google.com/')->then(function (Psr\Http\Message\ResponseInterface $response) {
    var_dump($response->getHeaders(), (string)$response->getBody());
}, function (Exception $e) {
    echo 'Error: ' . $e->getMessage() . PHP_EOL;
});

This is an HTTP server which responds with Hello World! to every request.

<?php

require __DIR__ . '/vendor/autoload.php';

$http = new React\Http\HttpServer(function (Psr\Http\Message\ServerRequestInterface $request) {
    return React\Http\Message\Response::plaintext(
        "Hello World!\n"
    );
});

$socket = new React\Socket\SocketServer('127.0.0.1:8080');
$http->listen($socket);

See also the examples.

Most importantly, this project provides a Browser object that offers several methods that resemble the HTTP protocol methods:

$browser->get($url, array $headers = []);
$browser->head($url, array $headers = []);
$browser->post($url, array $headers = [], string|ReadableStreamInterface $body = '');
$browser->delete($url, array $headers = [], string|ReadableStreamInterface $body = '');
$browser->put($url, array $headers = [], string|ReadableStreamInterface $body = '');
$browser->patch($url, array $headers = [], string|ReadableStreamInterface $body = '');

Each of these methods requires a $url and some optional parameters to send an HTTP request. Each of these method names matches the respective HTTP request method, for example the get() method sends an HTTP GET request.

You can optionally pass an associative array of additional $headers that will be sent with this HTTP request. Additionally, each method will automatically add a matching Content-Length request header if an outgoing request body is given and its size is known and non-empty. For an empty request body, if will only include a Content-Length: 0 request header if the request method usually expects a request body (only applies to POST, PUT and PATCH HTTP request methods).

If you're using a streaming request body, it will default to using Transfer-Encoding: chunked unless you explicitly pass in a matching Content-Length request header. See also streaming request for more details.

By default, all of the above methods default to sending requests using the HTTP/1.1 protocol version. If you want to explicitly use the legacy HTTP/1.0 protocol version, you can use the withProtocolVersion() method. If you want to use any other or even custom HTTP request method, you can use the request() method.

Each of the above methods supports async operation and either fulfills with a PSR-7 ResponseInterface or rejects with an Exception. Please see the following chapter about promises for more details.

Sending requests is async (non-blocking), so you can actually send multiple requests in parallel. The Browser will respond to each request with a PSR-7 ResponseInterface message, the order is not guaranteed. Sending requests uses a Promise-based interface that makes it easy to react to when an HTTP request is completed (i.e. either successfully fulfilled or rejected with an error):

$browser->get($url)->then(
    function (Psr\Http\Message\ResponseInterface $response) {
        var_dump('Response received', $response);
    },
    function (Exception $e) {
        echo 'Error: ' . $e->getMessage() . PHP_EOL;
    }
);

If this looks strange to you, you can also use the more traditional blocking API.

Keep in mind that resolving the Promise with the full response message means the whole response body has to be kept in memory. This is easy to get started and works reasonably well for smaller responses (such as common HTML pages or RESTful or JSON API requests).

You may also want to look into the streaming API:

• If you're dealing with lots of concurrent requests (100+) or
• If you want to process individual data chunks as they happen (without having to wait for the full response body) or
• If you're expecting a big response body size (1 MiB or more, for example when downloading binary files) or
• If you're unsure about the response body size (better be safe than sorry when accessing arbitrary remote HTTP endpoints and the response body size is unknown in advance).

The returned Promise is implemented in such a way that it can be cancelled when it is still pending. Cancelling a pending promise will reject its value with an Exception and clean up any underlying resources.

$promise = $browser->get($url);

Loop::addTimer(2.0, function () use ($promise) {
    $promise->cancel();
});

This library uses a very efficient HTTP implementation, so most HTTP requests should usually be completed in mere milliseconds. However, when sending HTTP requests over an unreliable network (the internet), there are a number of things that can go wrong and may cause the request to fail after a time. As such, this library respects PHP's default_socket_timeout setting (default 60s) as a timeout for sending the outgoing HTTP request and waiting for a successful response and will otherwise cancel the pending request and reject its value with an Exception.

Note that this timeout value covers creating the underlying transport connection, sending the HTTP request, receiving the HTTP response headers and its full response body and following any eventual redirects. See also redirects below to configure the number of redirects to follow (or disable following redirects altogether) and also streaming below to not take receiving large response bodies into account for this timeout.

You can use the withTimeout() method to pass a custom timeout value in seconds like this:

$browser = $browser->withTimeout(10.0);

$browser->get($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    // response received within 10 seconds maximum
    var_dump($response->getHeaders());
}, function (Exception $e) {
    echo 'Error: ' . $e->getMessage() . PHP_EOL;
});

Similarly, you can use a bool false to not apply a timeout at all or use a bool true value to restore the default handling. See withTimeout() for more details.

If you're using a streaming response body, the time it takes to receive the response body stream will not be included in the timeout. This allows you to keep this incoming stream open for a longer time, such as when downloading a very large stream or when streaming data over a long-lived connection.

If you're using a streaming request body, the time it takes to send the request body stream will not be included in the timeout. This allows you to keep this outgoing stream open for a longer time, such as when uploading a very large stream.

Note that this timeout handling applies to the higher-level HTTP layer. Lower layers such as socket and DNS may also apply (different) timeout values. In particular, the underlying socket connection uses the same default_socket_timeout setting to establish the underlying transport connection. To control this connection timeout behavior, you can inject a custom Connector like this:

$browser = new React\Http\Browser(
    new React\Socket\Connector(
        [
            'timeout' => 5
        ]
    )
);

This library supports HTTP Basic Authentication using the Authorization: Basic … request header or allows you to set an explicit Authorization request header.

By default, this library does not include an outgoing Authorization request header. If the server requires authentication, if may return a 401 (Unauthorized) status code which will reject the request by default (see also the withRejectErrorResponse() method below).

In order to pass authentication details, you can simply pass the username and password as part of the request URL like this:

$promise = $browser->get('https://user:pass@example.com/api');

Note that special characters in the authentication details have to be percent-encoded, see also rawurlencode(). This example will automatically pass the base64-encoded authentication details using the outgoing Authorization: Basic … request header. If the HTTP endpoint you're talking to requires any other authentication scheme, you can also pass this header explicitly. This is common when using (RESTful) HTTP APIs that use OAuth access tokens or JSON Web Tokens (JWT):

$token = 'abc123';

$promise = $browser->get(
    'https://example.com/api',
    [
        'Authorization' => 'Bearer ' . $token
    ]
);

When following redirects, the Authorization request header will never be sent to any remote hosts by default. When following a redirect where the Location response header contains authentication details, these details will be sent for following requests. See also redirects below.

By default, this library follows any redirects and obeys 3xx (Redirection) status codes using the Location response header from the remote server. The promise will be fulfilled with the last response from the chain of redirects.

$browser->get($url, $headers)->then(function (Psr\Http\Message\ResponseInterface $response) {
    // the final response will end up here
    var_dump($response->getHeaders());
}, function (Exception $e) {
    echo 'Error: ' . $e->getMessage() . PHP_EOL;
});

Any redirected requests will follow the semantics of the original request and will include the same request headers as the original request except for those listed below. If the original request is a temporary (307) or a permanent (308) redirect, request body and headers will be passed to the redirected request. Otherwise, the request body will never be passed to the redirected request. Accordingly, each redirected request will remove any Content-Length and Content-Type request headers.

If the original request used HTTP authentication with an Authorization request header, this request header will only be passed as part of the redirected request if the redirected URL is using the same host. In other words, the Authorizaton request header will not be forwarded to other foreign hosts due to possible privacy/security concerns. When following a redirect where the Location response header contains authentication details, these details will be sent for following requests.

You can use the withFollowRedirects() method to control the maximum number of redirects to follow or to return any redirect responses as-is and apply custom redirection logic like this:

$browser = $browser->withFollowRedirects(false);

$browser->get($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    // any redirects will now end up here
    var_dump($response->getHeaders());
}, function (Exception $e) {
    echo 'Error: ' . $e->getMessage() . PHP_EOL;
});

See also withFollowRedirects() for more details.

As stated above, this library provides you a powerful, async API by default.

You can also integrate this into your traditional, blocking environment by using reactphp/async. This allows you to simply await async HTTP requests like this:

use function React\Async\await;

$browser = new React\Http\Browser();

$promise = $browser->get('http://example.com/');

try {
    $response = await($promise);
    // response successfully received
} catch (Exception $e) {
    // an error occurred while performing the request
}

Similarly, you can also process multiple requests concurrently and await an array of Response objects:

use function React\Async\await;
use function React\Promise\all;

$promises = [
    $browser->get('http://example.com/'),
    $browser->get('http://www.example.org/'),
];

$responses = await(all($promises));

This is made possible thanks to fibers available in PHP 8.1+ and our compatibility API that also works on all supported PHP versions. Please refer to reactphp/async for more details.

Keep in mind the above remark about buffering the whole response message in memory. As an alternative, you may also see one of the following chapters for the streaming API.

As stated above, this library provides you a powerful, async API. Being able to send a large number of requests at once is one of the core features of this project. For instance, you can easily send 100 requests concurrently while processing SQL queries at the same time.

Remember, with great power comes great responsibility. Sending an excessive number of requests may either take up all resources on your side or it may even get you banned by the remote side if it sees an unreasonable number of requests from your side.

// watch out if array contains many elements
foreach ($urls as $url) {
    $browser->get($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
        var_dump($response->getHeaders());
    }, function (Exception $e) {
        echo 'Error: ' . $e->getMessage() . PHP_EOL;
    });
}

As a consequence, it's usually recommended to limit concurrency on the sending side to a reasonable value. It's common to use a rather small limit, as doing more than a dozen of things at once may easily overwhelm the receiving side. You can use clue/reactphp-mq as a lightweight in-memory queue to concurrently do many (but not too many) things at once:

// wraps Browser in a Queue object that executes no more than 10 operations at once
$q = new Clue\React\Mq\Queue(10, null, function ($url) use ($browser) {
    return $browser->get($url);
});

foreach ($urls as $url) {
    $q($url)->then(function (Psr\Http\Message\ResponseInterface $response) {
        var_dump($response->getHeaders());
    }, function (Exception $e) {
        echo 'Error: ' . $e->getMessage() . PHP_EOL;
    });
}

Additional requests that exceed the concurrency limit will automatically be enqueued until one of the pending requests completes. This integrates nicely with the existing Promise-based API. Please refer to clue/reactphp-mq for more details.

This in-memory approach works reasonably well for some thousand outstanding requests. If you're processing a very large input list (think millions of rows in a CSV or NDJSON file), you may want to look into using a streaming approach instead. See clue/reactphp-flux for more details.

All of the above examples assume you want to store the whole response body in memory. This is easy to get started and works reasonably well for smaller responses.

However, there are several situations where it's usually a better idea to use a streaming approach, where only small chunks have to be kept in memory:

• If you're dealing with lots of concurrent requests (100+) or
• If you want to process individual data chunks as they happen (without having to wait for the full response body) or
• If you're expecting a big response body size (1 MiB or more, for example when downloading binary files) or
• If you're unsure about the response body size (better be safe than sorry when accessing arbitrary remote HTTP endpoints and the response body size is unknown in advance).

You can use the requestStreaming() method to send an arbitrary HTTP request and receive a streaming response. It uses the same HTTP message API, but does not buffer the response body in memory. It only processes the response body in small chunks as data is received and forwards this data through ReactPHP's Stream API. This works for (any number of) responses of arbitrary sizes.

This means it resolves with a normal PSR-7 ResponseInterface, which can be used to access the response message parameters as usual. You can access the message body as usual, however it now also implements ReactPHP's ReadableStreamInterface as well as parts of the PSR-7 StreamInterface.

$browser->requestStreaming('GET', $url)->then(function (Psr\Http\Message\ResponseInterface $response) {
    $body = $response->getBody();
    assert($body instanceof Psr\Http\Message\StreamInterface);
    assert($body instanceof React\Stream\ReadableStreamInterface);

    $body->on('data', function ($chunk) {
        echo $chunk;
    });

    $body->on('error', function (Exception $e) {
        echo 'Error: ' . $e->getMessage() . PHP_EOL;
    });

    $body->on('close', function () {
        echo '[DONE]' . PHP_EOL;
    });
}, function (Exception $e) {
    echo 'Error: ' . $e->getMessage() . PHP_EOL;
});

See also the stream download benchmark example and the stream forwarding example.

You can invoke the following methods on the message body:

$body->on($event, $callback);
$body->eof();
$body->isReadable();
$body->pipe(React\Stream\WritableStreamInterface $dest, array $options = []);
$body->close();
$body->pause();
$body->resume();

Because the message body is in a streaming state, invoking the following methods doesn't make much sense:

$body->__toString(); // ''
$body->detach(); // throws BadMethodCallException
$body->getSize(); // null
$body->tell(); // throws BadMethodCallException
$body->isSeekable(); // false
$body->seek(); // throws BadMethodCallException
$body->rewind(); // throws BadMethodCallException
$body->isWritable(); // false
$body->write(); // throws BadMethodCallException
$body->read(); // throws BadMethodCallException
$body->getContents(); // throws BadMethodCallException

Note how timeouts apply slightly differently when using streaming. In streaming mode, the timeout value covers creating the underlying transport connection, sending the HTTP request, receiving the HTTP response headers and following any eventual redirects. In particular, the timeout value does not take receiving (possibly large) response bodies into account.

If you want to integrate the streaming response into a higher level API, then working with Promise objects that resolve with Stream objects is often inconvenient. Consider looking into also using react/promise-stream. The resulting streaming code could look something like this:

use function React\Promise\Stream\unwrapReadable;

function download(Browser $browser, string $url): React\Stream\ReadableStreamInterface {
    return unwrapReadable(
        $browser->requestStreaming('GET', $url)->then(function (Psr\Http\Message\ResponseInterface $response) {
            return $response->getBody();
        })
    );
}

$stream = download($browser, $url);
$stream->on('data', function ($data) {
    echo $data;
});
$stream->on('error', function (Exception $e) {
    echo 'Error: ' . $e->getMessage() . PHP_EOL;
});

See also the requestStreaming() method for more details.

Besides streaming the response body, you can also stream the request body. This can be useful if you want to send big POST requests (uploading files etc.) or process many outgoing streams at once. Instead of passing the body as a string, you can simply pass an instance implementing ReactPHP's ReadableStreamInterface to the request methods like this:

$browser->post($url, [], $stream)->then(function (Psr\Http\Message\ResponseInterface $response) {
    echo 'Successfully sent.';
}, function (Exception $e) {
    echo 'Error: ' . $e->getMessage() . PHP_EOL;
});

If you're using a streaming request body (React\Stream\ReadableStreamInterface), it will default to using Transfer-Encoding: chunked or you have to explicitly pass in a matching Content-Length request header like so:

$body = new React\Stream\ThroughStream();
Loop::addTimer(1.0, function () use ($body) {
    $body->end("hello world");
});

$browser->post($url, ['Content-Length' => '11'], $body);

If the streaming request body emits an error event or is explicitly closed without emitting a successful end event first, the request will automatically be closed and rejected.

You can also establish your outgoing connections through an HTTP CONNECT proxy server by adding a dependency to clue/reactphp-http-proxy.

HTTP CONNECT proxy servers (also commonly known as "HTTPS proxy" or "SSL proxy") are commonly used to tunnel HTTPS traffic through an intermediary ("proxy"), to conceal the origin address (anonymity) or to circumvent address blocking (geoblocking). While many (public) HTTP CONNECT proxy servers often limit this to HTTPS port 443 only, this can technically be used to tunnel any TCP/IP-based protocol, such as plain HTTP and TLS-encrypted HTTPS.

$proxy = new Clue\React\HttpProxy\ProxyConnector('127.0.0.1:8080');

$connector = new React\Socket\Connector([
    'tcp' => $proxy,
    'dns' => false
]);

$browser = new React\Http\Browser($connector);

See also the HTTP proxy example.

You can also establish your outgoing connections through a SOCKS proxy server by adding a dependency to clue/reactphp-socks.

The SOCKS proxy protocol family (SOCKS5, SOCKS4 and SOCKS4a) is commonly used to tunnel HTTP(S) traffic through an intermediary ("proxy"), to conceal the origin address (anonymity) or to circumvent address blocking (geoblocking). While many (public) SOCKS proxy servers often limit this to HTTP(S) port 80 and 443 only, this can technically be used to tunnel any TCP/IP-based protocol.

$proxy = new Clue\React\Socks\Client('127.0.0.1:1080');

$connector = new React\Socket\Connector([
    'tcp' => $proxy,
    'dns' => false
]);

$browser = new React\Http\Browser($connector);

See also the SOCKS proxy example.

You can also establish your outgoing connections through an SSH server by adding a dependency to clue/reactphp-ssh-proxy.

Secure Shell (SSH) is a secure network protocol that is most commonly used to access a login shell on a remote server. Its architecture allows it to use multiple secure channels over a single connection. Among others, this can also be used to create an "SSH tunnel", which is commonly used to tunnel HTTP(S) traffic through an intermediary ("proxy"), to conceal the origin address (anonymity) or to circumvent address blocking (geoblocking). This can be used to tunnel any TCP/IP-based protocol (HTTP, SMTP, IMAP etc.), allows you to access local services that are otherwise not accessible from the outside (database behind firewall) and as such can also be used for plain HTTP and TLS-encrypted HTTPS.

$proxy = new Clue\React\SshProxy\SshSocksConnector('alice@example.com');

$connector = new React\Socket\Connector([
    'tcp' => $proxy,
    'dns' => false
]);

$browser = new React\Http\Browser($connector);

See also the SSH proxy example.

By default, this library supports transport over plaintext TCP/IP and secure TLS connections for the http:// and https:// URL schemes respectively. This library also supports Unix domain sockets (UDS) when explicitly configured.

In order to use a UDS path, you have to explicitly configure the connector to override the destination URL so that the hostname given in the request URL will no longer be used to establish the connection:

$connector = new React\Socket\FixedUriConnector(
    'unix:///var/run/docker.sock',
    new React\Socket\UnixConnector()
);

$browser = new React\Http\Browser($connector);

$client->get('http://localhost/info')->then(function (Psr\Http\Message\ResponseInterface $response) {
    var_dump($response->getHeaders(), (string)$response->getBody());
}, function (Exception $e) {
    echo 'Error: ' . $e->getMessage() . PHP_EOL;
});

See also the Unix Domain Sockets (UDS) example.

The React\Http\HttpServer class is responsible for handling incoming connections and then processing each incoming HTTP request.

When a complete HTTP request has been received, it will invoke the given request handler function. This request handler function needs to be passed to the constructor and will be invoked with the respective request object and expects a response object in return:

$http = new React\Http\HttpServer(function (Psr\Http\Message\ServerRequestInterface $request) {
    return React\Http\Message\Response::plaintext(
        "Hello World!\n"
    );
});

Each incoming HTTP request message is always represented by the PSR-7 ServerRequestInterface, see also following request chapter for more details.

Each outgoing HTTP response message is always represented by the PSR-7 ResponseInterface, see also following response chapter for more details.

This class takes an optional LoopInterface|null $loop parameter that can be used to pass the event loop instance to use for this object. You can use a null value here in order to use the default loop. This value SHOULD NOT be given unless you're sure you want to explicitly use a given event loop instance.

In order to start listening for any incoming connections, the HttpServer needs to be attached to an instance of React\Socket\ServerInterface through the listen() method as described in the following chapter. In its most simple form, you can attach this to a React\Socket\SocketServer in order to start a plaintext HTTP server like this:

$http = new React\Http\HttpServer($handler);

$socket = new React\Socket\SocketServer('0.0.0.0:8080');
$http->listen($socket);

See also the listen() method and the hello world server example for more details.

By default, the HttpServer buffers and parses the complete incoming HTTP request in memory. It will invoke the given request handler function when the complete request headers and request body has been received. This means the request object passed to your request handler function will be fully compatible with PSR-7 (http-message). This provides sane defaults for 80% of the use cases and is the recommended way to use this library unless you're sure you know what you're doing.

On the other hand, buffering complete HTTP requests in memory until they can be processed by your request handler function means that this class has to employ a number of limits to avoid consuming too much memory. In order to take the more advanced configuration out your hand, it respects setting from your php.ini to apply its default settings. This is a list of PHP settings this class respects with their respective default values:

memory_limit 128M
post_max_size 8M // capped at 64K

enable_post_data_reading 1
max_input_nesting_level 64
max_input_vars 1000

file_uploads 1
upload_max_filesize 2M
max_file_uploads 20

In particular, the post_max_size setting limits how much memory a single HTTP request is allowed to consume while buffering its request body. This needs to be limited because the server can process a large number of requests concurrently, so the server may potentially consume a large amount of memory otherwise. To support higher concurrency by default, this value is capped at 64K. If you assign a higher value, it will only allow 64K by default. If a request exceeds this limit, its request body will be ignored and it will be processed like a request with no request body at all. See below for explicit configuration to override this setting.

By default, this class will try to avoid consuming more than half of your memory_limit for buffering multiple concurrent HTTP requests. As such, with the above default settings of 128M max, it will try to consume no more than 64M for buffering multiple concurrent HTTP requests. As a consequence, it will limit the concurrency to 1024 HTTP requests with the above defaults.

It is imperative that you assign reasonable values to your PHP ini settings. It is usually recommended to not support buffering incoming HTTP requests with a large HTTP request body (e.g. large file uploads). If you want to increase this buffer size, you will have to also increase the total memory limit to allow for more concurrent requests (set memory_limit 512M or more) or explicitly limit concurrency.

In order to override the above buffering defaults, you can configure the HttpServer explicitly. You can use the LimitConcurrentRequestsMiddleware and RequestBodyBufferMiddleware (see below) to explicitly configure the total number of requests that can be hand