Problem/Motivation
With #3259709: [PP-1] Create the database driver for MySQLi we'll have a database driver that is able to execute asynchronous queries.
I think we need potentially two APIs to handle that.
1. An interface indicating support for async queries, something like:
AsyncInterface{
function executeAsync();
}
As well as something to get the query when it comes back.
Ideally we can encapsulate at least MySQL and PostgreSQL's implementations so they'll work through one API. Core is only likely to need to execute one async query at a time rather than sending multiple, although you never know.
2. A way to conditionally execute a query async when in a Fiber and the driver supports it, this doesn't necessarily need to live in the database API as such.
Could look something like this:
class AsyncQueryHelper {
public function getAsyncConnectionIfInFiber($database, $target) {
// Given a database and target, get a dedicated connection for executing
// an asynchronous database query. This can maintain a stack so that existing
// connections where the query has already returned can be re-used. Avoids
// "commands out of sync; you can't run this command now".
if (\Fiber::getCurrent() === NULL) {
return Database::getConnection($database, $target);
}
return $this->getAsyncConnectionFromStack($database, $target);
}
public function executeAsyncSuspendFiber($query) {
// If the driver doesn't support async quer
if (!$query instanceof AsyncInterface) {
return $query->execute();
}
$fiber = \Fiber::getCurrent();
// If we're not within a fiber, we just have to return when the query comes back
if ($fiber === NULL) {
throw new \Exception('If you're not in a fiber and you have an async query interface, you didn't use getAsyncConnectionIfInFiber so shouldn't be using this API);
}
// Now we're cookin'.
$query->execute();
$fiber->suspend();
// @todo: infinite loop protection.
// The query holds a reference to the connection so can we poll the result from that?
while (!$query->hasResult()) {
$fiber->suspend();
}
return $query->getResult();
}
}
PHP's MySQL implementation, even with mysqli async support, can only handle one query at a time. If you execute two async queries without waiting for the first to come back (or an async query then a non-async query), you get Commands out of sync; you can't run this command now.
However, this can be worked around via a connection pool.
You have your main, non-async database connection.
When you want to make an async query, you request an async database connection via the skeleton API above. This would use a connection pool - if there's no async connection, create a new one, if there's an existing async connection, check if it's free and use it if it is, if there are existing, busy async connections, create new ones up to some kind of limit, if the limit gets hit, wait until one returns then use it.
Examples of this logic are in https://github.com/amphp/mysql/tree/3.x however we are going to want it to work with Drupal's database API, so probably can't use it directly, however it'll be a good reference point regardless.
The drawback of the connection pool approach is there's overhead in creating a new database connection, but if we only do that for listing queries and similar, then the absolute number of connections should be restricted, and we can artificially cap it too.