Problem/Motivation
Drupal core and a number of contributed modules expect to perform operations after the HTTP response is sent to the client, but prior to script termination. Examples are automated cron, storing JSON:API normalizations to the cache, and any operations that are deferred to the "destruction" of a service by implementing DestructableInterface.
In validating my implementation of the latter case (deferring sending push notifications) I noticed that client receipt of response contents were blocked until all code paths from index.php were executed.
I found #2579775: Missing Content-Length header blocks response until all registered shutdown functions execute and the Kernel terminate events run which is related to a similar symptom of blocking the response to the client side, but I think that is more sensitive to certain backends (e.g., FastCGI.) In any event, applying a re-rolled version of the PoC patch on that issue did not resolve the issue of blocked requests.
By way of level-setting, PHP can be set to have output buffering turned on or off by default through a PHP ini setting. Drupal does not explicitly start output buffering, e.g. by calling ob_start(), though such a thing was discussed back in Drupal 5 days but never went in to core.
Response::send(), which is called in Drupal's front matter, does flush the output buffer, if one is present. In my case (PHP run through Apache's mod_php - I'm not looking to debate your love of FPM here) I do not have output buffering on by default, so ob_end_flush() is never called.
Even if I add an explicit ob_start() before $response->send() in index.php, the output buffer is flushed but the PHP "backend" (I'm using the PHP docs terminology here) does not flush its response buffer to the client side. I need to further call flush() to do so.
All of this seems a little "too obvious" to me for this to be an oversight or purposeful omission on Drupal core's part, but this is:
- hard to test through an automated regime/I don't think we have test coverage for kernel termination/
DestructableInterface, and - These features are rather expert-level and the symptoms of this not working as advertised are easy to miss (your request just takes a little longer than it should.
Users of automated cron are probably most affected, though again, power users are less likely to use it.
Steps to reproduce
Perform a request with an implementation of DestructableInterface to do work after the response is sent to the client. Note that the response is blocked by work that is performed downstream of $kernel->terminate().
Proposed resolution
Call flush() in DrupalKernel::terminate(). Or perhaps do this in a stack middleware, though that seems a little heavy-handed. Alternatively, call it direclty in index.php?