private images do not work with PHP 5.4 builtin webserver

To reproduce:
Create a node with a private image field.
Start a server with drush rs --server=builtin (or with php -S)
View the node.
The image gives a 404.
Restart drush with CGI.
The image works fine (it has the same path as before).
Version 7.19 has the same behavior.
PHP version: 5.4.4