Problem/Motivation
- Container compilation is one of the slowest parts in cold cache startup. (now that YAML parsing is cached in FileCache)
- Container compilation is one of the most slowest parts during installation / uninstallation of modules
In the test suite this compilation happens again and again.
The major part in the compilation process taking long is Symfony's dependency checking and graph operations.
However:
- Caching the container across test configurations is very hard as there is many variations and modules are allowed to change everything.
Therefore we have:
- Poor cache granularity (hash is changing too much)
- BC problems as caching the container across tests could not take into account undeclared dependencies that modules have, so caching e.g. just on the module list is not feasible as there can be any outside parameters.
Proposed resolution
While I tried to cache the hash of the full container definition minus the parameters based on all parsed yml files (which worked great and had not too many variations), there is an even better way:
As seen in the motivation section, the slowest part are the dependency checks.
Those are defined here:
https://github.com/symfony/dependency-injection/blob/master/Compiler/Pas...
However, all this services do not depend on anything external and do also not depend on any parameters (except for the ResolveParameterPlaceHoldersPass, which can be removed as the used container supports run-time parameter resolving)
That means:
Whatever input those passes get in the form of the current ContainerBuilder(), the output will always be the same.
That however means this is cacheable based on the input and fortunately we already can transform a container builder into a container definition array using the PhpArrayDumper() (which is also really fast (8 ms)).
The idea is now to replace all those services with a single service _if_ and only if no one added a compiler pass before or after this - unless it is whitelisted using a special interface.
And doing that we can skip the most expensive compilation steps.
The new pass that replaces those passes, would then):
- Compile to a definition
- Store and remove the parameters
- Create a CID from the hash of the definition
- Put that in drupal.container_definition_cid
Cache Hit case:
- Retrieve the cached definition as the final output result
- Store a parameter: drupal.cached_container_definition (and re-add the original parameters to it).
- Return
Drupal would check the drupal.cached_container_definition parameter and use that instead of dumping the container builder.
Cache miss case:
- Just run the container things
- Create a dump like usual
- Check presence of drupal.container_definition_cid
- Store the result of the definition
Storage would just be in APCu or FileCache with a custom class that does only use the backends, but not the mtime part.
Remaining tasks
- Do it!