Problem/Motivation
Long running processes - migrations and any big drush command can run into memory issues, especially with entity static caching.
Proposed resolution
#375494: Restrict the number of nodes held in the node_load() static cache added a memory cache - putting the 'static' cache into a service that can be swapped out, and cleared independently from entity persistent caching. This will allow drush's memory reclaim logic to work the same way that it did in Drupal 7 (when it cleared drupal_static).
However we can go a step or two further, and add an LRU implementation of the memory cache API, which holds a maximum number of items and cycles out once it goes over. This would allow drush or migrate to replace the service, and just load entities as much as it likes without requiring manual memory reclamation.
Remaining tasks
User interface changes
API changes
Data model changes
#375494: Restrict the number of nodes held in the node_load() static cache adds an LRU cache for the entity cache, I wanted to see if we could make a generic in-memory LRU cache that could maybe be used elsewhere.
Turns out to be reasonably simple to write. I haven't benchmarked this or anything yet, just wanted to get something that works first. It does not yet allow for adding multiple items to the cache at once, that may be a useful feature to add for the entity case.
Patch only adds the class and a unit test, but marking CNR for the bot and hopefully feedback.