Problem/Motivation
When loading large numbers of entities, especially during migrations and updates, it's quite easy to run out of PHP memory.
Migrate works around this by trying to clear the entity static cache, but this also results in a persistent cache clear.
Static caching is hard coded in storage classes which prevents swapping it out unless you change the whole class.
Proposed resolution
Add a 'static cache' service, which does not serialize at all, but which conforms to CacheBackendInterface.
Rework entity storage to rely on this service instead of a raw class property.
This will allow for two things:
- once this issue has landed, migrate will be able to reset the static cache via the service, therefore not affecting the persistent entity cache.
- in a follow-up, we'll be able to add a simple Least Recently Used implementation, allowing the static cache to have a fixed maximum number of items.
- beyond this, there is the possibility to go even further:
- have a cache backend limited by the actual size of the items in it (will require estimating memory usage of entities)
- add a cache backend, probably in contrib, using PHP weak references http://php.net/manual/en/intro.weakref.php which theoretically will allow cached entities to be removed by garbage collection.
Remaining tasks
User interface changes
API changes
Data model changes
Spinning this out of #1302378: Use multiple specialized entity controllers.
The base EntityController class currently handles keeping it's own 'static' cache internal to the class, or always fetching from the database all the time.
In Drupal 7, this means that http://drupal.org/project/entitycache needs to provide it's own controller class to provide persistent caching for entities.
We can make this a lot more flexible by just replacing the custom caching within the Entity controller class, with a configurable cache controller key instead of the current 'static cache' key. There's no need for entity-specific cache controllers, since any class implementing CacheBackendInterface will do fine.
To allow for 'static' caching, I've added a new ArrayBackend which simply stores cache items in a php array. Apart from expires which didn't seem worth working on until #730060: Replace CACHE_TEMPORARY, cache_clear_all() and minimum cache lifetime with cache tags support is in, this (untested) backend supports everything the database backend does, including tags etc., meaning it should be useful for unit testing eventually.
For no static caching at all, the NullBackend works fine.
To actually implement entitycache in D8 (whether core or contrib), you'd need a backend that proxies the MemoryBackend + whichever one is implemented for the cache_entity bin, since we're placing responsibility for both in a single class doing things this way. There's some inconsistencies with resetCache() etc. that would need to be sorted out for that.
Completely untested patch that will likely blow up, but just to show what it looks like.
One feature gets dropped here, which is fetching from cache when $conditions is passed to entity_load(). That's already a bit wonky as it is, and the $conditions parameter is already on its way out and documented as deprecated in 7.x per #1184272: Remove deprecated $conditions support from entity controller.
