Problem/Motivation
A request that immediately follows a cache rebuild or a memcached flush results in a major performance issue, as Drupal tries to cache all available blocks at once. We ran into an issue on a couple of applications where memcached didn't have enough memory allocated to withstand the onslaught of memcache_set. This resulted in multiple calls for all of the blocks to be set en masse in memcached - because it attempts to fulfill its initial request ("write ~2300 blocks to cache_discovery, please"), then runs out of room (hits the memory ceiling), then tries to complete the job by taking a second swipe at it. You can see this at work in this xhprof run:
MemcachePool_set is being called over 4600 times here - which is approximately 2X the number of custom blocks in the database:
mysql> SELECT COUNT(*) FROM block_content;
+----------+
| COUNT(*) |
+----------+
| 2294 |
+----------+
1 row in set (0.00 sec)
Proposed resolution
Skip the entity api completely and go directly to the tables for the required data. The entirety of the required data for the current deriver is contained in block_content and block_content_field_revision. A single simple join is much faster and more performant for sites with heavy custom block usage.
Remaining tasks
Validate the approach in the patch and update as necessary.
User interface changes
None
API changes
None
Data model changes
None