Drupal 8 updating issue - Drupal\Component\Plugin\Exception\PluginNotFoundException: The "<whatever>" plugin does not exist

Problem/Motivation

When running a config import immediately after updating the codebase (most of the time via git pull), the plugin discovery cache does not contain newly created / renamed plugins and so the configuration import will bomb with a plugin does not exist error message.

It's possible core does something to avoid this but drush doesn't and IMO it's not the responsibility of drush to do something. So if there's something in the config UI already for this then it needs to be moved into ConfigImporter.

Proposed resolution

I have two ideas:

1. Catch the PluginNotFoundExceptionand call the plugin.cache_clearer service. This has poorer performance.
2. Augment the PluginNotFoundException with a method to get/set the thrower object and call the clearCachedDefinitions() method on the thrower plugin manager if it is an instance of CachedDiscoveryInterface and then attempt to import again. There are six non-test calls in core but since two of them are in base classes, it's likely most of contrib will fall in line automatically and it's not like anything gets more broken if a contrib doesn't do a set -- it's just they don't benefit from this fix. One challenge with this approach is the possibility of multiple plugin types bombing in a single import and if we decide to act on that, how do we avoid an infinite retry if there's legit a plugin missing. Perhaps just put a sensible count limit on the retries -- say, no sane config will contain more than five failing plugin types, And there's always the drush cr before the drush cim if some unfortunate edge case engineer does stumble upon on a config object containing more than five...

Remaining tasks

Agree, code, test, commit, party!

User interface changes

None. (Except less error messages when importing config. A command line interface is a user interface too, isn't it?)

API changes

None.

Data model changes

None.

Release notes snippet

Temporary Workarounds

Cache Rebuild

Some have reported that a cache rebuild will resolve the issue:

$ drush cr

Delete the config entity and re-import [ CAUTION! ]

Use this with extreme caution, because it probably only works in very specific scenarios. This won't work in an automated fashion unless you do it in an update hook.

In this scenario, we have a block type config entity that we know is correct in the config in the repo, but is not correct in the database. It's pointing to a layout plugin that doesn't exist. So, we delete the config entity and re-import:

$ drush cdel core.entity_view_display.block_content.myblocktype.default
$ drush cim sync

This isn't a permanent solve, because when you export config at this point it doesn't see any diff between the database and code (as expected). But, at least the configuration will import successfully.