FATAL error when a view with a placed block is disabled or removed

from #1879774: Catch plugin exceptions for invalid views display plugins, reported by oadaeh:

I did a brand new install with D8 code downloaded less than an hour ago (which means the committed changes from this issue are in it) into a new database, and then performed the following steps:

Created a view (of users) with a block display
Placed that block in a region
Deleted the view
Displayed or refreshed a page that has the block on it
I get a WSOD. (When I did this last night, I was also getting an error message, but I'm not getting it now, so I can't post it).

I get this from Apache:
PHP Fatal error: Call to a member function access() on a non-object in /.../core/modules/views/lib/Drupal/views/Plugin/Block/ViewsBlock.php on line 56

If I delete the associated .yml file, everything is fine again.