We currently have a hook_block_access() for modules other than the one implementing the block to have a say in determining block access. This hook is invoked on the block entity, not on the block plugin, meaning if the block is rendered in a non-block-entity context (e.g., from Panels), the hook is not called.
Do we need another hook specific to the plugin? The plugin itself can implement the blockAccess() method, so the hook would only be for other modules to get involved. Note that modules can already implement hook_block_view_alter() or one of its more targeted variants and set #access on the render array. But a dedicated access hook could be seen as either better code organization or for optimizing the block render array from getting built needlessly.