Problem/Motivation
While investigating whether https://www.drupal.org/node/2642848 needs to be forward ported (it needs to) I found the call chain to whitelist rebuild be absurdly long and of course such a long call chain ought to contain a bug: system_path_delete does not pass the source so the whitelist is always rebuilt on alias delete. That's quite a performance hit.
Proposed resolution
Trival: pass the $path['source']