Wrong DRUPAL_ROOT with non-standard code structure

Problem description and motivation

Drupal ships with a standard code structure where the web root contains the usual directories: core, modules, etc. Advanced site-builders might prefer to customise it:

1. Use symbolic links. This worked fine in D7 but is broken in D8.
2. #1672986: Option to have all php files outside of web root. - this issue does not solve that case but is a pre-requisite.

In both of these cases, the site is inoperable because Drupal guesses the application root incorrectly. This issue has been raised a bug not a feature because Drupal includes mechanisms that seem clearly designed to allow a non-standard code structure - but they are bugged.

This issue excludes changes to scripts/tests where there are more uncertainties how to move forward - see #3176691: Allow scripts/tests to work with non-standard code structure.

Proposed resolution

We need to place some simple, minimal restrictions on what can be customised. A few key files will need to remain in the web root without the use of symbolic links - this allows working out the application root by calling __DIR__ within these pages.

• Any file that serves a page: index.php, update.php, authorize.php, rebuild.php, install.php, statistics.php.
• Legacy include files included from these pages in core/include.

Based on these restrictions, we can ensure that there is no use of __DIR__ in any other files.

1. The key files must pass the $app_root parameter to methods on DrupalKernel.
2. All other files must call DrupalKernel->getAppRoot() rather than using __DIR__.

Related tasks

• #3176691: Allow scripts/tests to work with non-standard code structure
• Deprecate using $app_root = NULL.

Original summary

I usually link drupal inside my www directory like seen in the following ls output.
I think symlinking like that is not unusual, as it helps with version controlling of own projects.

core -> ../drupalcheckout/core
.htaccess
../drupalcheckout/index.php
../drupalcheckout/modules
profiles
robots.txt -> ../drupalcheckout/robots.txt
sites
themes -> ../drupalcheckout/themes

i've attached a script which will setup such an environment in the current directory.

With this setup BASE_ROOT in install.php will be set to the drupalcheckout directory.
Install then tries to find a settings.php in ../drupalcheckout/sites/default and not ./sites/default.

same Problem and fix should be considered for update.php and authorize.php

related discussions:
#1055856: Optimize settings.php discovery
#484554: Stop relying on Apache for determining the current path