Problem/Motivation
- Reduce disk space on local server. Automated tests greatly increase the size of a bare Drupal installation on disk. (As of
609912bced(8.8.x, July 15, 2019), they accounted for 43M of 94M (46%).) For many sites, especially simple ones, they provide no benefit to end users in production. The number and size of files can also be a problem for users of shared hosting (c.f. #2338671: Create a smaller Drupal core download, removing tests, for production sites.) - Test files can contribute to security vulnerabilities. See comments #21/#24/.
- Including test files increases download times and the load on drupal.org.
It's common to apply patches even on live sites. Patches often contain changes to test code and would by default fail if it is missing (see #33. Therefore as a dependency of this issue, we need an enhancement to the patching mechanism to allow applying patches when test code is missing - see this issue.
Obviously some sites specifically want to include the tests, so there must be a mechanism for that as simply as possible. This issue should try not to make this scenario worse or use more resources than in the past. In particular note that if we force sites to 'prefer source' to get the test files then this is much worse for the load on Drupal.org servers, see #22.
We need to carefully consider what counts as test code. One commenter (#37) noted that they would be happy to lose the specific core module tests but they would need the testing framework. We could consider including the framework even when excluding tests, depending on what portion of the code size it accounts for and how many sites we judge are in this category.
The primary target of this issue is Drupal Core. However we should ensure that whatever we do is generic so that any contrib projects with lots of tests can copy the same approach.
Proposed resolution 1
Use the approach documented in Use gitattributes to keep your tests out of other people’s production to exclude all tests/ directories from release packages. Since the Drupal.org packaging system already uses git-archive to create releases, the change would involve nothing more than the addition of a few lines to our .gitattributes file and would take effect immediately. Note: This would have no effect on Git clones, so neither the testbot nor core contributors would be affected.
In this scenario, anyone who wants tests uses prefer-source. We need to test and document exactly how to do this targetting just one composer project. A downside is that downloading source is considerably slower and will adversely affect Drupal.org servers. If this is estimated to be a major problem then a possible mitigation would be to produce a second composer project something like drupal/core-dev that replaces drupal/core and includes test files. However this becomes a non-trivial change to Drupal.org infrastructure and there might be further side-effects e.g. the extra storage for two sets of releases.
Alternative resolution 2
Remove test files after download using the vendor hardening Composer plugin: #3086277: Add a composer pluign to clean up Drupal test directories. An advantage is that this doesn't force anyone to prefer-source in order to get the tests.
However there are various disadvantages
- Does not solve goal 3 at all.
- It requires sites to "opt in" by requiring drupal/core-vendor-hardening, which drupal/recommended-project currently does not do. We could fix that, but it wouldn't help sites that are already installed. It's somewhat counter-intuitive to use a tool named "vendor hardening" to harden core and we would need to fix #3104291: drupal-core-vendor-hardening should not assume vendor directory to allow it.
- Any site that requires a single test would AFAICS need to disable vendor hardening globally.
- Vendor hardening currently makes
composer statuspretty much unusable because every single deleted test file is displayed and any 'real' changes are lost amongst that.
Remaining tasks
- Decide which resolution to use.
- Decide which directories to exclude - maybe keep the test infrastructure?
- Fix any issues that are dependencies.
- Carefully document and test the procedure for sites that do want to keep test code. Consider any possible adverse implications of this for load on Drupal.org infrastructure.
- Consider possible unintended consequences.
- Review/test (forthcoming) patch.
- Write a change notice.
- Decide when to commit/release the change.
User interface changes
If tests can be considered to belong to the user interface in any sense, the change is that they will disappear from tarballs.
Release notes snippet
TBD.