Problem/Motivation
We have hit problems with the security release of 8.6.7 and the bug fix release of 8.6.8. See:
- #3026386: Drush fatal error after upgrading to 8.6.6, 8.5.9, or 7.62: PHP Fatal error: Uncaught TYPO3\PharStreamWrapper\Exception
- #3026443: \Drupal\Core\Security\PharExtensionInterceptor is incompatible with GeoIP and other libraries that use phar aliases or Phar::mapPhar()
- #3031128: Update from 8.6.7 to 8.6.8 warnings - Drupal\Core\Extension\Extension has no unserializer
The technical reasons for each one are different but our tests that extends UpdatePathTestBase have not caught these problems. We need a way in which we can address the security issues caused by phar and the PHP bug behind #2701829: Extension objects should not implement \Serializable without causing these update woes.
The update programme - #3029353: First launch of the beta testing program - is one tack but this issue proposes to add a new test type.
General specifications for the new testing type:
- It will not use the branch being tested as a starting codebase
- It will not use the drupal environment to run - it will be based on Drupal's unit test case or PHP's.
- It will install the minimum version of Drupal supported determined by our policy and the branch that is being tested \Drupal::VERSION
- It will install Drupal codebase using composer
- It will install Drupal somehow
- It will log into the site as user 1
- It will then be able to do upgrades via composer / drush / tarball?
- It will then test the resulting upgrade.
Proposed resolution
- Create a new test suite. We’ll call it ‘build.’
- HTTP requests must be able to run in isolation from the system running the tests.
- Tests must be able to run in parallel under run-tests.sh.
- Each build test case will encapsulate a workspace in the file system, and potentially an HTTP server to allow for making requests against the assembled site.
- Tests will be able to execute external commands on the command line, such as git or composer, in order to manipulate the workspace.
- Tests will be able to copy the patched version of Drupal to the workspace.
- Using the provided HTTP server, tests will be able to make a request, and assert against the results.
- We’ll use Mink as a framework for assertions against content.
Remaining tasks
Remove the proof-of-concept tests from the patch before committing. Create followups as necessary to re-add them.
Determine which sorts of tests are out of scope for this framework.
Ensure we’re not re-running a large number of tests. See #44.