KernelTestBaseTNG™

Mission

1. Run all functional integration tests via PHPUnit.
2. Convert KernelTestBase into a PHPUnit test base class.
3. Add a second Kernel test suite, separate from the Unit test suite.

Why

• Drupal should not be in the business of developing a testing framework.

1. It will be possible that ~60% of Drupal's total test coverage is "rebased" onto PHPUnit, running entirely in memory.
2. Many kernel tests should be refactored into pure unit tests. Now they already operate in PHPUnit. Major milestone.
3. 2 of 3 test suites can be executed sans any Drupal [QA] infrastructure. → Proof.
4. PHPUnit is much more strict regarding unintended errors than Simpletest. It prevents more bugs and thus improves software quality. Also, it cleanly isolates tests (if necessary).

How

1. • Step 1: Add KernelTestBase itself as well as some basic integration test and fixing of related problems
   • Step 2: Convert the existing tests, which will fix various bits of code in multiple places and improve Drupal itself, see #2456477: [meta] Convert existing \Drupal\simpletest\KernelTestBase tests to KernelTestBaseNG

   After 4 weeks of permanent work on porting KernelTestBase + fixing PHPUnit upstream, mission accomplished:

   OK (904 tests, 10081 assertions)
   Time: 14.04 minutes, Memory: 269.25Mb

   This work revealed many bogus tests in HEAD + fixes most remaining SQLite Database driver issues. (See Child issues.)

2. Discuss whether to restructure PHPUnit test directories for discovery purposes:

   Test "type"    Location
   -------------- -------------------------------
   Unit           ./tests/src/Unit/*Test.php
   Kernel         ./tests/src/Kernel/*Test.php

   → #2260121: PHPUnit Tests namespace of modules is ambiguous with regular runtime namespace (+ Simpletest tests)

   Based on that decision,

   1. Either move all kernel test files + adjust their namespace and use statements
   2. Or adjust the use statements only.

3. Rebase, squash, and merge the final branch into Drupal core.

Technical overview

1. Drop-in replacement for Drupal\simpletest\KernelTestBase.
2. Operates entirely in memory; no artifacts. (vfsStream, sqlite::memory:)
3. Highly optimized for performance.
4. Leverages latest & greatest features of Drupal's dependencies. (PHPUnit, vfsStream, Psr\Log)
5. Forward-compatible with PHPUnit 4.3.

API additions/changes

1. New: Drupal\Tests\KernelTestBase
2. New BC layer: Drupal\Tests\AssertLegacyTrait— to be removed before 8.0.0.

Drupal\simpletest\KernelTestBase remains unchanged — to be removed before 8.0.0.

Upgrading

1. Use the new base class:
   -use Drupal\simpletest\KernelTestBase;
+use Drupal\Tests\KernelTestBase;
2. Ensure that all custom class properties are properly declared (and not overloaded).
3. Run the test with --verbose (enabled by default) and check the test output, especially for deprecated warnings.

In 99% of all cases, step (1) is sufficient.

Dependencies

1. Fix virtual filesystem: #2313989: Update vfsStream to 1.3.0
2. Fix test class property overloading: #2306539: FieldUnitTestBase::createFieldWithInstance() pollutes test class with arbitrary properties
3. Fix bogus tests: #2314123: Fix various tests
4. Fix bogus ScriptTest: #2261477: Remove broken Drupal\system\Tests\ScriptTest
5. Fix bogus theme calls + error reporting: #2301245: Entity system invokes non-existing theme hooks: "Theme hook $entity_type_id not found."
6. Fix bogus Entity Query Aggregate test: #2315269: Entity/Query/Sql/QueryAggregate: PDOException: General error: GROUP BY clause is required before HAVING

Related issues

Full (rebased/polished) history: sun/drupal#2 + sun/drupal#1

Beta phase evaluation