Problem/Motivation
Drupal is using its own code to discover tests to be executed, which along time lead to discrepancies in the tests executed betwen PHPUnit CLI and run-tests.sh.
PHPUnit 10 (Feb 2023) introduced an API to discover the test suites, that can be used instead. Also, PHPUnit 10 introduced attributes to replace the annotations that were previously used to indicate test configuration to PHPUnit. PHPUnit 11 (Feb 2024) is triggering deprecations if it detects tests that use annotations. PHPUnit 12 (Feb 2025) no longer supports annotations.
Drupal's TestDiscovery class, that builds the tests to be executed by run-tests.sh, mandates tests to use @group annotations. It fails if only #[Group(...)] attributes are used.
Proposed resolution
- Deprecate
TestDiscovery::getTestClasses()and related methods. - Introduce a like-for-like replacement that uses PHPUnit's API for the discovery, and enriches the information returned by the API to provide same info as
TestDiscovery::getTestInfo(). - Make the new code ready for PHpUnit's 10+ attributes instead of annotations.
- In the process, fix some bugs of
TestDiscovery::getTestClasses()that were identified:- for each test class, it only reports groups present in the class-level annotations: this means that method level annotations do not qualify test classes for a group, potentially missing the execution of some tests. PHPUnit is more accurate in bubbling up method-level annotations/attributes to the class
- it does not check duplication of @group annotations that specify the same group. PHPUnit make them unique.
- when filtering by test suite, it reports some test groups with an empty array, since in any case it iterates all test directories and files. PHPUnit only scans the directories indicated in phpunit.xml for each test suite, providing more accurate results.
- Very interesting side effect of this - since PHPUnit discovery actually executes the dataproviders, and includes in the discovery list as many test cases as the number of datasets provided for a test method, we can now be precise in the number of test cases run for each test class, and use that information in run-tests.sh instead of the current approximate count.
Remaining tasks
User interface changes
nope
API changes
nope
Data model changes
nope
Release notes snippet
Drupal tests now can use PHPUnit's 10+ attributes instead of the legacy annotations. New tests MUST use attributes.