Quantcast
Channel: Issues for Drupal core
Viewing all articles
Browse latest Browse all 293211

Try to optimize test ordering when run-tests.sh is used with a mixture of test types

$
0
0

Problem/Motivation

Core gitlab runs use run-tests for concurrency, on only one test suite at a time (unit, kernel, functional, functional js).

In contrib gitlab runs, concurrent test runs run all groups at once, and this appears to go in the unit, kernel, functional, functional js order.

See the order of initial output on:
https://git.drupalcode.org/project/paragraphs/-/jobs/629259

i.e. If we run functional javascript -> functional -> kernel -> unit instead, we should be able to get overall faster test runs.

This is because functional and functional js tests take dozens or hundreds of times as long to run as unit tests, so if concurrency is maxed out, several unit tests can be finished in the space of one functional test. If a functional test is already running and there are spare threads for unit tests, this time is 'free', but if the unit tests all run first, then you end up with long running tests occupying just a single thread at the end of the job, with other CPUs idle.

Steps to reproduce

Proposed resolution

When multiple tests are run by run-tests.sh, reverse the order of the test groups so that slowest tests run first.

We will need to decide whether @group #slow should be used to determine the order only within a test group, or the order of tests within the whole run - it might make sense for it to be within the whole run?
If we only change the default order of test discovery, @group #slow will continue to work to modify the order of tests regardless of type, which seems fine and makes this a very small change.

Remaining tasks

This will have no effect on core test runs, so ideally we'd see how much difference it makes with a contrib module that has a decent test suite like paragraphs.

User interface changes

API changes

Data model changes

Release notes snippet


Viewing all articles
Browse latest Browse all 293211

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>