Problem/Motivation
Nightwatch testing is extremely unreliable and the cause of many random pipeine test failures.
The js world in general, and Drupal outside of core, is using either Cypress or Playwright for similar kinds of tests, and very few projects at all use Nightwatch these days (see links in issue).
For accessibility testing, we can also use Functional JavaScript tests with Axe enabled.
Steps to reproduce
Look at related issues that were or are problematic
- #3371963: Update Nightwatch to 3.x
- #3387927: Failing Nightwatch jobs result in a passed pipeline job
- #3389763: Impossible to run only Nightwatch tests in a given directory (f.e. for contrib modules)
- #3421202: Enable W3C-compliant webdriver testing
- ...
Proposed resolution
Add Playwright testing to Drupal core.
Convert nightwatch tests to playwright.
Remove the nightwatch tests and core pipeline.
Also convert some Nightwatch tests to Functional JavaScript + Axe where appropriate if that makes more sense than playwright.