Configuration schema & required keys

Problem/Motivation

Quoting @bircher, maintainer of https://www.drupal.org/project/config_split, https://www.drupal.org/project/config_filter, etc., after I asked him What concrete things have gone wrong due to bugs in configuration, especially if they were caused by overriding/splitting/… — any of the advanced Configuration Management things?

what can go wrong, well sorting mostly... oh and to know if a key is required or not. So sometimes when a value is null or the array empty we can omit the key and sometimes not

👉 This issue aims to address the "sometimes […] we can omit the key and sometimes not" part. See #3364109: Configuration schema & required values: add test coverage for `nullable: true` validation support for the other part.

When talking about "keys" here, we're referring to the keys in a type: mapping. The root of every configuration object is a mapping — so the top-level keys you see in any MODULENAME.settings.yml file for example is a key.

For configuration management purposes it's critical to know which keys are required and which ones can be omitted. This can be entirely arbitrary: it depends on the PHP code that interacts with the stored configuration whether the key is required or optional.

In other words: today requiredness is implicit: it depends on the module's code. It's impossible to know based on the config schema. Only with trial & error can you figure this out … and even then, a single new code path could possibly cause a key that seemed optional to suddenly be required.

In general, most keys are implicitly required, with the exception of third_party_settings and subkeys of dependencies.

This leads to configuration management modules needing to add hardcoded hacks such as

              // @todo find a better way to know which elements are required.
              if ($type->getDataDefinition()->getDataType() === 'config_dependencies') {
                // Except for sub keys of dependencies.
                unset($diff[$key]);
              }

Steps to reproduce

Use modules like https://www.drupal.org/project/config_split and https://www.drupal.org/project/config_filter.

Proposed resolution

#3324150: Add validation constraints to config_entity.dependencies introduced the ValidKeys constraint. It can be configured in one of two ways:

ValidKeys: ['foo', 'bar']→ allows these two keys
ValidKeys: '<infer>'→ automatically infers the allowed keys based on the keys defined for this mapping in the config schema — this removes the need to sprinkle the entire config schema with statements like the above

But that only determines which keys are allowed, not which ones are required.

So let's update the existing ValidKeys constraint to not just allowing keys, but requiring keys.
⚠️ DO NOT CHANGE ANY BEHAVIOR FOR EXISTING CODE/CONFIG SCHEMAS! ⚠️ That is the mistake this issue was making until #77. All the points below apply ONLY to config schema type definitions that have
```
  constraints:
    FullyValidatable: ~
```
.
In this issue, ~~only a single type~~ four types get that (the one below, system.menu.* and shortcut.set.*, because these already have validation constraints for everything, plus editor.editor.* to demonstrate how much more helpful schema can be if we use it intentionally):
```
config_test.types.fully_validatable:
  type: config_test.types
  constraints:
    FullyValidatable: ~
```
The infrastructure to power ValidKeys detecting missing required keys was added in #3401883: Introduce Mapping::getValidKeys(), ::getRequiredKeys() etc.. This issue is only modifying the validation constraint.
Assuming the config object/config entity type's config schema type has opted in (IOW: is marked as fully validatable), then the ValidKeysConstraintValidator will respect the required & optional keys for every type: mappings it contains.
Add test coverage to prove that simple config is not affected until they opt in: see SchemaCheckTraitTest.
Add test coverage to prove that config entity types are not affected until they opt in: see \Drupal\KernelTests\Core\Config\ConfigEntityValidationTestBase::testRequiredPropertyKeysMissing()

Consequences

This has zero effect on contributed modules' config, because A) config schema type definitions must opt in, B) config validation does not yet run for them (not in tests, not in production, since #3361423: [meta] Make config schema checking something that can be ignored when testing contrib modules.
This has zero effect on core modules' config, because A) config schema type definitions must opt in.
The only way this can affect Drupal core/contrib in production: since #3364506: Add optional validation constraint support to ConfigFormBase, simple configuration forms that were updated to not use form-based validation, but config schema-based validation, will be affected, if the simple config they're editing (e.g. update.settings, system.site…) have the FullyValidatable constraint. This issue does not add it to any config schema types in Drupal core. Simple config forms already using do run validation constraints, but since setRequired() will never be called, the NotNull constraint validator will never run, and hence this does not cause any behavioral changes.