Problem/Motivation

Migration::set() was provided as a work-around for the inability to properly modify migration plugin configuration at run time (#2681869: Provide clean way to merge configuration into migration plugins) - once that is committed, set() should be deprecated. The only core usages remaining after that commit are in tests, which should be changed to use the plugin manager to merge configuration.

Note that set() is documented with {@inheritdoc}, yet is not in MigrationInterface and thus is not properly documented. This fix should properly document it in the Migration class.

Proposed resolution

Add individual setters to MigrationInterface and implement them in Migration. Setters added are

• setDependencies(array $value)
• setRequirements(array $value)

In MigrateTestBase, add an optional $configuration parameter to getMigration() and executeMigration(), where $configuration is an array of values to merge into the migration definition.

In MigrateTestBase and the traits for file migrations, remove prepareMigration(). Instead, the traits add fileConfiguration(). Tests that run file migrations pass that to executeMigration().

Change the use of Migration::set() in

• MigrationPluginManager
• d6/MigrateUploadTest
• d6/FileMigrationTestTrait
• d7/FileMigrationSetupTrait
• ReferenceBase.php

Move test in MigrationTest to MigrationPluginConfigurationTest.php
Add doc bloc for Migration::set() and deprecate it.

Theses changes may require changes to migration tests in contrib.

Remaining tasks

If #2953111: Only migrate role permissions that exist on the destination is fixed before this issue, then update the test. See Comment #43.14 there.

Add follow-up issues for the following contrib modules (see #85 here):

• media_migration
• location_migration
• migrate_tools
• migrate_upgrade

User interface changes

API changes

Problem/Motivation

Migrate source plugins use checkRequirements() method and RequirementsException to notify migration runners that migration requirements are not met. Typically, there is a message like one of these:

• The module comment is not enabled in the source site
• No database connection configured for source plugin

If I run drush ms for migration that doesn't meet the requirements, it won't show up in the list and I won't see any additional messages. If I add -vvv flag, I'll see the following message:
[debug] Migration 'foo' is skipped as its source plugin has missed requirements: . [0.13 sec, 24.65 MB]

As we can see, requirements details are missing. 🤷‍♂️

Turns out, the requirements message is not retrieved from the actual exception message, but from the requirements array, which is a second argument for RequirementsException. See the following code:

foreach ($migrations as $migrationId => $migration) {
    try {
        $sourcePlugin = $migration->getSourcePlugin();
        if ($sourcePlugin instanceof RequirementsInterface) {
            $sourcePlugin->checkRequirements();
        }
    } catch (RequirementsException $exception) {
        $this->logger()->debug("Migration '{$migrationId}' is skipped as its source plugin has missed requirements: " . $exception->getRequirementsString());
        unset($migrations[$migrationId]);
    }
}

See: https://github.com/drush-ops/drush/blob/11.x/src/Drupal/Commands/core/Mi...

$exception->getRequirementsString() is:

public function getRequirementsString() {
    $output = '';
    foreach ($this->requirements as $requirement_type => $requirements) {
      if (!is_array($requirements)) {
        $requirements = [$requirements];
      }

      foreach ($requirements as $value) {
        $output .= "$requirement_type: $value. ";
      }
    }
    return trim($output);
  }

This means, that the following messages are never visible on drush execution:

• d7/CommentEntityTranslation.php#L117
• d7/CommentEntityTranslation.php#L121
• SqlBase.php#L209
• d6/VariableTranslation.php#L126
• ProfileField.php#L112

I noticed this while working on #3232214: Table source plugin - validate the sql table before executing the query.

I was curious if this is something Drush-related, but using migrate status from Migrate Tools module is even worse - there are no visible log messages at all.

Steps to reproduce

drush ms your_migration_with_missing_requirements -vvv

Proposed resolution

It seems using ::getRequirementsString() instead of ::getMessage() for RequirementsException is by design. We already have many usages that follows this pattern and provides array requirements as a second argument.

We should probably adapt the cases mentioned above to make sure the requirements are clear on migrate status command.

Remaining tasks

Propose a merge request, review and commit.

Problem/Motivation

IE browser add element in webform tab switch not working

Steps to reproduce

After enable webform module, add element button text color has changed to black, and in IE browser click 'add element' button, the tab switch component is not working

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Problem/Motivation

Pretty much all of the other issues that are asking for OO hook implementations all run into the same problem: we have an unsettlingly large number of places throughout core that build a function name and then call it with some arguments, rather than calling ModuleHandler->invoke() or ModuleHandler->invokeAll().

This actively prevents implementation of a contrib module that, as an example, extends ModuleHandler and emits events when hooks are invoked. Pretty much anywhere where ModuleHandlerInterface->getImplementations() is used outside of ModuleHandler, there's likely a function name being constructed, and in the vast majority of these cases, it's unnecessary.

Proposed resolution

• Introduce two new hook invocation methods that take an additional callable (often, but not limited to, closures) that takes care of calling each individual hook implementation. Code currently calling ModuleHandlerInterface::getImplementations() to invoke hooks themselves can use custom callables to customize their hook invocations. This is useful to pass on extra parameters or process return values.
• Mark ModuleHandlerInterface::getImplementations() deprecated, to encourage developers to use the (new) ::invoke*() methods instead.

Remaining tasks

Convert all remaining calls to ModuleHandlerInterface::getImplementations() that do not cause test failures. If that's because of a lack of coverage, decide what to do about that.

Review, evaluate DX.

User interface changes

None.

API changes

None. Existing APIs are merely extended. ModuleHandlerInterface is extended, which is not considered a break (@catch in #29)

Invoking modules must now replace their code which builds functions with calls to Module Handler. The call simply includes the hook name, and a closure which in turn calls the original hook via a closure. Modules that originally dealt with the return values of the invoked hooks, or relied on mutation of hook arguments, should pass along state via a use definition in the Module Handler closure.

Data model changes

None.

Repeatable: always

Steps to repeat:

1. Install fresh Drupal 7 site
2. Enable Forum.module
3. Disable Forum.module
4. Go to Reports->Field list. These notices are listed:
   

   • Notice: Undefined index: comment_node_forum in _field_ui_bundle_admin_path() (line 320 of /home/medieva/public_html/skplus/usdkexpats/modules/field_ui/field_ui.module).
   • Notice: Undefined index: comment_node_forum in field_ui_fields_list() (line 35 of /home/medieva/public_html/skplus/usdkexpats/modules/field_ui/field_ui.admin.inc).
   • Notice: Undefined index: forum in _field_ui_bundle_admin_path() (line 320 of /home/medieva/public_html/skplus/usdkexpats/modules/field_ui/field_ui.module).
   • Notice: Undefined index: forum in field_ui_fields_list() (line 35 of /home/medieva/public_html/skplus/usdkexpats/modules/field_ui/field_ui.admin.inc).
   • Notice: Undefined index: forum in _field_ui_bundle_admin_path() (line 320 of /home/medieva/public_html/skplus/usdkexpats/modules/field_ui/field_ui.module).
   • Notice: Undefined index: forum in field_ui_fields_list() (line 35 of /home/medieva/public_html/skplus/usdkexpats/modules/field_ui/field_ui.admin.inc).

5. Uninstall Forum.module
6. Go to Reports->Field list. Notices are still listed, although now only 4 instead of 6:
   

   • Notice: Undefined index: comment_node_forum in _field_ui_bundle_admin_path() (line 320 of /home/medieva/public_html/skplus/usdkexpats/modules/field_ui/field_ui.module).
   • Notice: Undefined index: comment_node_forum in field_ui_fields_list() (line 35 of /home/medieva/public_html/skplus/usdkexpats/modules/field_ui/field_ui.admin.inc).
   • Notice: Undefined index: forum in _field_ui_bundle_admin_path() (line 320 of /home/medieva/public_html/skplus/usdkexpats/modules/field_ui/field_ui.module).
   • Notice: Undefined index: forum in field_ui_fields_list() (line 35 of /home/medieva/public_html/skplus/usdkexpats/modules/field_ui/field_ui.admin.inc).

7. Enable Forum.module
8. Go to Reports->Field list. All notices are gone.

Environment:
OS: linux
Database: MySQL version 5.1.56-log
Web Server: Apache version 2.2.21
PHP version 5.2.17
Drupal 7.9

Manual Workarounds

Uninstall disabled modules

Ensure that all disabled modules are fully uninstalled. This may resolve the error.

Enable and manually delete the fields, then uninstall

1. Enable module (e.g. Forum, Blog, etc .... ) that defined the content type that has problematic field
2. Delete any fields that are attached to the type, including any comment fields
3. Disable and uninstall the module

Note that if these fields had a lot of data, you may need to run cron multiple times to delete these first. Core only deletes 10 rows at a time, so this process can take multiple cron runs. Once the database field tables are empty, you can disable and uninstall the module.

i.e. 1000 items would take 100 cron runs to clear, or just over 4 days if cron is only running once an hour.

Delete invalid config from the database directly. (advanced)

Ensure you have a backup first!

DELETE FROM {field_config_instance} WHERE entity_type = 'ENTITY' AND bundle = 'BUNDLE' AND field_name = 'FIELDNAME'

1. {field_config_instance} means the database table with the correct table prefix.
2. ENTITY means the entity type.
   For example, node is the standard entity type for content. Taxonomy fields use taxonomy_term, Users user, etc.
3. BUNDLE means the bundle type.
   For example page and article are the two bundles that are common on the entity type node. Terms have the vocabularies machine name as their bundle names, User is simply user, etc.
4. FIELDNAME is the missing field name triggering the error.

Use all 3 WHERE conditions to avoid deleting the wrong rows.

i.e. DELETE FROM field_config_instance WHERE entity_type = 'node' would delete each and every field on every content type and likely kill your site instantly.!!!

Problem/Motivation

I am getting the following error when used Better Exposed Filter and Grouped Filter with checkbox. Warning: Invalid argument supplied for foreach() in Drupal\taxonomy\Plugin\views\filter\TaxonomyIndexTid->validateExposed() (line 365 of core/modules/taxonomy/src/Plugin/views/filter/TaxonomyIndexTid.php)

Steps to reproduce

Create a view and add an exposed filter (Term field or EntityReference Field) with Grouped Filter option and set Better Exposed Filter for the same field as Single On/Off Checkbox as screenshot attached. Now you will see the checkbox option on the page and when you select the checkbox and submit the form, will cause the error.

We have:

process:
  plugin: skip_on_empty
  method: row

process:
  plugin: skip_on_empty
  method: process

process:
  plugin: skip_row_if_not_set

We need the 4th combination: skipping a process if a value is not set.

For consistency and to avoid BC pain/suffering, this should probably be a whole new process plugin, "skip_if_not_set" that supports method: (row|process) and then deprecate skip_row_if_not_set.

Problem/Motivation

In #2976098: MigrateExecutable should add details for the migration & destination property to exceptions that cause a row failure, the migration ID and destination property was added to migration failure messages.

For a process pipeline with several plugins, that can still leave some detective work finding the source of the problem.

It would be useful to add the process plugin ID, and also the index number, since the same process plugin can occur several times in a single pipeline.

Steps to reproduce

Proposed resolution

Move the catch in import():

        catch (MigrateException $e) {
          $this->getIdMap()->saveIdMapping($row, [], $e->getStatus());
          $msg = sprintf("%s:%s: %s", $this->migration->getPluginId(), $destination_property_name, $e->getMessage());
          $this->saveMessage($msg, $e->getLevel());
          $save = FALSE;
        }

to processPipeline(), which has the plugin name, and add an $index to the foreach loop.

This should then re-throw the exception with the process plugin details prefixed to the message.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Problem/Motivation

The FileUrlGeneratorInterface::generate() function accepts a string $uri and converts it into a URL object. The $uri can be in multiple formats, one of which being a stream wrapper. e.g. "public://path/to/my/file.jpg".

However, if the stream wrapper is for an external service (i.e. hosted outside of the current Drupal instance), then this doesn't work. The base path of the current site is prepended, and you end up with a url object that converts back to string like this: "http://www.mydrupalsite.com/https%3A//www.myexternalhost.com/path/to/my/file.jpg"

You can see here where base: is prepended to the uri string https://git.drupalcode.org/project/drupal/-/blob/3e4b17a7c9a926021202140...

    elseif ($wrapper = $this->streamWrapperManager->getViaUri($uri)) {
      // Attempt to return an external URL using the appropriate wrapper.
      return Url::fromUri('base:' . $this->transformRelative(urldecode($wrapper->getExternalUrl()), FALSE));
    }

Also see relevant Slack conversation about this issue: https://drupal.slack.com/archives/C1BMUQ9U6/p1639524487401900

Steps to reproduce

As there are no externally hosted stream wrappers as part of core, this can only be reproduced with a module.

We encountered the issue using flysystem_s3. With this module, files are stored using uri's like: s3://path/to/image.jpg and these are then converted into full S3 urls. E.g. https://s3-eu-west-2.amazonaws.com/my-s3-bucket-name/path/to/image.jpg.

Whilst there is a FileUrlGeneratorInterface::generateAbsoluteString(), which does work correctly with the use case above. This is a slightly different function, as it returns an absolute uri string, whereas FileUrlGeneratorInterface::generate() returns a url object.
Our issue is that there is code in core that calls FileUrlGeneratorInterface::generate(), such as in template_preprocess_file_link(). This was working with our remote stream wrappers fine before the FileUrlGenerator service was introduced.

This may also be an issue for the s3fs module (although this has not been confirmed yet).

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Problem/Motivation

If a file url has query parameters, or a hash, added to it. The file-link component strips these out.

Drupal would not normally use query parameters on a file url. But in our case they are being added in by a stream wrapper, from the flysystem_s3 module (we are using them to pass through a url signature, to allow the user access to the file).

Query parameters could also be added in by hook_file_url_alter(), these would again be stripped out.
The core public file stream wrapper url encodes any query parameters added by hook_file_url_alter(). So these the are retained.

The issue is caused by the new file handling in template_preprocess_file_link(), which used to create the Url object itself, but now uses the new file_url_generator service. Since this changed was introduced, the $options is being overwritten by $url->setOptions($options)

Steps to reproduce

Proposed resolution

Use
$url->mergeOptions($options)
Instead of
$url->setOptions($options)

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

In a multilingual site, if you create a view with aggregation showing "fields", plural form of an aggregated field cannot be translated.

Steps to reproduce:

1) Standard installation of Drupal
2) Add a non-English language
3) Activate "content translation" and "config translation" core modules.
4) In "Article" content type edit image field to allow more than 1 value.
5) Create an article with more than an image and another one with 1 image (or just use Devel Generate).
6) Create a view type fields, with a display "page". Allow aggregation and show node title and the COUNT of images of the node. Use "format plural" option and write "1 photo" for singular form and "@count photos" for plural form.

If you try to translate these texts, they're not translatable by translation interface nor views translation interface.

Problem/Motivation

Since Drupal 8 we've done some work to remove 'magic array keys'. But we still have render arrays, which every time you use them cause you to need to look-up the allowed keys. This is really bad DX. It's not clear what the allowed keys are, or where to find them. For backend developers who don't have years of experience with the nuts and bolts of Drupal's rendering system, this is more of a learning cliff than a learning curve, and it's very discouraging.

Proposed resolution

Provide builder classes for each element type to make defining render arrays discoverable from IDEs.

before

$variables['table'] = array(
    '#type' => 'table',
    '#header' => $headers,
    '#rows' => $rows,
    '#attributes' => array(
      'id' => $table_id,
    ),
    '#tabledrag' => array(
      array(
        'action' => 'order',
        'relationship' => 'sibling',
        'group' => $weight_class,
      ),
    ),
  );

after

$variables['table'] = TableElement::create()
    ->setHeader($headers)
    ->setRows($rows)
    ->setAttributes(array(
      'id' => $table_id,
    ))
    ->setTableDrag(array(
      array(
        'action' => 'order',
        'relationship' => 'sibling',
        'group' => $weight_class,
      ),
    ))->toRenderArray();

This sort of interface is a lot clearer, and borrows heavily from both Url helper and FieldDefinition stuff.

In this issue, we should only introduce a base class for these builders, plus one or two concrete implementations that we use in core, to prove they work. Then, in follow-up child issues, we can introduce more builders for the various core elements, piecemeal.

Remaining tasks

Refactor the patch in #106 to modernize it -- i.e., adding type hints -- and remove most of the builders it implements (punt to follow-ups). Keep one or two useful builders, and change core to use them where possible. Add unit or kernel test coverage. Then review and commit.

User interface changes

None

API changes

Creates a new optional way to create render elements.

Problem/Motivation

When Content Moderation module is enabled, different revisions of a node each have their own moderation state, e.g.:

- r5: draft
- r4: draft
- r3: published - current revision
- r2: published
- r1: draft

These revisions are listed on the Revisions tab on a node, at /node/ID/revisions, but that table does not show the state.

Currently, reverting a past revisions preserves its state, which means that:

- if you revert a draft, the new copy revision that is made is also a draft: e.g. new r6 is a copy of r1, and r3 is still the current revision
- if you revert a published revision, the new copy revision is also published: e.g. new r6 is a copy of r2, and is the *new current revision*.

Therefore, it's important to know the state of what you are reverting, as it changes what is live public content! #2906568: Provide users with the option to select an appropriate moderation state when reverting to a previous revision seeks to allow selecting the moderation state of the new copy in a revert operation, but even with this change, it would still be useful to see the state of the different revisions.

The work for creating a unified entity revisions UI (#2350939: Implement a generic revision UI) will break any patch here, but I think work on the node UI could be adapted to the new generic revisions controller class in that issue, when it eventually lands.

Steps to reproduce

Proposed resolution

Alter the node route to change the controller.
New controller inherits from the existing one, and hands over to the parent class for a bundle that's not under moderation.

Once the generic UI lands, alter routes for all moderated entity types based on their 'version-history' link template.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Problem/Motivation

In #3106216: Remove unused variables from core we introduced the sniff DrupalPractice.CodeAnalysis.VariableAnalysis to detect unused variables in Drupal Core.
To narrow the scope the decision was made to exclude all the test-classes.

This issue is created to make that sniff run without errors on all test-classes.

Steps to reproduce

Proposed resolution

- Remove the line <exclude-pattern>*/tests/*</exclude-pattern> from sniff DrupalPractice.CodeAnalysis.VariableAnalysis in core/phpcs.xml.dist
- Fix all and any errors

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

I often get this during translation import of custom .po files:

 [notice] Translations imported: 0 added, 404 updated, 0 removed.
 [warning] No configuration objects have been updated.

I get no idea if I did something wrong or if that warning could/should be removed ?

I've been very confused with the addition of canonical links in Drupal 7. The default behaviour seems to be to assign the actual node url of the node being displayed on the front page as the canonical uri. But that must be incorrect, since going to that uri will only redirect to the front page. So I've made a few changes to two files:

common.inc, line 2143, before:

  if ($path == '<front>') {
    $path = '';
  }

changed to:

  if ($path == '<front>' || $path == variable_get('site_frontpage', 'node')) {
    $path = '';
  }

Now the front page has "/" as canonical url. Also, other tags like < meta about ..> get updated.

Once I got going, I made two other changes as well in node.module, line 2592:

  $uri['options']['absolute'] = TRUE;
  // Set the node path as the canonical URL to prevent duplicate content.
  $canonical_href = url($uri['path'], $uri['options']);
  drupal_add_html_head_link(array('rel' => 'canonical', 'href' => $canonical_href), TRUE);
  // Set the non-aliased path as a default shortlink.
  $shortlink_href = url($uri['path'], array_merge($uri['options'], array('alias' => TRUE)));
  if(strlen($shortlink_href) < strlen($canonical_href)) {
    drupal_add_html_head_link(array('rel' => 'shortlink', 'href' => $shortlink_href), TRUE);
  }

This does two things:
1. Sets canonical links as absolute urls, which seems better if site can be accessed from different urls. Should perhaps be an option somewhere?
2. Only adds the shortlink IF its url is actually shorter than the aliased path. It seems weird to say that the shortlink of "/" is "/node/12345", or that another shortlink of "/abc" is "/node/34567".

What do you think about these changes? I think it's crucial SEO-wise to get it right. I'm not an SEO expert but I've asked around a bit and this seems to be along the right tracks. I hope that the Drupal core can be updated some way to be more optimized for SEO and web standards.

Problem/Motivation

Sometimes a render array includes the value of a state variables, it would be good to be able to provide a cache tag which tells Drupal to invalidate caching of said render array when that state variable changes.

Steps to reproduce

Please see [the question "What cache tags can be triggered by a change in state variables?" on Drupal Answers](https://drupal.stackexchange.com/questions/309121/what-cache-tags-can-be...).

Proposed resolution

Invalidate cache tags when state variables change.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Problem/Motivation

The documentation in Form API's EntityAutocomplete class, of what properties are allowed in different configuration arrays, is difficult to find. Some properties are described in method docblocks; some on the class docblock; other docblocks are incomplete.

Proposed resolution

Rewrite inline documentation for EntityAutocomplete and DefaultSelection to clarify what property keys are permitted in config arrays:

* Move explanation of common config properties to class-level docblocks.
* Reference these in method level docblocks, to avoid duplication.
* Expand descriptions of properties where appropriate.
* Two classes to reference each other with @see, to avoid duplication.

Remaining tasks

Approve this issue summary change.

User interface changes

None.

API changes

None.

Data model changes

None.

Problem/Motivation

After a full cache clear, Drupal has to build up various caches before it can serve a request. Router, menu links, various plugin registries, twig templates, CSS assets (#1014086: Race conditions, stampedes and cold cache performance issues with css/js aggregation) etc.

This both takes a long time, and massively increases the memory usage of a request.

Steps to reproduce

drush cr, load a heavy-ish page, watch that spinny wheel / check mem_get_usage()

Proposed resolution

PHP 8.1 adds Fibers https://php.watch/versions/8.1/fibers / https://wiki.php.net/rfc/fibers

This allows very simple concurrency, and it might allow us to offload some work.

A good use case would be router rebuild. If we create a Fiber for rebuilding the router, we can call that, let it finish, and continue with the main request, but offload all the memory usage of parsing routing YAML etc. to the Fiber.

There might later be the opportunity to move some processing async - for example triggering an async router rebuild from ::setRebuildNeeded() or a more general cache pre-warming step after drupal_flush_caches().

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Problem/Motivation

#2796173: Add experimental Field Layout module to allow entity view/form modes to switch between layouts adds an experimental module that extends and alters the EntityDisplay entity classes and forms.

It provides no additional paradigms or functionality other than enhancing the existing UI.

Proposed resolution

Once the module is approaching a stable release, merge it directly into those core classes instead of having an optional module, even one in the Standard profile.

Remaining tasks

N/A

User interface changes

N/A

API changes

Added EntityDisplayInterface::getDefaultRegion()
Added EntityViewDisplayInterface::getFieldFromBuild()
Added EntityViewDisplayInterface::setFieldOnBuild()

Data model changes

entity_view_display and enitty_form_display both get a layout_id (string) and layout_settings (array)