Problem/Motivation

There is no need to use t() in tests, unless we're testing translations, however in core we do not follow this consistently, which does not set a good example for new contributions.

In #3133726: [meta] Remove usage of t() in tests not testing translation we identified there are severals of calls to t() in calls to assertEqual() and assertEquals() and that removing all these in one go seems to be a suitable way of attacking this problem.

Proposed resolution

Identify and remove all calls to t() wrapped in calls to assertEqual() and assertEquals(), except those used by translation-related code (if any).

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Problem/Motivation

AssertLegacyTrait::assertEscaped() is deprecated in drupal:8.2.0 and is removed from drupal:10.0.0. Use $this->assertSession()->assertEscaped() instead. See https://www.drupal.org/node/3129738

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Problem/Motivation

There is no need to use t() in tests, unless we're testing translations, however in core we do not follow this consistently, which does not set a good example for new contributions.

In #3133726: [meta] Remove usage of t() in tests not testing translation we identified there are severals of calls to t() in calls to linkExists() and linkNotExists() and that removing all these in one go seems to be a suitable way of attacking this problem.

Proposed resolution

Identify and remove all calls to t() wrapped in calls to linkExists() and linkNotExists(), except those used by translation-related code (if any).

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

When creating the image style and hover on the anchor instead of hovering over the particular box it is showing a hover effect on the whole row.
Steps to Reproduce

1. Create new image style
2. Add Scale and Crop as a style

For example Hovering on A11 Brings hover effect on A12 and A13 also

Say, you fill up some form fields. Now, if you accidentally leave the page by clicking the back button (or if you have an apple magic mouse and you end up doing the wrong gesture), you lose all the form changes without any warning!

One of the targets of Drupal is to be a editor-friendly CMS. Thus, I'd strongly recommend having something like the confirm leave module built into the core.

Problem/Motivation

See #2746541-400: Migrate D6 and D7 node revision translations to D8
#2746541-403: Migrate D6 and D7 node revision translations to D8
#2746541-405: Migrate D6 and D7 node revision translations to D8

It is possible that this happens with draft revisions, for example when workbench moderation is used.

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Unlike nodes, block entities do not have theme suggestions for view modes on custom block types. This means that if you need to create a template for a block in a specific display mode, then you need to write custom code to pick it up.

This patch provides additional theme suggestions for block view modes, and is based on the same code in the node module.

Problem/Motivation

In preparation for adding JSON:API to Drupal core, we released the 2.x branch. In it, we tried to clean up many of JSON:API's rough edges. One of which was that while we represented entities as "resource objects" that don't directly match the structure of the underlying entity, when you filtered collections, one would have to filter by entities using their exact structure, even bits that were missing from the resource object representation. We published a change record to this effect.

That change introduced and unintended consequence, anyone filtering by the target_id property of an entity reference item would have their clients broken. This target_id property is omitted from resource identifiers because it was believed that it was unnecessary because it sort of duplicated the id member.

This was a wrong assumption, it may be necessary to filter a the target ID when filtering for an entity referencing a config entity. This might be the case if your client is filtering users by a role machine name, for example.

Proposed resolution

Add a drupal_internal__target_id property to resource identifiers and make it possible to filter by this value (this ensures that filters still closely match JSON:API's output structure (the purpose of the CR that broke this behavior))

User interface changes

None.

API changes

A new property will be added to JSON:API's output. This is not a breaking change.

Data model changes

None.

Release notes snippet

TBD

Original report

:

I am trying to upgrade my site from the 1.x branch to 2.x but am running in to some confusion and a bit of a performance problem.

When filtering by an entity reference field, I used to be able to do something as simple as /api/entity_type?filter[ref_field]=123, where 123 is the ID of the entity. This is beneficial to using the UUID because it does not require a JOIN on the referenced entity table. The query would be something like:

SELECT base_table.id AS id, base_table.id AS base_table_id FROM {entity_type} WHERE ref_field = 123

Now, when trying to perform the same API request, I get this error: "Invalid nested filtering. The field `ref_field`, given in the path `ref_field` is incomplete, it must end with one of the following specifiers: `id`."

If I change the filter to ?filter[ref_field.id]=123, the query becomes overly complex and performance degrades seriously (we have over 2M entities):

SELECT base_table.id AS id, base_table.id AS base_table_id FROM @entity_type base_table INNER JOIN @entity_type entity_type ON entity_type.id = base_table.id LEFT OUTER JOIN @ref_field ref_field ON ref_field.id = entity_type.ref_field INNER JOIN @ref_field ref_field_2 ON ref_field_2.id = ref_field.id WHERE ref_field_2.uuid LIKE :db_condition_placeholder_0

I am curious: 1) Why was this change added? and 2) Is there anyway I can force the former, more simple approach?

There are a few places in the jsonapi tests where there is a reference to issue #2940339. It states that the testRelated test should be re-enabled when that issue is fixed. This didn't happen.

I tried just removing the skips, but it seems there is still some work to be done to make sure the code in testRelated can handle NULL. Currently the tests fails as follows:

The 'data' member was not as expected.
Failed asserting that null is identical to Array &0 ().
 /home/localcopy/drupal/core/modules/jsonapi/tests/src/Functional/ResourceTestBase.php:789
 /home/localcopy/drupal/core/modules/jsonapi/tests/src/Functional/ResourceTestBase.php:715
 /home/localcopy/drupal/core/modules/jsonapi/tests/src/Functional/ResourceTestBase.php:1377
 /home/localcopy/drupal/core/modules/jsonapi/tests/src/Functional/ResourceTestBase.php:1296
 /home/localcopy/drupal/core/modules/jsonapi/tests/src/Functional/CommentTest.php:396

I added a patch with the tests enabled inline to make debugging easier.

From the JSON:API spec: https://jsonapi.org/format/#document-meta

Where specified, a meta member can be used to include non-standard meta-information. The value of each meta member MUST be an object (a “meta object”).

Any members MAY be specified within meta objects.

Handling of metadata in the JSON:API normalizers already exists.

RelationshipNormalizer

    return CacheableNormalization::aggregate([
      'data' => $this->serializer->normalize($object->getData(), $format, $context),
      'links' => $this->serializer->normalize($object->getLinks(), $format, $context)->omitIfEmpty(),
      'meta' => CacheableNormalization::permanent($object->getMeta())->omitIfEmpty(),
    ]);

ResourceIdentifierNormalizer

    if ($object->getMeta()) {
      $normalization['meta'] = $this->serializer->normalize($object->getMeta(), $format, $context);
    }

The problem is there's no way to add meta information to these objects. See ResourceObjectNormalizer::serializeField

      if ($field instanceof EntityReferenceFieldItemListInterface) {
        // Build the relationship object based on the entity reference and
        // normalize that object instead.
        assert(!empty($context['resource_object']) && $context['resource_object'] instanceof ResourceObject);
        $resource_object = $context['resource_object'];
        $relationship = Relationship::createFromEntityReferenceField($resource_object, $field);
        $normalized_field = $this->serializer->normalize($relationship, $format, $context);
      }

createFromEntityReferenceField takes a meta and links parameter. But there's no way to add information.

This is useful for building a headless application on Drupal to provide context about the information.

Problem/Motivation

Lastly discussed from #3098426-10: EntityQueryTest::testToString fails with non-core db drivers onwards, but really popped up several times already,

+    if (!in_array(Database::getConnection()->driver(), ['mysql', 'sqlite', 'pgsql'])) {
+      $this->markTestSkipped("This test only runs for Drupal core database drivers");
+    }

feels fragile.

We should have a more structured way to include/exclude tests that are database specific in the test runs.

Proposed resolution

Add a new annotation for test classes called @overridesTestClass. The annotation must be used with a fully-qualified class name of the test class that is to be overrridden. When used will the overridden test class not be run. It is best to use this annotation in combination with extending the overridden class by current test class. The current test class can then override anything it needs to. The current test class will then be run instead of the overridden test class.

Remaining tasks

TBD

User interface changes

None

API changes

New annotation @overridesTestClass will be ccreated for test classes.

Data model changes

None

Right now each field is multi-valued by default as we enforce this structure on all of our fields. This creates one intermediate, unused layer for all single-valued fields what has a performance impact memory and speed wise.

Considering we usually have quite some single-valued fields I think we should consider dropping the multi-valued enforcement for performance.
Related issue: #1869562: Avoid instantiating EntityNG field value objects by default

Impact:
You cannot rely on being able to do $entity->field[0] any more to get the first field item. You might have to do $entity->field instead. Still, we could support $entity->field->value to always work such that code won't break if you change a single valued field to become multiple valued. The other way round it breaks though.
Then, another question is whether we want to allow single valued fields for configurable fields (=field API fields) also. We could leave this possibility API-only and only allow developers to create them such that field API could continue to work on top of multiple-valued fields only.
Having single-valued fields would make some things easier also though. E.g. when serializing to JSON-LD (not drupal json-ld) I assume we could use the information which field is supposed to be single valued. Also, when I think of e.g. token replacements implemented on top of that, we'd naturally want single valued fields to appear single valued in a token browser... (related: #1869582: Leverage the Typed Data API for token replacements)

Problem/Motivation

If you look at the following services definitions you will see that there are quite of them which don't need to be public:

  asset.css.collection_optimizer:
    class: Drupal\Core\Asset\CssCollectionOptimizer
    arguments: [ '@asset.css.collection_grouper', '@asset.css.optimizer', '@asset.css.dumper', '@state' ]
  asset.css.optimizer:
    class: Drupal\Core\Asset\CssOptimizer
  asset.css.collection_grouper:
    class: Drupal\Core\Asset\CssCollectionGrouper
  asset.css.dumper:
    class: Drupal\Core\Asset\AssetDumper
  asset.js.collection_renderer:
    class: Drupal\Core\Asset\JsCollectionRenderer
    arguments: [ '@state' ]
  asset.js.collection_optimizer:
    class: Drupal\Core\Asset\JsCollectionOptimizer
    arguments: [ '@asset.js.collection_grouper', '@asset.js.optimizer', '@asset.js.dumper', '@state' ]
  asset.js.optimizer:
    class: Drupal\Core\Asset\JsOptimizer
  asset.js.collection_grouper:
    class: Drupal\Core\Asset\JsCollectionGrouper
  asset.js.dumper:
    class: Drupal\Core\Asset\AssetDumper
  library.discovery:
    class: Drupal\Core\Asset\LibraryDiscovery
    arguments: ['@library.discovery.collector']

The following could be marked as public: false

• library.discovery.parser
• library.discovery.collector
• asset.js.collection_grouper
• asset.js.dumper
• asset.js.optimizer
• asset.css.collection_grouper
• asset.css.dumper
• asset.css.optimizer

Proposed resolution

Mark them as public: falseand let the testbot decide whether this is okay.

Remaining tasks

User interface changes

API changes

Beta phase evaluation

Problem/Motivation

Coming from fixing the CSS Variables po[ln]yfill for IE, i stumbled upon this:

How to reproduce:

* Want to add library CSS/JS browser-conditional like so:

my_library:
  js:
    run-cssvars-polyfill.js:
      browsers: { '!IE': false }

Result:
* CSS/JS is wrapped in a browser-switch comment like this:

<!--[if IE]>
<script src="/modules/contrib/cssvars/polyfill/libraries/css-vars-polyfill.min.js?q16qed"></script>
<![endif]-->

Alas, M$ dropped the conditional script comment tag in 2016 since IE10 (SO). So it seems that the resulting browser switch comments only work up to IE9 (see #2390621: [policy, no patch] Update Drupal's browser support policy on why this is EOL but still supported).

Proposed resolution

* Update docs.

Once we drop IE9,

* Remove the functionality.
* Error out or add warning if developers use it.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Problem/Motivation

Conditional comments are only supported by IE <=9, and any core libraries that make use of them are already removed (see https://www.drupal.org/node/3089508)

Proposed resolution

Remove HtmlTag::preRenderConditionalComments()

Remaining tasks

User interface changes

API changes

The 'browsers' attribute of library assets is ignored.

Data model changes

Release notes snippet

I suddenly got this weird error after adding a new text field to a taxonomy. I thought it had something to do with cache, but clearing cache didn't help. I also deleted the field I had created, thinking that might be the cause, but the error is still there.

Something's getting mixed up with the Update module I guess. I have been working with this site for a while now and haven't seen this error before.

Tried Google but couldn't find anything helpful, so what might be the cause and the solution for this?

Here's what I got:

Notice: Undefined index: title in Drupal\update\ProjectSecurityRequirement::createFromProjectDataAndSecurityCoverageInfo() (line 103 of core/modules/update/src/ProjectSecurityRequirement.php).
Drupal\update\ProjectSecurityRequirement::createFromProjectDataAndSecurityCoverageInfo(Array, Array) (Line: 43)
update_requirements('runtime') (Line: 186)
update_page_top(Array) (Line: 336)
Drupal\Core\Render\MainContent\HtmlRenderer->buildPageTopAndBottom(Array) (Line: 135)
Drupal\Core\Render\MainContent\HtmlRenderer->renderResponse(Array, Object, Object) (Line: 90)
Drupal\Core\EventSubscriber\MainContentViewSubscriber->onViewRenderArray(Object, 'kernel.view', Object)
call_user_func(Array, Object, 'kernel.view', Object) (Line: 111)
Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch('kernel.view', Object) (Line: 156)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 68)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 57)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 106)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 85)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 52)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 23)
Stack\StackedHttpKernel->handle(Object, 1, 1) (Line: 694)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

Problem/Motivation

Adding third party library systems (such as for web components and pattern libraries) and customizing the behavior of assets (js / css parsing & collections settings) is difficult, if not impossible, with the current core system.

If a module wanted to add library support for additional systems, it would need to recreate core's library handling, and use a workaround (such as drupalSettings: {}) for libraries consisting only of new systems (see https://drupal.org/project/patternkit or any other module that attempts to add support for HTML fragment libraries).

The Asset Libraries system is over-complicated:

• AssetResolver
  • LibraryDependencyResolver
    • LibraryDiscovery + Interface
      • LibraryDiscoveryCollector
        LibraryDiscoveryParser

and over-specific:

• CSSCollectionOptimizer
  • CSSCollectionOptimizer
    • CSSCollectionGrouper
    • CSSOptimizer
    • CSSDumper
• CSSCollectionRenderer

• JSCollectionOptimizer
  • JSCollectionOptimizer
    • JSCollectionGrouper
    • JSOptimizer
    • JSDumper
• JSCollectionRenderer

Proposed resolution

Each item here will be created as sub-issues under this META issue's Remaining Tasks section to keep this issue centered on discussion and the sub-issues centered on implementation.

1. Merge LibraryDiscovery and LibraryDiscoveryCollector into a single \Drupal\Core\Asset\Discovery\Library class #2279553: Move library discovery into Core\Asset\Discovery namespace and add 'asset.' prefix to service IDs.
2. Create a new Library class to replace the arbitrary library arrays, to make development easier and make it clearer which fields are required.
3. Move LibraryDiscoveryParser to \Drupal\Core\Asset\Parser\Base class.
4. Break out the parser tooling into subclasses \Drupal\Core\Asset\Parser\JS & CSS.
5. Add support for Library plugins to the new Library class.
   • Add Plugin\Library annotation class.
   • Add LibraryPluginDefault class.
   • Add LibraryPluginDefinition class.
   • Add LibraryPluginInterface interface.
   • Add LibraryPluginManager class and plugin.manager.library service.
6. Genericize AssetType* classes into \Drupal\Core\Asset\AssetCollectionOptimizer etc.
7. Move specific AssetType* class functionality into Library plugins (JS, CSS).
8. This would possible enable a future separate issue of: Add patternkit module to core to support HTML fragment libraries such as Twig and WebComponents.

Remaining tasks

(reviews needed, tests to be written or run, documentation to be written, etc.)

User interface changes

None.

API changes

Addition of Library Plugin support.

Data model changes

None.

Release notes snippet

(Major and critical issues should have a snippet that can be pulled into the release notes when a release is created that includes the fix)

Follow-up to #2225775: Migrate Drupal 6 core node translation to Drupal 8

Problem/Motivation

In D8, a node revision has data for all languages. In D6 and D7, each node revision has data for just one language. This means currently D6 -> D8 and D7 -> D8 migrations yield a revision history where each node revision has only one language present, even if previous revisions had translations. This is strange and likely to confuse users.

Priority is given to the D7 migration because the D6 one might be effected by the term node migrations.

This is for migrating core translation revisions, migrating revisions for entity translation is now being done in #3076447: Migrate D7 entity translation revision translations. It is intended that this issue is completed first and then the approach used here is also used for entity translations.

The following notes use the term 'master' which was the original term what is now the 'complete' node migration.

Meeting notes

Things to consider, taken from notes of meeting with alexpott, Gábor Hojtsy, heddn, mikelutz and quiteone

1. Migration dependencies
   • Possible: easy to do
   • BC concerns: none
   • BC what about contrib? Commerce / metatag
   • Possible incompatible
2. How do we figure out you need the new complex master migration?
   • Possible: moderate to do
   • BC concerns: none
   • Don’t effect expose new d*_node_master unless explicitly enabled to existing sites
   • This handles incremental, no UI option available to enable
   • For new installs, give an option to upgraders to enable disable the new d*_node_master migration and use the old method.
3. What about migration lookups on dn_node etc.
   • Possible: easy to do
   • BC concerns: none
4. What about incremental migrations that have already begun
   • Not allowed. You can only use the new approach on a site that has not run migrations. So we need some of flag where existing installs use the old migrations.
5. Contrib/Custom
6. Does it make sense to use d*_node_master instead of d*_node migrations?

Proposed resolution

The current focus is to get the D7 version working and then move those changes to D6.

Using the existing method where the final revision is migrated first doesn't work instead the revisions need to be migrated in order. The first proof of concept patch to show this is in #121, thanks to mikelutz. This new approach is called the Complete node migration and the migrations and files use that name.

The migrate_drupal_ui functional tests use the complete node migration, as this is now the default. The exception is d6/NodeClassicTest which as the name suggests uses the classic node migration.

Do not deprecate the classic node migrations
"Removing it [the classic node migrations] will break the whole ecosystem. Deprecating it means that it’s not used anymore which in and of itself will break the eco system, and for what benefit? To remove it in Drupal 10, a year after D7 is EOL? At that point we are looking at deprecating the entire D7 upgrade path anyway (Probably for removal in 11, not 10) We really don’t want to break the migration ecosystem right now at this critical time when we are trying to get everyone off of D7 in the next 18-20 months". by mikelutz in #migration on Drupal Slack). See #3109819: [PP-2] Deprecate classic node migration plugin in Drupal 9 for removal in Drupal 10 for discussion of when deprecation of classic node migrations would happen.

Run Complete or classic node migrations
However, the Complete approach doesn't integrate with the current migrations, now being called Classic, that is the one either runs the existing node migrations or the new complete one, not both. Choosing between the two methods is done at run time by checking the node migration map tables in MigrationConfigurationTrait() and migrate_drupal_migration_plugins_alter(). In the trait the classic method is selected if any of the classic node migrate_map tables has data (using processedCount) and in the plugins_alter it is selected if any classic node migrate_map exists and there are no complete node migrate_maps.

The selection of node migration method also allows tests to decide which method to use. The test does this by adding either 'node_migrate_classic' or 'node_migrate_complete' to $modules.

Destination IDs
The entity complete destination plugin creates up to three IDs. First is the entity ID key. Then the revision key and the langcode, if they exist.

Altered migrations
There are several migrations that use migration_lookup on the node migration. The processes and dependencies for these are altered when the complete node migration is being run. The changes include new processes to convert the 3 destination IDs returned by complete node to the correct destination ID expected when using the classic migration.
The migrations altered are: d6_term_node, d6_term_node_revision, d6_term_node_translation, d6_comment, d6_url_alias,d7_comment, d7_url_alias, statistics_node_counter, node_translation_menu_links.

Incremental migrations
For the core UI, if map tables for the classic mode migrations exist and at least one has a row then the classic migrations will run and not the complete node migrations. This is determined at run time. Therefor, existing incremental migration should continue to function.

From the original IS
D6 and D7 node revisions do not directly indicate which other translated node revisions they are associated with. But we can attempt to guess this, based on the order of revisions.

Remaining tasks

As of 2020-05-11, the only remaining task is to update the patch for 8.8.x so that classic migrations are the default (set in settings.php).

Fix tests, add comments
Review

For Reviewers

• All level of code reviews welcome.
• There are BC issues raised in meeting notes, above

For Testers

1. Use the lastest patch patch in this issue.
2. There are instructions in the Issue summary of the META issue #2208401: [META] Remaining multilingual migation paths

From #55

For developers

1. Make this work with a multilingual source and a non multilingual source
2. Discuss and document implications for drush, especially drush migrate-upgrade --all
3. Make sure the d6 and d7 tests includes testing a fields on each revision
4. Handle the case where the vid of the first revision is higher the vid of a later revision. This is true for both d6 and d7.
5. Make sure that the use of migration_tag 'translation' is documented. It is used in
   • d6_node_translation.yml
   • d7_node_entity_translation.yml
   • d7_node_translation.yml
   • d7_node_revision_translation.yml
   • d7_user_entity_translation.yml
   • d7_taxonomy_term_entity_translation.yml/li>
   • d7_comment_entity_translation.yml
   • d6_node_revision_translation.yml
6. Make sure that the source plugin 'translation' property is documented
7. Make sure that the destination plugin 'translations' property is documented

Release notes snippet

A new node migration, d*_node_complete.yml, referred to as the 'complete node migration', now exists that will migrate all nodes, the node revisions, including translated nodes and the translated node revisions. The complete node migration will eventually replace the existing trio of node migrations, d*_node, d*_node_revision and d*_node_translation, now collectively referred to as the classic node migrations. See https://www.drupal.org/node/3105503 for more information.

Problem/Motivation

Adding a Term reference field is not working as expected: link on profile page leads to taxonomy/term page which says "There is currently no content classified with this term."

Proposed resolution

There are 2 different solutions:
- create a custom view for the taxonomy/term/% pages;
#1016942: Allow taxonomy/term pages to list multiple entity types
- change core to show not only 'node' entities;
#1137558: Show multiple entity types on Core taxonomy/term pages, not just 'node'

Related Issues

See above.

Problem/Motivation

The Content Security Policy module needs to add 'unsafe-inline' exceptions for scripts and styles in order to support AJAX responses (#3100068: Script/style included in AJAX responses blocked without 'unsafe-inline').
This is because CSP blocks script and style elements dynamically added to the page by appending HTML, even if their sources comply with the page's policy.
Since AJAX is widely used, this opens a significant gap in the Cross-Site Scripting protections that Content Security Policy offers.

Proposed resolution

- Add a new Drupal\Core\Ajax\CommandInterface() implementation that serializes an array of assets the calling page needs to add.
- Update \Drupal\Core\Ajax\AjaxResponseAttachmentsProcessor::buildAttachmentsCommands() to use the new command instead of rendering the assets to HTML and using an append/prepend command.
- Update Drupal.AjaxCommands with a new method to execute the corresponding command and add assets in a CSP-safe manner (see https://github.com/jquery/jquery/issues/3969).
- Deprecate \Drupal\Core\Ajax\AddCssCommand, which expects a rendered HTML string.

Remaining tasks

- Determine whether retaining support for to-be-deprecated 'browsers' is necessary (@see #3095113: Deprecate IE conditional comments support)

User interface changes

None

API changes

Release notes snippet