Problem/Motivation

Make a plan of migrate priorities for D10 which is scheduled for release in December 2022

Proposed resolution

High priority

#3039240: Create a way to declare a plugin as deprecated

Problem/Motivation

If a migration database connection fails you get a very generic error message like

In Connection.php line 197:

  SQLSTATE[HY000] [1045] Access denied for user 'USER'@'IPADDRESS' (using password: YES)

If dealing with a situation with multiple databases that can be very unhelpful.

Steps to reproduce

Run migrate task that hits an unavailable database or uses an incorrect password/username

Proposed resolution

Catch the Database/PDO exception and throw a more detailed exception.
Suggesting something like:

Migration MIGRATIONNAME failed to connect to CONNECTIONNAME:default. SQLSTATE[HY000] [1045] Access denied for user 'USER'@'IPADDRESS' (using password: YES)

Problem/Motivation

Let's assume two users enable two different modules on the admin/modules page at the same time.
Then the slightly slower attempt will win, as its comes after the first one

Proposed resolution

Add a lock around module install.

Inject the lock service into the module installer service (\Drupal\Core\Extension\ModuleInstaller)
Wrap the ::install method in a lock - see the Lock API for details.
If the lock can't be acquired after waiting, throw an exception
Add a test to \Drupal\KernelTests\Core\Extension\ModuleInstallerTest to ensure the lock prevents two people from installing at the same time
Add code to \Drupal\system\Form\ModulesListForm::submitForm and \Drupal\system\Form\ModulesListConfirmForm::submitForm to catch the thrown exception and display a message to the user
Add tests to \Drupal\Tests\system\Functional\Form\ModulesListFormWebTest for these scenarios

Remaining tasks

Review + Merge

User interface changes

N/A

API changes

ModuleInstaller now has a 6th (currently optional) parameter, Lock. This is optional for BC purposes only, this is actually required and defaulted if not present.

Data model changes

N/A

Problem/Motivation

Drupal's mail command sets the Return-Path header directly. This is in violation of the relevant RFCs.

https://tools.ietf.org/html/rfc2821#section-4.4

A message-originating SMTP system SHOULD NOT send a message that
already contains a Return-path header. SMTP servers performing a
relay function MUST NOT inspect the message data, and especially not
to the extent needed to determine if Return-path headers are present.
SMTP servers making final delivery MAY remove Return-path headers
before adding their own.

More recent RFCs:

• https://tools.ietf.org/html/rfc5321#section-4.4
• https://tools.ietf.org/html/rfc5322#section-3.6.7

The Return-Path header is set by the SMTP server to the value of the envelope sender (MAIL FROM: SMTP command). When using sendmail or other local MTA this can often be set using the "-f" option.

When Drupal sets the Return-Path header the recipient mail server may reject the email or ignore the header. If the email gets through, it results in 2 headers which can be different and which the recipient might flag as spam.

External mail services, such as Mailgun or Sendgrid, set their own envelope sender (therefore Return-Path header) to capture any bounces. Drupal should not try to influence the header at all.

External SMTP services may use Variable Envelope Return Path.

Proposed resolution

Do not set the "Return-Path" header within Drupal and provide more detailed documentation on Drupal's mail system.

That is, remove Return-Path altogether and set the envelope sender to $message['from']. See https://api.drupal.org/api/drupal/core!core.api.php/function/hook_mail_a...

PHPMailer has resolved the issue:

• https://github.com/PHPMailer/PHPMailer/issues/144
• https://github.com/PHPMailer/PHPMailer/pull/210

Problem/Motivation

A request that immediately follows a cache rebuild or a memcached flush results in a major performance issue, as Drupal tries to cache all available blocks at once. We ran into an issue on a couple of applications where memcached didn't have enough memory allocated to withstand the onslaught of memcache_set. This resulted in multiple calls for all of the blocks to be set en masse in memcached - because it attempts to fulfill its initial request ("write ~2300 blocks to cache_discovery, please"), then runs out of room (hits the memory ceiling), then tries to complete the job by taking a second swipe at it. You can see this at work in this xhprof run:

MemcachePool_set is being called over 4600 times here - which is approximately 2X the number of custom blocks in the database:

mysql> SELECT COUNT(*) FROM block_content;
+----------+
| COUNT(*) |
+----------+
|     2294 |
+----------+
1 row in set (0.00 sec)

Proposed resolution

Skip the entity api completely and go directly to the tables for the required data. The entirety of the required data for the current deriver is contained in block_content and block_content_field_revision. A single simple join is much faster and more performant for sites with heavy custom block usage.

Remaining tasks

Validate the approach in the patch and update as necessary.

User interface changes

None

API changes

None

Data model changes

None

Problem/Motivation

PermissionsHandler:moduleProvidesPermissions costs quite a lot and so should be static cached.

Proposed resolution

Static cache result
Let ::getPermissions accept an optional module name.

Remaining tasks

Patch

User interface changes

None

API changes

New optional argument to PermissionsHandlerInterface::getPermissions - BC is maintained

Beta phase evaluation

Problem/Motivation

It looks like menu links that are localized are not migrated properly. I discovered this while working on #2975461: Convert query string to array for d6 menu_link migration and I was only looking at the because it came up as the triage issue for the day in #bugsmash earlier this week. So, well done bugsmash!

Steps to reproduce

Run a menu link migration from d6 and localized ones, not in the default language, get migrated with the default language instead of the menu language.

Proposed resolution

Add a migration for localized menu links, d6_menu_links_localized.

Remaining tasks

Patch
Tests
Review
Commit

User interface changes

N/A

API changes

N/A

Data model changes

N/A

Release notes snippet

N/A

With some providers (Youtube from what I could find) we have aspect ratio problems when we render them using the OembedFormatter.

The problem I discovered is primarily with Youtube: the resource returned from Youtube does not contain the requested width/height but the default size (or simply A size) of the video. This is fine if the formatter is configured to use a width and height that corresponds to the aspect ratio of that particular video. But that won't ever happen. When we configure the formatter to a specific max width but a height of 0, there is at least some expectation that the rendered iframe maintains the aspect ratio. At the moment, since the formatter is configured to a *max* width and no *max* height the iframe gets that that width attribute and the default height that comes from the resource. So the result is that the iframe size doesn't match the size of the video. A solution for this is to calculate the aspect ratio of the video using the values in the resource and recalculate the height attributed based on that in case the max height is configured as 0.

Discussing with @phenaproxima we might make a checkbox (default unchecked) for this in order to prevent BC.

Other providers, like Vimeo, may not have this issue. So in these cases this should not apply. But youtube is pretty important.

Problem/Motivation

Not sure if this is something particular to my installation, but after upgrading core from 8.6.13 to 8.7.0-beta1, a node save operation started throwing warnings of type:

Trying to get property 'value' of non-object

at

\Drupal\content_moderation\Plugin\Validation\Constraint\ModerationStateConstraintValidator::getOriginalOrInitialState()

/var/www/html/docroot/core/modules/content_moderation/src/Plugin/Validation/Constraint/ModerationStateConstraintValidator.php line 160

Stepping there with the debugger shows that when the error is thrown, $original_entity is empty, and that is because $entity->getLoadedRevisionId() some lines above also returns NULL.

See below the full trace, in case it helps.

Proposed resolution

Not sure why in the \Drupal\content_moderation\Plugin\Validation\Constraint\ModerationStateConstraintValidator::isFirstTimeModeration() some lines below, the $original_entity is retrieved differently. It seems like using the same approach in the ::getOriginalOrInitialState() method is enough to solve the issue. I ignore however if this is just a band-aid to hide the issue or it could be a legitimate fix.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Description

With a drupal-site, that has translations enabled and e.g. search_api or bef-module installed executing drush cim will cause the system to throw an exception in TranslatableMarkup Line133 and depending on core-version, loop infinitely or fail although the configuration itself was properly imported.
Rerunning drush cim will then finish without a problem, as all the configurations have been imported and the translations that were missed are not recreated
This problem is neither compatible with with a CD-approach nor does it inspire confidence.

Reason

The problem arises because LocalConfigManager:178 checks that the value is neither an empty string nor null, but it does not check if it is a string at all.
BEF-configuration in yaml:
....
bef_slider_step: !!float 1
....

Fix
The following patch will test that the value is a string and not empty.

Problem/Motivation

It is pretty common to want to use a token for a URL when editing text. For instance, you might want something like:

<a href="[my:token:here]">link text here</a>

However, if you try to do this in a body field that is using CKEditor, with a text format that uses the "Limit allowed HTML tags and correct faulty HTML" filter, if you enter a token like that in the field, and then later edit the content that contains that field and either look at the HTML source or save the content, the token is corrupted. This seems to be independent of which CKEditor buttons you have configured, so it isn't a problem (apparently) with a particular button plugin.

Additionally, some (but not all) other attributes in HTML get corrupted in the same way, if your text format allows these attributes. Some examples that were tested:

Tokens in the text that are outside of HTML attributes do not get corrupted.

Steps to reproduce:
a) In a content item with a field that has a text format that is configured to use CKEditor for editing, and which contains the "Limit allowed HTML tags and correct faulty HTML" filter, and allows the A tag, type some link text in the editor.
b) Highlight to select the link text.
c) Click the Link button (chains) in the editor toolbar, and enter [my:token:here] as the URL in the popup.
d) Click Save in the popup. Verify that the HTML source looks like

<a href="[my:token:here]">link text here</a>

e) Save the content item you are editing.
f) Test -- the link works fine (assuming you are running a token replace so it gets replaced by the right URL).
g) Edit the content item again.
h) When you get back to the editor, look at the HTML source. Instead of seeing what was there before, you will see something like this:

<a href="en:here]">link text here</a>

So that's the bug: if you re-edit some HTML text using CKEditor and the "Limit allowed HTML tags and correct faulty HTML" filter, and there is an A tag with a token in the URL (or tokens in various other attributes, but not all attributes), CKEditor truncates and mangles the token, leading to data loss. According to Priority Levels of Issues, this means it is a Critical bug (or at least Major?) because it leads to data loss.

Note: We are specifically seeing this in the proposed Help Topics module (see related issue #2943974: Work-around for route tokens in Help text format getting truncated after editing a help topic).

Proposed resolution

Fix Drupal so that CKEditor doesn't mangle tokens in URLs in A tags.

The problem was traced down to a bug in \Drupal\Component\Utility\Xss::attributes(). If you pass in an attribute string like href="[something:something:config_basic]" to this function, you get out something that looks like href="config_basic]".

Remaining tasks

Fix the bug in Xss::attributes().

User interface changes

CKEditor will not mangle HTML containing tokens for URLs in A tags, or other HTML tag attributes.

API changes

None.

Data model changes

None.

Follow-up to #2105841: Xss filter() mangles image captions and title/alt/data attributes

Problem/Motivation

\Drupal\Component\Utility\Xss::filter() cleans potentially dangerous protocols like "javascript:" from element attributes. It does this by stripping any set of characters that ends with a colon, unless the string is "http:" or "https:".

The filter strips out valid attribute name/value combinations that provide RDF metadata, such as rel="schema:author" or property="foaf:name".

Some attributes are exempt from this treatment, including `alt`, `title`, and any `data-*` attribute. In #2105841: Xss filter() mangles image captions and title/alt/data attributes, the decision was made to hard-code the exempt attributes list, and possibly make the list configurable in a follow-up issue.

Proposed resolution

None yet.

Remaining tasks

None.

User interface changes

None.

API changes

None.

Beta phase evaluation

-->

See test failures at https://travis-ci.org/drush-ops/drush/jobs/237376086.

To reproduce, make sure you have php7 on your path. The steps below build a Composer project in your temp directory. That project contains latest Drush (symlinked in) and latest Drupal 8.4.x. The final line, 'sut', runs the Drush site-install command in that Composer project. You can set the environment variable UNISH_DB_URL=[mysql://root:@127.0.0.1] if you don't like the default I have there.

git clone https://github.com/drush-ops/drush.git
cd drush
COMPOSER=composer-highest.json ./unish.sut.php
./sut site-install testing --db-url=[URL] -d -y

Here's a very simple patch...

I noticed that the default pager's links were missing rel attributes, which can be useful if you happened to have become addicted to some Site Navigator browser extension... With this Drupal 5.0 patch, the pager will make the browser aware of first, prev, next and last links, where that feature is available, of course...

Included with this post is a diff file for the includes/pager.inc script I generated from winMerge. I'm sorry I couldn't post a more standard compliant patch. As I'm still just getting acquainted with Drupal's internals, I haven't set up my installation under a full development environment.

I hope it helps more than it annoys...

Follow-up to: #1168246-136: Freedom For Fieldsets! Long Live The DETAILS.

Details should have ARIA added to them so that the open/closed state is semantically determinable:
http://wet-boew.github.com/wet-boew/demos/details/details-eng.html

<summary aria-expanded="false" aria-pressed="false" role="button" aria-controls="details-1" style="-moz-user-select: none;" tabindex="0">Example 2</summary>
<div style="display: none;">

<summary aria-expanded="true" aria-pressed="true" role="button" aria-controls="details-1" style="-moz-user-select: none;" tabindex="0">Example 2</summary>
<div style="display: block;">

Coming from #2045039-7: Cleanup HTML heading structure @terrill:

That said, I still think we should add aria-label attributes to all elements. There are many of these on each page, and without a label they're each identified by screen readers as "Navigation region" - there's no means of clarifying which navigation region is which. If we add aria-label, screen readers use the value of aria-label when identifying the region (e.g., "Toolbar Items Navigation Region", "Breadcrumb Navigation Region", and the like)

@falcon03 and later on:

Well, I think that hearing messages like "entering navigation", "entering main" and so on doesn't give a real "context". So, as a blind user, I'm completely in favor of adding aria-labels.

@pratikp1 and finally:

Having an effective label is essential. I would like some discussion about that for D8 or D9. Let's see if we can make it more understandable. It will go a long way toward having users understand that landmarks can actually be useful.

I don't know why there isn't any in Seven, but in Bartik, we can find the <nav> element in:
core/themes/bartik/templates/comment.html.twig
core/themes/bartik/templates/page.html.twig

And in modules it's in:
core/modules/book/templates/book-all-books-block.html.twig
core/modules/book/templates/book-navigation.html.twig
core/modules/system/templates/breadcrumb.html.twig
core/modules/system/templates/page.html.twig
core/modules/toolbar/toolbar.module

Where else do we find them?

Some more info on using aria-labels:
http://www.last-child.com/debugging-aria-label/

Problem/Motivation

template_preprocess_node() and template_preprocess_comment both call user_view($author, 'compact'); but the compact entity display settings are defined in standard.profile and hence may not exist.
In best case scenario, extra fields are rendered (these are rendered by default for a non-existent display mode).
In worst case scenario, you end up with recursion (#2114887: Maximum nesting level when attaching comment field to User.)

Proposed resolution

Node module comes with a new user view mode (node_author) and configured display mode
Comment modules comes with a new user view mode (comment_author) and configured display mode
Standard profile comes with overrides of the configured display modes and include image field settings for user pictures (comment and node cannot know if image module is enabled in their config).
Upgrade path.

Remaining tasks

Everything

User interface changes

None

API changes

None

Data model changes

New view mode and entity display config objects.

Problem/Motivation

In #2986452: Database reserved keywords need to be quoted as per the ANSI standard we fixed working with reserved keywords for queries. For the class Drupal\Core\Database\Schema there is nothing done. In #3023794: MySQL >= 8.0.2 Bug: Exception during module activation because of reserved keyword "GROUPS" we have a problem with using sites with the "group" module on Drupal 8 with MySQL 8. The problem should be fixed for Drupal 9, only for queries not for the class Drupal\Core\Database\Schema.
In #3033043: SQlite index names not quoted we have the problem: "An example of such a module is restrict_ip (https://www.drupal.org/project/restrict_ip) which defines an index named 'type-path'. The index name is perfectly valid on both MySQL and SQlite."

Proposed resolution

Add testing for working with reserved keywords in combination with the class Drupal\Core\Database\Schema. Add fixes where necessary.

Remaining tasks

TBD

User interface changes

None

API changes

None

Data model changes

None

Release notes snippet

TBD

For the committer: Please give user @jsst commit credits for his work on #3033043: SQlite index names not quoted.

Problem/Motivation

We have defined a workflow with these states:

1. Draft
2. Needs Review
3. Published

We have two roles:

• author (Who creates content in draft and sets it to "needs review")
• admin (Who can transition from "needs review" to "published")

Steps to reproduce

1. Enable the Content moderation and Content translations modules, and their requirements.
2. Add another language to the site.
3. Enable translation for Content (nodes) for at least one content type.
4. Enable the Editorial workflow for at least one translatable content type.
5. Configure the workflow with the 3 states listed above (draft, needs review, published), and 4 transitions:
   1. draft -> draft
   2. draft -> needs review
   3. needs review -> draft
   4. needs review -> published
6. Configure 2 roles:
   • author (with permission to use draft -> draft and draft -> needs review transitions)
   • editor (with permission to use all of the above workflow transitions)
7. An author has created a node in English, and sets it to "needs review".
8. Now the author wants to add a translation for this node. They go to the Translate tab and click on the "Add" Button.
9. The form where they can edit the node and add the german texts is displayed.
10. When they click "Save", they gets the error message "Invalid state transition from Needs review to Needs review".

Logically that is correct, because the author role doesn't have the permission to edit nodes which are in the state "needs review".

But shouldn't new translations be treated as new nodes and therefore be created as "draft"?

Proposed resolution

Preserve the existing behavior for BC, but add a way to configure this when defining a workflow.

Whatever new UI is introduced should only be visible if both Content moderation and Content translation modules are enabled.

Option B

(Other rejected proposals removed, this is what we're going with).

• Have a simple toggle for the current behavior vs using the default moderation state (same as A)
• UI is 2 radios.
• Setting stores a string, ID would be 'translation_default_moderation_state_behavior' or so, with values like 'current' vs. 'default'.
• In some UIs, explicit radios for different options can be more clear than a checkbox (if each choice requires a little explanation).

Example UI:

Initial moderation state when adding a new translation:
(*) Default moderation state
( ) Current moderation state of the original content

Open questions

1. Which UI is best for the setting? Option B - radio buttons. See #45
2. Since all the code handles no value for this setting as if you want the current behavior, do we need an upgrade path and all that complication? Yes (@larowlan in #42)
3. Do we want to add this setting to the default workflows shipped with core? YES
   1. core/profiles/standard/config/optional/workflows.workflow.editorial.yml? YES
   2. If so, should we set it to 'default' or 'current'? Using 'current' per @larowlan in #42
   3. Also demo_umami? YES
4. Does this issue need a CR? YES: https://www.drupal.org/node/3240957
5. Do we need more test coverage? YES:
   1. Do we need to test the UI for this setting when configuring a workflow? YES (per @larowlan in #44). DONE in patch #47
   2. A kernel test covering the scenario in the issue summary YES (per @larowlan in #42). DONE in patch #47
6. Should the #default_value for these radios use 'default' for new workflows? It's now defaulting to 'current' if missing from the config for BC on existing sites. But presumably we could notice when creating a new workflow and use a different #default_value to be the more intuitive 'default'. NO, leave it as 'current' (per @larowlan in #42).
7. Where is the right layer of the entity / translation / content_moderation systems to specify the default moderation state of a new translation? See comment #33. We don't want to have to rely on field widgets to enforce this setting. It'd be better if we directly changed the moderation_state when populating a new translation so that field widgets don't need to support this setting, and so it works if there is no widget for moderation_state. Should we?
   1. Use hook_entity_translation_create() YES.
   2. Directly hack EntityOperations::updateOrCreateFromEntity()? This seems problematic since we'll have a hard time knowing if the entity has a value from a widget or if its own override logic should kick-in. NO
   3. Other?
8. What should the UI help text say for the "Legacy" config option? See #101.

Remaining tasks

• Answer the open questions.
• Implement the new setting / UI.
• Don't check if the user has access to this transition (because it should be treated as new).
• Add a DB update for existing sites
• Add tests:
  1. What you see when creating a new translation (the bug)
  2. The two different roles case described in the summary. Done in #47
  3. The upgrade path
  4. The UI for this when creating a workflow. Done in #47.
• Decide what to do that the legacy behavior is not "Current moderation state of the original content" as the UI suggests, but actually a hardcoded choice between exactly 'published' (if the original content is currently in any published state) or 'draft' (for any unpublished current state). See #91 UX team proposed a solution in #101, implemented in #104
• RTBC

User interface changes

A new setting when creating/editing a workflow:

Before

What you see without the patch applied, or when content_moderation is enabled but content_translation is not, at paths like /admin/config/workflow/workflows/manage/editorial:

After

With the patch applied, and both content_moderation and content_translation enabled:

Initial moderation state when translating existing content*
(*) Default moderation state
( ) Published if existing content is published, otherwise Draft

API changes

None.

Data model changes

New config schema for a setting on workflow type plugins.

Release notes snippet

TBD.

Problem/Motivation

Across the admin interface there are several pages with buttons with a grey background color, regular buttons as well as drop buttons. all with a background-color of #C1C2C7 on either on a white backgroud #FEFFFE (1.8:1) or light blueish table headers #f3f4f9 (1.6:1). In each case that fails WCAG21 SC 1.4.11 which requires a contrast ratio of 3:1.
I've asked @mgifford for feedback and advice about the buttons cuz I was uncertain if I was overcautious or if it is a necessary step. But he confirmed that the grey buttons either need an outline with more contrast or more contrast for the background color. Either should be fine and he doesn't have a strong preference and he doesn't know which would be best for the design. I'll add aside the Accessibility tag also the needs design tag since aside the change of color there also needs to be some design decision.

Steps to reproduce

- go for example to admin/structure/views/view/content or admin/content
- check the background color of the grey buttons