Problem/Motivation

Entity query has a use case for "disabling" a db query condition (i.e. a kill switch on the condition). The current implementation does not work if we decide to escape all identifiers/fields of a condition by default, as discussed in #2966523: MySQL 8 Support.

Contrib also has this use case, for example in the Tree module.

Proposed resolution

Add a alwaysFalse() method to \Drupal\Core\Database\Query\ConditionInterface and implement it in \Drupal\Core\Database\Query\QueryConditionTrait.

Remaining tasks

Review, etc.

User interface changes

Nope.

API changes

API addition, a new method on ConditionInterface.

Data model changes

Nope.

Problem/Motivation

Starting from #2351919: Replace uses of drupal_get_path() with __DIR__ where possible, when the PHP code needs to include or parse files inside the module or theme directory space, will simply use _DIR_ instead of drupal_get_path(). This is sufficient, intuitive and more performant than calling drupal_get_path().

However, drupal_get_path() is widely used to refer files (.js, .css, images, assets) outside a module or theme. This isn't very intuitive for new developers. Stream wrappers make much more sense and simplify the API for developers trying to refer directories or files from outside the current extension in code.

Proposed resolution

Introduce extension stream wrappers:

Examples

Profile: profile://

Assuming the standard profile is installed:

• Referring a directory:
  • Actual: drupal_get_path('profile', 'standard') . '/config'
  • Proposed: profile://config
  • Path: core/profiles/standard/config
• Referring a file:
  • Actual: drupal_get_path('profile', 'standard') . '/config/install/automated_cron.settings.yml'
  • Proposed: profile://config/install/automated_cron.settings.yml
  • Path: core/profiles/standard/config/install/automated_cron.settings.yml

Module: module://

Assuming the node module is enabled but color module is not:

• Referring a directory:
  • Actual: drupal_get_path('module', 'node') . '/config'
  • Proposed: module://node/config
  • Path: core/modules/node/config
• Referring a file:
  • Actual: drupal_get_path('module', 'node') . '/config/install/node.settings.yml'
  • Proposed: module://node/config/install/node.settings.yml
  • Path: core/modules/node/config/install/node.settings.yml
• Referring a resource in a uninstalled or inexistent module:
  • Actual: drupal_get_path('module', 'color') . '/config'
  • Proposed: module://color/config
  • Path: Throws \RuntimeException

Theme: theme://

Assuming the bartik theme is installed but seven theme is not:

• Referring a directory
  • Actual: drupal_get_path('theme', 'bartik') . '/config'
  • Proposed: theme://bartik/config
  • Path: core/themes/bartik/config
• Referring a file
  • Actual: drupal_get_path('theme', 'bartik') . '/color/color.inc'
  • Proposed: theme://bartik/color/color.inc
  • Path: core/themes/bartik/color/color.inc
• Referring a resource in a uninstalled or inexistent theme:
  • Actual: drupal_get_path('theme', 'seven') . '/config'
  • Proposed: theme://seven/config
  • Path: Throws \RuntimeException

Remaining tasks

None.

API changes

New abstract class for extensions stream wrappers:

• \Drupal\Core\StreamWrapper\ExtensionStreamBase

New stream wrapper classes:

• \Drupal\Core\StreamWrapper\ModuleStream
• \Drupal\Core\StreamWrapper\ThemeStream
• \Drupal\Core\StreamWrapper\ProfileStream

Data model changes

None.

Problem/Motivation

The core migrate system needs a way to import things from CSV files.

Immediate use case is for the Umami Demo profile now in core in 8.6.x. The imported content is in CSV files and we have code to read it and create the relevant entries. Having migrate_source_csv in core would allow us to remove that code and use migrate instead.

See #2809635-37: Create experimental installation profile for where @larowlan suggested bring migrate_source_csv into core.

Other good reasons to have CSV as a migration source in core:

• "I use this constantly. I've seen it forked into the Contenta distro, and a few of my own projects (I just want the source plugin, not another module.) It'll also be vital to Commerce integrations as CSV is a common way to import data from an ERP." @mglaman (#4)
• "CSV format is the first format used by non technical professionnals who need to import data and to generate/edit those datas thanks to software they knew (MS Excel without mention it). It's also the easiest way to export data from many ERP and big solutions with datas." @GoZ (#5)
• "CSV is simply a really common used format and it would make migrate more usable by default for many usecases. One could say: Oh it is easy to parse CSV, but it turns out it is not. On top of that the module also doesn't load the entire CSV file upfront, but rather reads line by line. I think this makes it a good candidate also for bigger migrations out there." @dawehner (#14)
• "This would help the Out of the Box Initiative. Let's do it! Consider this my maintainer +1 :)" @phenaproxima (#18)

Proposed resolution

The migrate_source_csv module has a stable release (2.0), has been downloaded almost 100k times, and comes complete with tests. The contenta CMS has been using this for their migrations, too.

Rename the files/namespaces, and move it in, as-is, to the core migrate module.

Since this is a straight port from contrib, discussions about fetchers, etc. are postponed to follow-ups. See #16, #36 and #37 for some of the highlights of the discussion.

Follow-up: #2962091: Adopt fetchers/parsers logic for use by source csv plugin

Remaining tasks

1. Patch that renames files/namespaces and moves the contrib code into core Done.
2. Tests Done.
3. Word-smithing PHPDocs Done.
4. Decide if we're okay with a straight port (done), or if we need to hash out all the fetcher/parser stuff first, figure out the design, class names, avoid conflicts, possible BC hell, etc. Done. See comments #80 through #83. Punted to #2962091: Adopt fetchers/parsers logic for use by source csv plugin.
5. Finish cleaning up the docs and tests.

None.

User interface changes

None.

API changes

Adds a new migrate source plugin called "csv" in migration yml definition files. The class providing the plugin (\Drupal\migrate\Plugin\migrate\source\CSV) is heavily commented with PHPDoc.

Adds a \Drupal\migrate\CSVFileObject class (which extends \SplFileObject).

Existing users of this plugin from contrib are unharmed. The name is the same, but we haven't changed anything about how it works. Contrib users simply disable migrate_source_csv module before upgrading to core 8.X.Y and none of their migration files have to be changed to continue working.

Data model changes

None.

Original report by @smaz

In #2809635: Create experimental installation profile, we are using CSV files to provide default content for a demo installation profile of Drupal.

In a review, @larowlan suggested bringing the CSV migration sources into core:
https://www.drupal.org/project/drupal/issues/2809635#comment-12381619

I agree that a CSV migrate source would be a very useful feature for core.

The migrate_source_csv module has a stable release (2.0), has been downloaded almost 100k times (not sure if that includes composer stats?), and comes complete with tests. The contenta CMS has been using this for their migrations, too.

So would it be ok to look at moving this into Core's migrate module? If so, I'm happy to try and turn the module into a patch.

Problem/Motivation

Over many years Drupal has held off checking queries for reserved words in column names, table names and aliases because it was a design decision. The idea is that we just just should not use reserved words. However, time has made it clear that this position (as stated in the long discussions in #371: resolve ANSI SQL-92/99/2003 reserved words conflict in query statements) is not really tenable. Here are the problems:

• The list of reserved words is different for different database drivers even those the support the various ANSI SQL standards. For example OFFSET is reserved in PostgreSQL but not in MySQL
• Database drivers add new reserved keywords in new versions. MySQL has been doing this in every new version for example 5.7 and 8

Proposed resolution

• Add a new syntax square bracket syntax to the query builder to know what identifiers (column names/ aliases) are. Leverage work in #371: resolve ANSI SQL-92/99/2003 reserved words conflict in query statements suggested using [COLUMN_NAME] to match {TABLE_NAME}. This supports db_query(), addExpression() and addWhere()
• Automatically quote fields and aliases in escapeField() and escapeAlias().
• All identifiers should be quoted. This helps us ensure that the database API is properly used when query building and will make edge-cases less likely.

Using [] to identify identifiers (field names / aliases)

There is precedence - this is the quote character in MSSQL.

The one consideration is that :variable[] is used to identify and expand array variables.

Discarded resolution(s)

We could deprecate SELECT, UPDATE, INSERT queries via \Drupal\Core\Database\Connection::query() since the methods like \Drupal\Core\Database\Connection::select() require fields to be specified. If we do that we have to worry about performance - see #1067802: Compare db_query() and db_select() performance. Also this does not work for columns added using ->addExpression() or ->addWhere(). Therefore this resolution has been discarded.

Remaining tasks

Fix the blockers:

• #2986334: Add a way to enforce an empty result set for a database condition query
• #2986539: \Drupal\user\Plugin\EntityReferenceSelection\UserSelection::entityQueryAlter() should escape the fake condition column on replacement

File follow-ups:

• add coder rule to try to check for missing []
• Ensure core uses new square bracket syntax

User interface changes

None

API changes

At the moment \Drupal\Core\Database\Connection::escapeField() and \Drupal\Core\Database\Connection::escapeAlias() in their core implementations are re-entrant. And some core code relies on this. That said the docs hint that this should not be case. In the new implementation methods also now quote identifiers BUT because the code removes the quotes before adding them they still are re-entrant. The new implementations result in strings like users_data.uid becoming "users_data"."uid".

\Drupal\Core\Database\Query\ConditionInterface::condition()'s first argument $field must be a field and cannot be (ab)used to do ->condition('1 <> 1'). The way to do this is to do ->addWhere('1 <> 1').

\Drupal\Core\Database\Connection::prepareQuery() now takes the array of query $options to allow databases to use the new allow_square_brackets to not swap square brackets for the identifier quotes.

\Drupal\Core\Database\Connection::prefixTables now quotes tables so {node} becomes "node".

Data model changes

None

Problem/Motivation

The filter Restrict images to this site checks if an image is local by trying to load image dimensions from the local file, if that fails, the image is marked as remote and removed from the markup:

      // Ensure the path refers to an actual image by prefixing the image source
      // with the Drupal root and running getimagesize() on it.
      $local_image_path = $local_dir . Unicode::substr($src, $base_path_length);
      $local_image_path = rawurldecode($local_image_path);
      if (@getimagesize($local_image_path)) {
        // The image has the right path. Erroneous images are dealt with below.
        continue;
      }

That code breaks for private:// files, because the image URL looks something like /system/files/inline-images/search.jpg, but that directory does not exist on disk.

It also breaks when using image style derivatives (particularly an issue when using a module like entity embed). The URL to the image includes a token as a query parameter, and this token is still present when checking for the file on the file system.

Proposed resolution

TBD

Remaining tasks

TBD

User interface changes

None. (Well, working private images.)

API changes

None.

Data model changes

None.

Updated: Comment #212

Problem/Motivation

Inserting an image in the text editor dialog today allows the user to fiddle with image dimensions. It doesn't even have aspect ratio locking.

It's not great for the authoring experience nor for structured content reasons that users are defining the specific dimensions of every single image they insert. It'd be much better to allow them to choose from image styles — just like we do for image fields.

Proposed resolution

Allow an image style to be selected in the image dialog, which gets stored in a data-imagestyle attribute, and is handled by a yet-to-be-added imagestyle filter.

Remaining tasks

1. Initial patch: select image style in dialog, inserting that sets a data- attribute, and a filter transforms the end result.
2. Get #1589176-4: Follow-up: Use data-* to store #states api informations— a blocker to this patch — committed.

User interface changes

• The new ability to select an image style.
• A new filter.

API changes

None.

Related Issues

• #1932652: Add image uploading to WYSIWYGs through editor.module
• #1589176-14: Follow-up: Use data-* to store #states api informations
• #2123251: Improve DX of responsive images; convert theme functions to new #type element

The batch process which is executed from the Migrate UI (/admin/structure/migrate/manage/{migration_group}/migrations/{migration}/execute) causes the PRE_IMPORT and POST_IMPORT events to be triggered once for every batch slice. Those events (described in https://www.drupal.org/node/2544874) should in fact run only once for an entire migration.

If I understand correctly, \Drupal\migrate_tools\MigrateBatchExecutable::calculateBatchLimit causes the migration to be sliced into max. 100 parts. (Introduced in #2470882: Implement running migration processes through the UI) This by itself should not be a problem, but \Drupal\migrate\MigrateExecutable::import() seems to treat each slice as a separate import. When I run my migration from the command line with Drush, this problem does not occur.

I'm not sure if this should be fixed in core or in migrate_tools, but I'm posting it in the migrate_tools queue because this doesn't happen with just Drupal core.

Problem/Motivation

Environment:
Drupal 8.6
Workspaces in core 8.6
Layout Builder in core 8.6

Process:
1. Fresh install of Drupal 8.6
2. Workspaces is enabled with two work spaces Live and Stage as generated out of the box when workspaces was enabled
3. Layout builder and Layout Discovery are also enabled
4. Create content type with layout enabled, and specific enable "Allow each content item to have its layout customized."
5. Create a piece of content using new content type in LIVE workspace, override layout and add blocks, works fine.
6. Change into STAGE workspace, try to override layout on same content item and layout UI will not allow to add blocks or remove components.

Proposed resolution

1. Should be able to override a layout for an individual content item in STAGE workspace.
2. Changes to layout in STAGE should not be visible on LIVE environment
3. Ability to then push those layout changes and any content revisions from STAGE to LIVE.

Remaining tasks

User interface changes

API changes

Data model changes

Problem/Motivation

When accessing Layout Builder overrides via REST (i.e. when accessing an overridden entity's layout field), the contents of sections appear empty.

Proposed resolution

We should add a normalizer for the layout_section DataType, which ensures that sections can be properly serialized and unserialized via the serialization API (and REST).

Remaining tasks

Write a patch, possibly discuss what setting (unserializing) sections looks like via an API like REST.

User interface changes

None.

API changes

A new normalizer will be added for the layout_section DataType.

Data model changes

None.

Problem/Motivation

Currently it's only possible to delete a file entity by the owner of an file entity. There is no additional condition defined for any permission or even an exception for user/1 (main admin user).

Proposed resolution

• Add existing or new permission to delete file entities.
• And maybe define an exception for user/1 to delete file entities like other entities.

Remaining tasks

User interface changes

API changes

Data model changes

Original Issue summary:

I've faced with problem when user with id '1' (main admin user) can't delete file entities created by another user. I used view_bulk_operations for deleting file entities. I thought user 1 has absolute access to all entities.

Problem/Motivation

There are several issues when attempting to use Layout Builder and Workspaces at the same time.

Proposed resolution

In the short term, prevent them from being installed at the same time.

Remaining tasks

User interface changes

API changes

Data model changes

We have an end-2-end test-suite running with Cypress.io (https://www.cypress.io/)
All pages that have tableheader.js included fail. One of those page is /admin/content so obviously this makes it hard to test the Drupal admin backend.

The error is Uncaught TypeError: Cannot read property 'displace' of undefined
on last line in tableheader.js

(function ($, Drupal, displace) {

   ...

})(jQuery, Drupal, window.parent.Drupal.displace);

Actually window.parent is the first undefined object here.

The reason is:
1. Cypress.io is running the test-suite in a browser.
2. It includes the Drupal website inside an iframe and starts interacting with it (that how Cypress.io works).
3. Now window.parent.Drupal.displace tries to interact with 1. the parent of the iframe. But this is forbidden due to XSS-prevention on browser level.

When reading javascript / browser documentation I read that window.parent refers to the enclosing page if the script is running in an iframe. Otherwise window.parent refers to the page itself. See https://developer.mozilla.org/en-US/docs/Web/API/Window/parent

When I replace window.parent.Drupal.displace with window.Drupal.displace my problems are fixed.

I cannot think about a use case where Drupal renders an iframe with a tableheader inside it.
So I wonder why we have window.parent in place now?

$form_state->getValue(), is returing the complete values list which should not happen.

For thisto happen we have $form_state->getValues()

Problem/Motivation

The configuration management system in Drupal 8 is great and allows configuration to be synchronized between installations of the same site in different environments. This is done by exporting the configuration to an intermediary storage (files in git or tarball) transferring the intermediary storage and then importing it on the second installation. The ConfigImporter is takes the storage and checks for differences and updates the database to reflect the new configuration, creating or dropping tables etc.

However, the current workflow presumes that the configuration will be exactly the same in the different environments. If they are not developers have to edit the configuration in the intermediate storage before it is imported again. The state of the art solution in contrib is Config Filter. It works great but has an important drawback as an API for both the DX and API architecture: It allows modifying the sync storage as it is manipulated but it has no concept of the intent. For example config filter plugins do not know if the sync storage is used for importing the configuration from it or exporting the configuration to it. A developer has to know which methods are called in which order of both the import and export process to correctly interact with it.

The concept of altering the configuration and importing it again is also used outside of the configuration synchronization process in contrib with Config Distro. The idea there is that the configuration importer can import the active configuration again after it has been modified by distro filters. This allows configuration to be updated with updated distribution configuration through the same ConfigImporter API but is not limited to distributions as such. Configuration can either be changed manually by calling methods on the configuration entities or it can be changed through importing all of the configuration through the ConfigImporter. The alter allows developers to change "arrays" and assures that all the appropriate side effects happen as they do when configuration is synchronized.

Proposed resolution

Add a config transfomer API that allows event subscribers to edit the configuration just before it is imported or exported, knowing which it is and in which context the action is performed.

Remaining tasks

1. Write patch
2. agree on architecture
3. refine patch
4. review patch
5. commit patch
6. celebrate (and update config_filter / config_split etc to work with new API)

User interface changes

None, the API is transparent for users if there are no systems using it.
New modules and functionality may be added to core on top of it.

API changes

New API to transform (alter) the configuration as it is being imported from or exported to auxiliary storages.

Data model changes

None

Hi,

After migrating (D6 to D8) I get this issue when I want to run cron with drush:

 drush @vmdevd8ce cron
PHP Fatal error:  Unsupported operand types in /var/www/drupal-8/core/lib/Drupal/Core/Utility/LinkGenerator.php on line 153
Drush command terminated abnormally due to an unrecoverable error.                                                                                                    [error]
Error: Unsupported operand types in /var/www/drupal-8/core/lib/Drupal/Core/Utility/LinkGenerator.php, line 153

            <div class="field field--name-field-lien-interne field--type-link field--label-hidden field__item">

via the UI I get a WSOD.

I don't know what I can check or set to avoid this error.
Possibly related to my uncomplete block migration ?

                                            Etat  Total  Imported  Unprocessed
 d6_block                                          Idle  415    27        0 

Thanks

Problem/Motivation

drupalPostAjaxForm() is simulating the behaviour of ajax.js, so using it, doesn't really provide fundamental guarantees.
#2809161: Convert Javascript/AJAX testing to use JavascriptTestBase suggests to convert them to JavascriptTestBase

Proposed resolution

1. Figure out which part of the test is testing PHP code and which part ajax behaviour
2. Extract the ajax behaviour into a test that extends JavascriptTestBase

Remaining tasks

User interface changes

API changes

Data model changes

It looks like Drupal 8.4 brought an update to CKEditor from 4.6 to 4.7 where essentially CKEditor ended functionality for the paste buttons, seemingly because they were a security issue. Clicking on any of the paste buttons (paste, paste as plain text, paste from Word) gets you a popup that reads some variation of, "Your browser doesn't allow you to paste this way. Press ⌘+V to paste." We're a Mac shop, hence the Command-V message. However the functionality when pasting from Google Docs is very different with ⌘+V, in that all of the text being pasted gets marked up in bold via many, many, maaaaannny STRONG tags. The only option is to delete all of the STRONG tags and then go back and re-bold anything that actually was bold beforehand.

This seems to be a CKEditor issue and in part a Google Docs issue, but from the perspective of my content editors -- who all use Google Docs and paste text from there -- a Drupal upgrade broke the site. In a way it did. They had a WYSIWYG approach that worked for 15 months and then they lost functionality after I updated to 8.4. It's also not the only thing that broke when upgrading from 8.3 to 8.4 so for me this is part of a broader issue where there is a fragility that is concerning for minor upgrades that will happen twice a year. Every minor update seems to mean a lot of backpedaling to regain functionality or work around newly introduced issues. If we do decide to go with the latest versions of other communities software, like CKEditor, should we not be sure they are still working well?

In any event, that is neither here nor there, but I thought I'd share that discussions in CKEditor issues imply that 4.6 still works even though the 4.7 error message implies that the browsers block this... which clearly they are not if 4.6 still works. While backtracking a version of CKEditor was a fine solution for others it does not seem to be recommended for Drupal 8 in the era of Composer. Drupal people on those forums seemed frustrated. Is there a recommended approach for going back to "works better for us" versions of other software like CKEditor bundled in Drupal 8?

And for anyone else with the same issue I thought I'd share that our two temporary workarounds. The first is our backup plan and involves pasting into Word and then pasting from Word into CKEditor. It messes up headings and one or two other things, but keeps most of the more tedious to replace stuff. The preferred approach is we added gd2md-html add-on from Google Docs. There are always security concerns with 3rd-party add-ons but this one seems okay and this risk is mitigated by the fact that the vulnerability is only on documents where you use this tool and documents we use this for are going to be public anyway.

When you add a new field to a content type and export configuration (via something like drush cex), the configuration for the teaser display mode is typically lacking a reference to the field you just created. The configuration for the teaser display mode will only get updated later once the display mode is actually used.

Steps to reproduce:

1. Start with a fresh install of Drupal (I'm using Lightning). Run drush cex to capture a baseline configuration.
2. Create a new content type foo and add a field to that content type (I think any field will do, but I used an entity reference)
3. Run drush cex to export the new content type, field, and display modes.
4. Open the config file core.entity_view_display.node.foo.teaser

Expected result:
The config file for the teaser display mode contains the field you just created under the "hidden" key.

Actual result:
The config file for the teaser display mode contains no reference to any field other than the default body field.

If you then go edit and save the teaser display mode in the UI, OR re-install the site from scratch using drush si --config-dir, and then re-export configuration, you'll note that the teaser display mode now properly contains the new field.

This may sound like a minor problem, but it actually wreaks havoc with a good configuration management / CI system, because a developer will typically add fields to a content type and capture the changes in config, thinking everything is fine, and then only much later in seemingly unrelated operations will the changes to the display mode suddenly come through. It also causes problems if you are attempting to verify configuration integrity (the way that BLT does), because this configuration will randomly show up later on and cause configuration integrity validation to fail.

The code comment prior to protected $rebuild = FALSE; contains the following text:

   * confirmation forms. Normally, self::$rebuild is set by a submit handler,
   * since its is usually logic within a submit handler that determines whether
   * a form is done or requires another step. However, a validation handler may

On the second line, please replace 'its' with 'it'.

I recently tried to update our website from Drupal 8.5.5 to Drupal 8.6.1. I keep getting multiple errors. The first two I found tickets for. This third one though, I am finding no information documented on this.

When I run `drush updb -y` I get the following errors at the end:

[error] No entity type for field name_1 on view facets
[error] Update failed: taxonomy_post_update_handle_publishing_status_addition_in_views
[error] Update aborted by: taxonomy_post_update_handle_publishing_status_addition_in_views
[error] Finished performing updates.

No error messages are appearing in the logs.

I do see the following at the top of the updb command:

---------- -------------------------------------------- ------------- ----------------------------------------------------------------------------------------------
Module Update ID Type Description
---------- -------------------------------------------- ------------- ----------------------------------------------------------------------------------------------
taxonomy handle_publishing_status_addition_in_views post-update Add a 'published' = TRUE filter for all Taxonomy term views and converts existing ones that
were using the 'content_translation_status' field.
---------- -------------------------------------------- ------------- ----------------------------------------------------------------------------------------------

I am unsure what is going on. There are no errors in the logs about this. Clearly the taxonomy module has some upgrades that are not quite ready to go live, given all the bugs still being found. I am sure these will be fixed soon though. I will see what I can do to fix it on my end.

The first bug fixes I found were made in these tickets:
https://www.drupal.org/project/drupal/issues/2997982
https://www.drupal.org/project/drupal/issues/2997960