Blocked by: #2853509: Don't render status messages if there are no messages but also include their assets if there might be

Problem/Motivation

Provide a consistent method for displaying messages in JavaScript.

Currently the Settings tray module needs to provide various message via JavaScript in:
#2773601: Display "You are now in edit mode" prompt when user enters edit mode
#2785047: In Outside In mode, form validation messages should appear in the off-canvas tray, not the main page

And also Inline Form Errors needs to be able to update messages through JavaScript:
#2876321: Update inline form errors summary after form validation with AJAX

Proposed resolution

Change div for {{ messages }} to always be rendered and add a data-drupal-messages attribute for JavaScript to select.

Add new methods for:

• Adding a message
• Removing one or multiple messages
• Clearing all messages of a specific type
• Clearing all messages

How to test manually

1. Apply the latest patch
2. Install standard
3. Install the js_message_test module. You need to ensure $settings['extension_discovery_scan_tests'] = TRUE; is in your settings.php in order to enable it.
4. Visit the url /js_message_test_link on your site.

Remaining tasks

Review patch
Write change record

API changes

Persistent div for {{ messages }} would be required in themes.

API Additions

const messageInterface = new Drupal.Message();

const messageId = messageInterface.add(message, options);
const messageList = messageInterface.select(type);

messageInterface.remove(messageId);

messageInterface.clear();

Original report by kkaefer

While developing some JavaScript enhancements for Drupal, I am facing the problem of how to indicate the status of an action properly. For example, when editing a label, the user needs to know if the action has been successful (saving the new title) or if it failed. Of course, some widgets can use their own status indicating system (like the autocomplete textfield), but others have to display text.

But – we already have status messages in Drupal! Take a look at theme_status_message(). We just need to make sure that the status message div is always there so that we can inject new items via JavaScript... How about that? Post your opinion or your own suggestion for this issue.

Problem/Motivation

After lots and lots of debugging, I discovered in #2869426-39: EntityResource should add _entity_access requirement to REST routes why _entity_create_access just wouldn't work for the REST module: because the REST module does not limit its routes that create entities to a single bundle. And some entity types check create access based on the bundle!

This is ONLY a DX problem. It does NOT have security implications.

Proposed resolution

Rather than doing

       // If we were unable to replace all placeholders, deny access.
       if (strpos($bundle, '{') !== FALSE) {
         return AccessResult::neutral();
       }

which is incredibly unhelpful, specify a reason!

       // If we were unable to replace all placeholders, deny access.
       if (strpos($bundle, '{') !== FALSE) {
         return AccessResult::neutral(sprintf("Could not find '%s' request argument, therefore cannot check create access.", $bundle));
       }

No more 🤔😡🙀!

Remaining tasks

None.

User interface changes

Helpful reason.

API changes

None.

Data model changes

None.

Problem/Motivation

CckFile is deprecated in Drupal 8.3.x and will be be removed before Drupal 9.0.x.

Proposed resolution

Use \Drupal\file\Plugin\migrate\process\d6\FieldFile instead.

Remaining tasks

User interface changes

API changes

Data model changes

Problem/Motivation

This issue was found while working on #2824935: [PP-1] Fix Squiz.ControlStructures.SwitchDeclaration coding standard.

Drupal\field\Plugin\migrate\process\d7\FieldInstanceDefaults::transform() contains a switch statement with only a default statement. This violates the Squiz.ControlStructures.SwitchDeclaration coding standard: SWITCH statements must contain at least one CASE statement.

This issue is fixed separately, since it allows for discussion regarding the migration system rather than coding standards.

Proposed resolution

Refactor the code to remove the switch statement.

Remaining tasks

1. Write a patch
2. Review
3. Commit

User interface changes

None.

API changes

None.

Data model changes

None.

Problem/Motivation

The settings tray currently uses the dialog system and is always rendered on the right side of the page. Some modules may have a need for the tray to appear at the top of the page.

Proposed resolution

Add a new value off_canvas_topfor the data-dialog-renderer attribute when creating a dialog link.
This will make the off-canvas dialog to render at the top of the page.

Remaining tasks

Fix the bug described in #54, which this patch introduces.

User interface changes

Module will be able to create off-canvas dialog links where the dialog comes down from the top of the page instead of the side.

API changes

There will be a new value off_canvas_topavailable for the data-dialog-renderer attribute when creating a dialog link.

To create a off-canvas-top dialog link us the following

'off_canvas_top_link_1' => [
        '#title' => 'Open top panel 1',
        '#type' => 'link',
        '#url' => Url::fromRoute('off_canvas_test.thing1'),
        '#attributes' => [
          'class' => ['use-ajax'],
          'data-dialog-type' => 'dialog',
          'data-dialog-renderer' => 'off_canvas_top',
        ],
      ],

Note this the same as the current off-canvas dialog except for the off_canvas_top value.

Data model changes

None

Manually testing

To manually test this patch

1. Enabled the off_canvas_test test module.
2. Goto /off-canvas-test-links
3. Click the various links to open the dialog either on the side or on the top.

To reproduce the error mentioned in #16 first open a top dialog and then a side one. The side dialog will be pushed down on the page.

When I try to save an entity after denormalization I get this error: Call to a member function getProcessedText() on string in core/modules/text/src/TextProcessed.php line 43. This happens because denormalized entity has the ''processed" property as a string eg. <p>Lorem Ipsum.</p>, not as FilterProcessResult object.

The issue seems to be a followup for #2626924: Include processed text in normalizations: "text" field type's "processed" computed property should be non-internal and carry cacheability metadata.

I've found this issue while working on making Relaxed module compatible with Drupal 8.5.x and this is the last blocker.

Attached a failing test and also a fix.

    $get_migration = $this->prophesize(MigrationLookup::class);

...
    $process_plugin_manager->createInstance('get', ['source' => ['string_id', 'integer_id']], $migration_plugin->reveal())
      ->willReturn($get_migration->reveal());

We're calling createInstance for the 'get' plugin. So the return should be an instance of Drupal\migrate\Plugin\migrate\process\Get, not Drupal\migrate\Plugin\migrate\process\MigrationLookup.

Problem/Motivation

MigrateCckFieldInterface is deprecated in Drupal 8.3.x and will be removed before Drupal 9.0.x.

Proposed resolution

Use \Drupal\migrate_drupal\Annotation\MigrateField instead.

Remaining tasks

User interface changes

API changes

Data model changes

Problem/Motivation

Drupal\system\Tests\Update\DbUpdatesTrait is deprecated in Drupal 8.4.0 and will be removed before Drupal 9.0.0.

Proposed resolution

Use \Drupal\FunctionalTests\Update\DbUpdatesTrait instead of Drupal\system\Tests\Update\DbUpdatesTrait.

See https://www.drupal.org/node/2896640.

Remaining tasks

User interface changes

API changes

Data model changes

Original report by [username]

Problem/Motivation

Discovered in #2869426.

See #2869426-28: EntityResource should add _entity_access requirement to REST routes, #2869426-29: EntityResource should add _entity_access requirement to REST routes and #2869426: EntityResource should add _entity_access requirement to REST routes in particular.

Proposed resolution

TBD

Remaining tasks

Investigate & discuss.

User interface changes

None.

API changes

None.

Data model changes

Yes: Dynamic Page Cache would vary not just by the Response's cacheability, but also by the route access result's cacheability.

The migration lookup plugin can be used in two ways:

  uid:
    plugin: migration_lookup
    migration: users
    source: author

There's a single source key, and its value is taken from the 'author' value in the row.

  uid:
    plugin: migration_lookup
      migration:
        - users
        - members
      source_ids:
        users:
          - author
        members:
          - id

The plugin will try multiple migrations in turn, and for each migration to lookup, the source is specified for each one.

In both cases, this is a single source key.

However, migrations often have more than one source key.

Because of how lookupDestinationIds() works, the key used by the lookup plugin is assumed to be the first one:

   * @param array $source_id_values
   *   The source identifier keyed values of the records, e.g. ['nid' => 5].
   *   If unkeyed, the first count($source_id_values) keys will be assumed.

What's missing is:

1. The ability to specify multiple source keys
2. The ability to specify specific source keys. E.g. the migration to use as a lookup might have keys alpha, beta, gamma, and in migration needs to look up using the beta key.

Problem/Motivation

It appears that the changes made in #2940513: ModerationStateFieldItemList fails after an entity has been serialized/unserialized. may have introduced an issue with entity serialization. If a node is serialized prior to EntityOperations::entityUpdate being called, then the moderation_state value is reset to the workflow's default state. This can happen, for instance, if a node is added to a queue item.

Proposed resolution

Retain the actual moderation state after serialization.

Remaining tasks

User interface changes

API changes

Data model changes

Problem/Motivation

- Container compilation is one of the slowest parts in cold cache startup. (now that YAML parsing is cached in FileCache)
- Container compilation is one of the most slowest parts during installation / uninstallation of modules

In the test suite this compilation happens again and again.

The major part in the compilation process taking long is Symfony's dependency checking and graph operations.

However:

- Caching the container across test configurations is very hard as there is many variations and modules are allowed to change everything.

Therefore we have:

- Poor cache granularity (hash is changing too much)
- BC problems as caching the container across tests could not take into account undeclared dependencies that modules have, so caching e.g. just on the module list is not feasible as there can be any outside parameters.

Proposed resolution

While I tried to cache the hash of the full container definition minus the parameters based on all parsed yml files (which worked great and had not too many variations), there is an even better way:

As seen in the motivation section, the slowest part are the dependency checks.

Those are defined here:

https://github.com/symfony/dependency-injection/blob/master/Compiler/Pas...

However, all this services do not depend on anything external and do also not depend on any parameters (except for the ResolveParameterPlaceHoldersPass, which can be removed as the used container supports run-time parameter resolving)

That means:

Whatever input those passes get in the form of the current ContainerBuilder(), the output will always be the same.

That however means this is cacheable based on the input and fortunately we already can transform a container builder into a container definition array using the PhpArrayDumper() (which is also really fast (8 ms)).

The idea is now to replace all those services with a single service _if_ and only if no one added a compiler pass before or after this - unless it is whitelisted using a special interface.

And doing that we can skip the most expensive compilation steps.

The new pass that replaces those passes, would then):

- Compile to a definition
- Store and remove the parameters
- Create a CID from the hash of the definition
- Put that in drupal.container_definition_cid

Cache Hit case:

- Retrieve the cached definition as the final output result
- Store a parameter: drupal.cached_container_definition (and re-add the original parameters to it).
- Return

Drupal would check the drupal.cached_container_definition parameter and use that instead of dumping the container builder.

Cache miss case:

- Just run the container things
- Create a dump like usual
- Check presence of drupal.container_definition_cid
- Store the result of the definition

Storage would just be in APCu or FileCache with a custom class that does only use the backends, but not the mtime part.

Remaining tasks

- Do it!

User interface changes

API changes

Data model changes

Problem/Motivation

The test of the Get migrate plugin uses a class that extends Get, rather than the plugin itself. As a result, the actual Get plugin is not tested and so a syntax 'error' in the annotation has gone untested and unnoticed (until #2971338: MigrationLookupTest::testMultipleSourceIds() uses wrong class for mocking.

Proposed resolution

Replace the TestGet plugin with the use of the actual Get plugin. Then resolve the syntax error.

Remaining tasks

Replace TestGet plugin.
Remove TestGet plugin.
Resolve syntax error.

User interface changes

API changes

Data model changes

I have been scratching my head over this issue. The source sometimes has a blank offers, sometimes it is an array with data and sometimes it is an empty array - like this one:
https://api.hel.fi/linkedevents/v1/event/matko:16134/

With the latter example I was getting a bunch of Array index missing, extraction failed exceptions from the extract plugin, despite the skip_on_empty plugin running before, as you can see from my yml file:

  field_offers_is_free:
    -
      plugin: skip_on_empty
      method: process
      source: offers
    -
      plugin: extract
      index:
        - 0
        - is_free
  field_offers_info_url:
    -
      plugin: skip_on_empty
      method: process
      source: offers
    -
      plugin: extract
      default: false
      index:
        - 0
        - info_url
        - fi

I managed - in the end - to track down the reason for why our skip_on_empty plugin was never called: When Get sets $multiple to true, processRow in MigrateExecutable runs a foreach on the value - causing the whole skip_on_empty plugin to be skipped.

With this quick fix in modules/migrate/src/Plugin/migrate/process/Get.php's transform function things work fine for me:

    if (is_string($source)) {
      $a = is_array($return[0]);
      $b = !empty($return[0]);
      $this->multiple = $a && $b;
      return $return[0];
    }

Did I encounter a bug in Migrate or should I change my migration configuration?

Fixed pagination and sorting of tables on pages with ajax

Problem/Motivation

The core migrate system needs a way to import things from CSV files.

Immediate use case is for the Umami Demo profile now in core in 8.6.x. The imported content is in CSV files and we have code to read it and create the relevant entries. Having migrate_source_csv in core would allow us to remove that code and use migrate instead.

See #2809635-37: Create experimental installation profile for where @larowlan suggested bring migrate_source_csv into core.

Other good reasons to have CSV as a migration source in core:

• "I use this constantly. I've seen it forked into the Contenta distro, and a few of my own projects (I just want the source plugin, not another module.) It'll also be vital to Commerce integrations as CSV is a common way to import data from an ERP." @mglaman (#4)
• "CSV format is the first format used by non technical professionnals who need to import data and to generate/edit those datas thanks to software they knew (MS Excel without mention it). It's also the easiest way to export data from many ERP and big solutions with datas." @GoZ (#5)
• "CSV is simply a really common used format and it would make migrate more usable by default for many usecases. One could say: Oh it is easy to parse CSV, but it turns out it is not. On top of that the module also doesn't load the entire CSV file upfront, but rather reads line by line. I think this makes it a good candidate also for bigger migrations out there." @dawehner (#14)
• "This would help the Out of the Box Initiative. Let's do it! Consider this my maintainer +1 :)" @phenaproxima (#18)

Proposed resolution

The migrate_source_csv module has a stable release (2.0), has been downloaded almost 100k times, and comes complete with tests. The contenta CMS has been using this for their migrations, too.

Rename the files/namespaces, and move it in, as-is, to the core migrate module.

Since this is a straight port from contrib, discussions about fetchers, etc. are postponed to follow-ups. See #16, #36 and #37 for some of the highlights of the discussion.

Follow-up: #2962091: Adopt fetchers/parsers logic for use by source csv plugin

Remaining tasks

1. Patch that renames files/namespaces and moves the contrib code into core Done.
2. Tests Done.
3. Word-smithing PHPDocs Done.
4. Decide if we're okay with a straight port (done), or if we need to hash out all the fetcher/parser stuff first, figure out the design, class names, avoid conflicts, possible BC hell, etc. Done. See comments #80 through #83. Punted to #2962091: Adopt fetchers/parsers logic for use by source csv plugin.

None.

User interface changes

None.

API changes

Adds a new migrate source plugin called "csv" in migration yml definition files. The class providing the plugin (\Drupal\migrate\Plugin\migrate\source\CSV) is heavily commented with PHPDoc.

Adds a \Drupal\migrate\CSVFileInterface interface that classes used for the optional "file_class" configuration key must implement.

Adds a \Drupal\migrate\CSVFileObject class (which extends \SplFileObject) that is the default class used if "file_class" is not overridden in your migration.

Existing users of this plugin from contrib are unharmed. The name is the same, but we haven't changed anything about how it works. So, contrib users simply disable migrate_source_csv module before upgrading to core 8.X.Y and none of their migration files have to be changed to continue working.

Data model changes

None.

Original report by @smaz

In #2809635: Create experimental installation profile, we are using CSV files to provide default content for a demo installation profile of Drupal.

In a review, @larowlan suggested bringing the CSV migration sources into core:
https://www.drupal.org/project/drupal/issues/2809635#comment-12381619

I agree that a CSV migrate source would be a very useful feature for core.

The migrate_source_csv module has a stable release (2.0), has been downloaded almost 100k times (not sure if that includes composer stats?), and comes complete with tests. The contenta CMS has been using this for their migrations, too.

So would it be ok to look at moving this into Core's migrate module? If so, I'm happy to try and turn the module into a patch.

After testing the jsonapi I came across this discovery.
I am able to create a comment of a comment (of entity_type "node") of a node. However, I can't go the next level which would be a comment of a comment (entity_type "comment") of a comment of a node.
I can do it from the drupal ui directly. I don't know if this is to protect against circular references or what, but this needs to be looked into.

Steps to reproduce.
Create 2 more comment types.
CommentOfNodeComment
CommentOfCommentComment

Then for Default comments add field General Comments CommentOfNodeComment
Then for CommentOfNodeComment add field General Comments CommentOfCommentComment

Then create an article w/ comments down to CommentOfCommentComment.

Then try to create a CommentOfCommentComment via a rest call. E.g.

POST {{protocol}}://{{host}}/jsonapi/comment/responses HTTP/1.1
Authorization: Basic token
Accept: application/vnd.api+json
Content-Type: application/vnd.api+json

{
  "data": {
    "type": "comment--commentofcommentcomment",
    "attributes": {
      "subject": "test CommentOfCommentComment",
      "entity_type": "comment",
      "field_name": "field_commentofcommentcomment",
      "comment_body": {
          "value": "some text",
          "format": "basic_html"
        },
      "entity_id": [{"target_id": 4}],
      "uid": [{"target_id": 1}]
    }
  }
}

I've decided to start taking a more active role in core Theme API development considering that we've never really gotten too far past all the Twig conversions since 8.x was initially released.