Problem/Motivation

Split from #2488836: Refactor prepareRow() to add events in addition to hook.

Migrate Plus module is dispatching this event and this is critical to migrate runner because it allows the Drush option --idlist. But as the migrate runner goes to Drush core, and the two should run independently but also together, it's better to have the event dispatching in Drupal core.

There are also the reasons from #2488836: Refactor prepareRow() to add events in addition to hook.

Proposed resolution

Add the event constant, the event class & dispatch the PREPARE_ROW event. Deprecate hook_migrate_prepare_row_alter() and hook_migrate_MIGRATION_ID_prepare_row_alter() hooks.

Remaining tasks

None.

User interface changes

None.

API changes

None.

Data model changes

None.

Problem/Motivation

With Drupal 8.5.x - and probably 8.4.x - Entity Reference fields do not render views.

This affects both the exiting ID EntityReferenceIdFormatterand Label Core formatters as well as custom formatters that rely on EntityReferenceFormatterBase. EntityReferenceEntityFormatter doesn't apply to views so it is unaffected in this case.

The reason they don't render is that the $entity->access() returns false (to be exact an AccessResult that evaluates to false eventually) in EntityReferenceFormatterBase::getEntitiesToView(). This is because, unlike most entities, views don't use entity based permissions but rather individual access controls per display per view. The per display side of things doesn't matter much for ID and Label formatters.

Proposed resolution

Basically 5 possible resolutions:

1. Create view specific formatters with different access checks. Would work but would require database updates and still could cause problems.
2. Add a views specific access handler equivalent to EntityAccessControlHandler
3. Adjust EntityReferenceFormatterBase::checkAccess() - something like inserting the following at the start of the function.
   

   if (method_exists($entity, 'getExecutable')) {
         if ($entity->getExecutable()->access('default')) {
           return AccessResult::allowed();
         }
         return AccessResult::forbidden();
       }

4. Implement a hook_view_access function.
5. Add an access function on /Drupal/views/Entity/View

Remaining tasks

Choose option, possibly write code for option - some code included here and patch attached for option #5. Write Tests.

User interface changes

None

API changes

None

Data model changes

None unless solution #1 was chosen.

Problem/Motivation

Parsing .info files for dependency information is already implemented on the modules administration page. Adding support for the same dependencies in theme.info files, and implementing the same behavior on the admin/build/themes page, would allow designers building heavily customized themes to make safer assumptions about their target sites.

A theme might require the existence of imagecache to auto-generate variations of a header image. This would be a nice compromise between systems like Wordpress and Joomla!, which give themes much greater control over the functionality of the site, and Drupal's module-centric approach.

It also creates following new UX requirements and non UI, API requirements

1. Represent the list of dependent modules in themes list page. Display of missing/disabled dependent modules.
2. Extension in Drush to download and enable modules dependent on themse.

Proposed resolution

• Allow themes to add the dependencies to .info.yml files
• Show these dependencies on /admin/appearance and make it impossible to install without the requirements
• Install dependencies automatically on API level.

Original report by [eaton]

Issue Summary
Parsing .info files for dependency information is already implemented on the modules administration page. Adding support for the same dependencies in theme.info files, and implementing the same behavior on the admin/build/themes page, would allow designers building heavily customized themes to make safer assumptions about their target sites.

A theme might require the existence of imagecache to auto-generate variations of a header image. This would be a nice compromise between systems like Wordpress and Joomla!, which give themes much greater control over the functionality of the site, and Drupal's module-centric approach.

Follow-up to #2650386: Comments fragment identifier has disappeared / does not work with multiple comment fields

Problem/Motivation

When Comment and History modules are enabled, there is a "New" link shown to authenticated users if new comments has been added to a node since their last visit. This link leads to the canonical URL of the content with the "new" fragment. The thing is that if there is more than one comment field on the content, the "new" fragment can be misleading.

Proposed resolution

Use the field name in the anchor so each link can lead to its own last unread comment.

Remaining tasks

Discuss
Patch
Review
Commit

User interface changes

None yet.
(How could a few "New" links be useful to the user? We might want to add some labels in the links titles to differentiate fields.)

API changes

None.

Data model changes

None.

Problem/Motivation

The core migrate system needs a way to import things from CSV files.

Immediate use case is for the Umami Demo profile now in core in 8.6.x. The imported content is in CSV files and we have code to read it and create the relevant entries. Having migrate_source_csv in core would allow us to remove that code and use migrate instead.

See #2809635-37: Create experimental installation profile for where @larowlan suggested bring migrate_source_csv into core.

Other good reasons to have CSV as a migration source in core:

• "I use this constantly. I've seen it forked into the Contenta distro, and a few of my own projects (I just want the source plugin, not another module.) It'll also be vital to Commerce integrations as CSV is a common way to import data from an ERP." @mglaman (#4)
• "CSV format is the first format used by non technical professionnals who need to import data and to generate/edit those datas thanks to software they knew (MS Excel without mention it). It's also the easiest way to export data from many ERP and big solutions with datas." @GoZ (#5)
• "CSV is simply a really common used format and it would make migrate more usable by default for many usecases. One could say: Oh it is easy to parse CSV, but it turns out it is not. On top of that the module also doesn't load the entire CSV file upfront, but rather reads line by line. I think this makes it a good candidate also for bigger migrations out there." @dawehner (#14)
• "This would help the Out of the Box Initiative. Let's do it! Consider this my maintainer +1 :)" @phenaproxima (#18)

Proposed resolution

The migrate_source_csv module has a stable release (2.0), has been downloaded almost 100k times, and comes complete with tests. The contenta CMS has been using this for their migrations, too.

Rename the files/namespaces, and move it in, as-is, to the core migrate module.

Since this is a straight port from contrib, discussions about fetchers, etc. are postponed to follow-ups. See #16, #36 and #37 for some of the highlights of the discussion.

Follow-up: #2962091: Adopt fetchers/parsers logic for use by source csv plugin

Remaining tasks

1. Patch that renames files/namespaces and moves the contrib code into core Done.
2. Tests Done.
3. Word-smithing PHPDocs Done.
4. Decide if we're okay with a straight port (done), or if we need to hash out all the fetcher/parser stuff first, figure out the design, class names, avoid conflicts, possible BC hell, etc. Done. See comments #80 through #83. Punted to #2962091: Adopt fetchers/parsers logic for use by source csv plugin.

None.

User interface changes

None.

API changes

Adds a new migrate source plugin called "csv" in migration yml definition files. The class providing the plugin (\Drupal\migrate\Plugin\migrate\source\CSV) is heavily commented with PHPDoc.

Adds a \Drupal\migrate\CSVFileInterface interface that classes used for the optional "file_class" configuration key must implement.

Adds a \Drupal\migrate\CSVFileObject class (which extends \SplFileObject) that is the default class used if "file_class" is not overridden in your migration.

Existing users of this plugin from contrib are unharmed. The name is the same, but we haven't changed anything about how it works. So, contrib users simply disable migrate_source_csv module before upgrading to core 8.X.Y and none of their migration files have to be changed to continue working.

Data model changes

None.

Original report by @smaz

In #2809635: Create experimental installation profile, we are using CSV files to provide default content for a demo installation profile of Drupal.

In a review, @larowlan suggested bringing the CSV migration sources into core:
https://www.drupal.org/project/drupal/issues/2809635#comment-12381619

I agree that a CSV migrate source would be a very useful feature for core.

The migrate_source_csv module has a stable release (2.0), has been downloaded almost 100k times (not sure if that includes composer stats?), and comes complete with tests. The contenta CMS has been using this for their migrations, too.

So would it be ok to look at moving this into Core's migrate module? If so, I'm happy to try and turn the module into a patch.

My module does some tasks via Batch API. To avoid duplicating code, I want to do those same tasks during a cron run by programmatically starting and executing the batch without trying to redirect the user to the standard batch job progress bar thingie. There seems to be the facility for this by setting the 'progressive' value of the $process_info array to FALSE, but by the same token, there doesn't seem to be a way to do that - it's hard-coded to TRUE . From D6's form.inc (D7's version is the same in this regard):

function batch_process($redirect = NULL, $url = NULL) {
  $batch =& batch_get();

  if (isset($batch)) {
    // Add process information
    $url = isset($url) ? $url : 'batch';
    $process_info = array(
      'current_set' => 0,
      'progressive' => TRUE, // <= HERE
      'url' => isset($url) ? $url : 'batch',
      'source_page' => $_GET['q'],
      'redirect' => $redirect,
    );
    $batch += $process_info;

    if ($batch['progressive']) {
      // Clear the way for the drupal_goto redirection to the batch processing
      // page, by saving and unsetting the 'destination' if any, on both places
      // drupal_goto looks for it.
      // […snip…]
      drupal_goto($batch['url'], 'op=start&id='. $batch['id']);
    }
    else {
      // Non-progressive execution: bypass the whole progressbar workflow
      // and execute the batch in one pass.
      require_once './includes/batch.inc';
      _batch_process();
    }
  }
}

Unless I'm missing something, this means that, though the functionality exists to do what I want to do, it's not actually possible to do so without hacking core - the code in that "else" branch will never, ever execute. One dead kitten later, I get the expected result.

However, I will allow that I'm just misunderstanding how to use an esoteric feature of this esoteric API and that doing what I need to do is possible without slaughtering kittens; in which case, a bit of edumacation would be appreciated.

If I use Entity argument validator with custom entity without bundles I got the following warning:

Warning: count(): Parameter must be an array or an object that implements Countable in Drupal\views\Plugin\views\argument_validator\Entity->validateEntity() (line 203 of core/modules/views/src/Plugin/views/argument_validator/Entity.php).

Drupal\views\Plugin\views\argument_validator\Entity->validateEntity(Object) (Line: 180)
Drupal\views\Plugin\views\argument_validator\Entity->validateArgument('22522') (Line: 999)
Drupal\views\Plugin\views\argument\ArgumentPluginBase->validateArgument('22522') (Line: 1034)
Drupal\views\Plugin\views\argument\ArgumentPluginBase->setArgument('22522') (Line: 1100)
Drupal\views\ViewExecutable->_buildArguments() (Line: 1264)
Drupal\views\ViewExecutable->build() (Line: 390)
Drupal\views\Plugin\views\display\PathPluginBase->execute() (Line: 180)
Drupal\views\Plugin\views\display\Page->execute() (Line: 1627)
Drupal\views\ViewExecutable->executeDisplay('page_users', Array) (Line: 77)
Drupal\views\Element\View::preRenderViewElement(Array)
call_user_func(Array, Array) (Line: 378)
Drupal\Core\Render\Renderer->doRender(Array, ) (Line: 195)
Drupal\Core\Render\Renderer->render(Array, ) (Line: 226)
Drupal\Core\Render\MainContent\HtmlRenderer->Drupal\Core\Render\MainContent\{closure}() (Line: 582)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 227)
Drupal\Core\Render\MainContent\HtmlRenderer->prepare(Array, Object, Object) (Line: 117)
Drupal\Core\Render\MainContent\HtmlRenderer->renderResponse(Array, Object, Object) (Line: 90)
Drupal\Core\EventSubscriber\MainContentViewSubscriber->onViewRenderArray(Object, 'kernel.view', Object)
call_user_func(Array, Object, 'kernel.view', Object) (Line: 111)
Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch('kernel.view', Object) (Line: 156)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 68)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 57)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 99)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 78)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 50)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 23)
Stack\StackedHttpKernel->handle(Object, 1, 1) (Line: 664)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

The user picture is provided by a field called user_picture.

If you perform a minimal installation of Drupal 8, this field is not created, so you have no user pictures what so ever. The "User pictures in posts" and "User pictures in comments" options in the theme settings are disabled as well.

If you create a new image field, there is no way to make this field the "user pricture" field. And due to the special naming of the "user_picture" field (instead of eg "field_user_picture") there seems to be no way to achieve this through the interface.

I think the default handling of multiple comment fields per content type could be improved:

Currently:

• Only one "Add comment" link is displayed, no matter how many comment fields are added.
• The label of this link doesn’t refer to the name of the comment field.
• If a comment field (that doesn’t happen to be the lucky one getting a link) has deactivated "Show reply form on the same page as comments", there doesn’t seem to be a way to add a comment of this type.

Proposal:

Display a link for each comment field, and use the comment field name in the label.

Problem/Motivation

I've been tracking down a bug that happen on a custom page that set some session variables but ended up being cached.

The result was that later users then mananged to get to that page (it's part of a multi-step process) without the session being initialized, leading to errors on the next page.

Pretty sure this is the same in 7.x.

Proposed resolution

Do not treat a page with session data as cacheable?

Remaining tasks

User interface changes

API changes

Data model changes

Right now it is hard to get a real validation error but with #2948064: Layout Builder should support inline creation of Custom Blocks there is more chance to get validation error.

Add all the references to the Change record: [#2831499]
For more information about the original change see: https://www.drupal.org/project/drupal/issues/2248983

If you have code like this:

drupal_set_message(t('My name is !name.', ['!name' => 'David']));

Drupal will not perform the replacement, and instead will output "My name is !name" to the screen. It will also trigger an error to notify developers about the problem.

However, if you have code like this:

drupal_set_message(t('My name is NAME.', ['NAME' => 'David']));

Drupal will still not perform the replacement (it will output "My name is NAME"). But instead of triggering an error, it will trigger an E_USER_DEPRECATED.

I don't think that makes sense. If it's broken code (not merely deprecated code) it should be an error just like the first case.

This issue is relatively minor, because it turns out Drupal logs/displays E_USER_DEPRECATED as if it were an error or a warning (so even if your development site is only configured to display errors/warnings to the screen, you'll probably still see it). That's presumably a bug though, and we shouldn't rely on it.

Also, my main interest in this issue is that it seems to be what the documentation in https://www.drupal.org/core/deprecation#how-unintended is based on. I think that documentation should be fixed as well as a followup to this issue - E_USER_DEPRECATED should only be used if something is deprecated but still works.

Problem/Motivation

#2831944: Implement media source plugin for remote video via oEmbed implements API-level support for remote YouTube and Vimeo videos via oEmbed, but there is no configuration for this out of the box in Standard. Users will have to create a media type if they want to use YouTube and Vimeo videos in their Drupal site.

Proposed resolution

Ship a new media type in Standard which supports YouTube and Vimeo videos.

Remaining tasks

Create a patch, iterate on it, bikeshed the naming for a while, then review and commit.

User interface changes

None, really. A new media type will be shipped with Standard, but I'm not sure if that counts as a UI change.

API changes

None.

Data model changes

None.

Hitting an average of 20 nodes (http full pages load, not Entity:Load) during a drush cr gives me impredictable results, some mentioning either that Field moderation_state is unknown or that Route entity.node.latest_version does not exist.

Enough for me to suspect a bug on content_moderation module. Unfortunately I have no more clues at the moment.

(Obviously a clear cache without traffic puts everything in place)

A variant :

InvalidArgumentException: Field moderation_state is unknown. in Drupal\Core\Entity\ContentEntityBase->getTranslatedField() 
(line 580 of core/lib/Drupal/Core/Entity/ContentEntityBase.php). 

Drupal\Core\Entity\ContentEntityBase->get("moderation_state") (Line: 285)
Drupal\content_moderation\EntityOperations->entityView(Array, Object, Object, "full") (Line: 172)
content_moderation_entity_view(Array, Object, Object, "full")
call_user_func_array("content_moderation_entity_view", Array) (Line: 402)
Drupal\Core\Extension\ModuleHandler->invokeAll("entity_view", Array) (Line: 270)
Drupal\Core\Entity\EntityViewBuilder->buildMultiple(Array) (Line: 220)
Drupal\Core\Entity\EntityViewBuilder->build(Array)

Another variant :

Symfony\Component\Routing\Exception\RouteNotFoundException: Route "entity.node.latest_version" does not exist. in Drupal\Core\Routing\RouteProvider->getRouteByName() (line 202 of core/lib/Drupal/Core/Routing/RouteProvider.php). 

Drupal\Core\Menu\LocalTaskDefault->getRouteParameters(Object) (Line: 310)
Drupal\Core\Menu\LocalTaskManager->getTasksBuild("entity.node.canonical", Object) (Line: 358)
Drupal\Core\Menu\LocalTaskManager->getLocalTasks("entity.node.canonical", 0) (Line: 94)
Drupal\Core\Menu\Plugin\Block\LocalTasksBlock->build() (Line: 203)
Drupal\block\BlockViewBuilder::preRender(Array)
call_user_func("Drupal\block\BlockViewBuilder::preRender", Array) (Line: 378)
Drupal\Core\Render\Renderer->doRender(Array) (Line: 450)
Drupal\Core\Render\Renderer->doRender(Array, ) (Line: 195)
Drupal\Core\Render\Renderer->render(Array) (Line: 490)
Drupal\Core\Template\TwigExtension->escapeFilter(Object, Array, "html", NULL, 1) (Line: 65)
__TwigTemplate_e0e5c03852788b844753634506d4ceb0082bd18845b7eea1c67638d5d649c658->doDisplay(Array, Array) (Line: 432)
Twig_Template->displayWithErrorHandling(Array, Array) (Line: 403)
Twig_Template->display(Array) (Line: 411)
Twig_Template->render(Array) (Line: 64)
twig_render_template("themes/bespoke/rdcmobile/templates/layout/page.html.twig", Array) (Line: 384)
Drupal\Core\Theme\ThemeManager->render("page", Array) (Line: 437)
Drupal\Core\Render\Renderer->doRender(Array, ) (Line: 195)
Drupal\Core\Render\Renderer->render(Array) (Line: 490)
Drupal\Core\Template\TwigExtension->escapeFilter(Object, Array, "html", NULL, 1) (Line: 90)
__TwigTemplate_be8c7bbb9c824f2826368d7c8da984c6279779db72a67fd8056a00bb23b816f2->doDisplay(Array, Array) (Line: 432)
Twig_Template->displayWithErrorHandling(Array, Array) (Line: 403)
Twig_Template->display(Array) (Line: 411)
Twig_Template->render(Array) (Line: 64)
twig_render_template("core/themes/classy/templates/layout/html.html.twig", Array) (Line: 384)
Drupal\Core\Theme\ThemeManager->render("html", Array) (Line: 437)
Drupal\Core\Render\Renderer->doRender(Array, ) (Line: 195)
Drupal\Core\Render\Renderer->render(Array) (Line: 147)
Drupal\Core\Render\MainContent\HtmlRenderer->Drupal\Core\Render\MainContent\{closure}() (Line: 582)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 148)
Drupal\Core\Render\MainContent\HtmlRenderer->renderResponse(Array, Object, Object) (Line: 90)
Drupal\Core\EventSubscriber\MainContentViewSubscriber->onViewRenderArray(Object, "kernel.view", Object)
call_user_func(Array, Object, "kernel.view", Object) (Line: 111)
Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch("kernel.view", Object) (Line: 156)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 68)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 57)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 184)
Drupal\page_cache\StackMiddleware\PageCache->fetch(Object, 1, 1) (Line: 121)
Drupal\page_cache\StackMiddleware\PageCache->lookup(Object, 1, 1) (Line: 75)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 50)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 23)
Stack\StackedHttpKernel->handle(Object, 1, 1) (Line: 664)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

API page: https://api.drupal.org/api/drupal/core!lib!Drupal!Core!Entity!EntityForm...

> Returns an array of supported actions for the current entity form.

This is a bit misleading. You could think this is an array of definitions, whereas in fact it's a Form API array, that is, form elements.

https://api.drupal.org/api/drupal/core%21lib%21Drupal%21Core%21Access%21... isn't useful in a lot of situations.

For example, consider the use case that I want to limit access to an entity field to only users with a certain permission.

If I implement hook_entity_field_access() and return AccessResult::allowedIfHasPermission(), then that has no effect, as it returns a neutral if the user doesn't have the permission.

What's needed is the opposite of allowedIfHasPermission() - a method that denies access if the permission is absent.

Problem/Motivation

In #2881788: Process-isolate Drupal\Tests\Core\Extension\ModuleHandlerTest we had to do some fancy footwork in order to fix a test of ModuleHandler.

That's because Drupal\Core\Extension\Extension has a dependency on DRUPAL_ROOT, which makes it difficult to write unit tests in some circumstances.

Specifically, when the Extension object is unserialized, it will try to find the Drupal application root by using the DRUPAL_ROOT constant. This is a problem when you're writing a unit test which is process-isolated, because DRUPAL_ROOT is (rightly) not defined for unit tests.

Proposed resolution

The goal of this issue is to change Drupal\Core\Extension\Extension::unserialize() so that it checks for the availability of \Drupal::root() and uses that service if available, falling back on DRUPAL_ROOT if it is not.

There should also be a KernelTestBase test which spoofs the app.root service to prove that the service is used in preference over the constant when unserialized.

Remaining tasks

User interface changes

API changes

Data model changes

Problem/Motivation

From #2743313-16: [policy, docs] Use post updates for empty 'clear the cache' updates

What about creating a patch as part of this issue to change the comments within all of our existing empty update hooks? Similar to what we have in block_content_update_8002()?

Proposed resolution

grep -rni 'function.*_update_\d' core
Use above command to find all the hook_update_N and then select empty hook_update_N used for cache clear only.

Remaining tasks

Find out all the empty hook_update_N used for cache clearing and add the following docs.

// Use of hook_post_update_NAME instead to clear the cache. 
// The use of hook_update_N to clear the cache has been deprecated
// see https://www.drupal.org/node/2960601 for more details.

User interface changes

API changes

Data model changes

Problem/Motivation

If a page has a URL like /blog?tag=security&page=4, the canonical URL will be rendered as /blog?tag=security&page=4.

This will cause problems with SEO and crawlers as they depend on canonical links to be valid. It also makes it impossible to meet Google's recommendations on how to indicate paginated content properly. This is why this issue is initially marked Major.

To reproduce this:

- Create a SimplyTest.me project using the MetaTags module
- Add the metatags field to the basic page content type
- Create a basic page and use the Metatag advanced setting to set the canonical URL to /blog?tag=security&page=4
- Open in Chrome and view the page source (Note: Inspect will hide the & value).
- The canonical href will have be encoded.

Proposed resolution

The underlying problem is in the HtmlTag code. It creates the tag markup using the Attribute class's __toString method. This method escapes all attribute labels and values.

The fix here is to handle link tag href attributes and script tag src attributes as special cases in the HtmlTag code.

The HREF and SRC attributes should not be added to the Attributes class. Instead, their values should be validated with UrlHelper::isValid(). If they fail validation, then they should be fully escaped. These attributes should be added to the tag directly. All other attributes should go thru the Attribute class.

Remaining tasks

- Write the patch
- Write some tests

User interface changes

None

API changes

None

Data model changes

None