Problem/Motivation

This is issue is spun off from UX team feedback on #2831944-203: Implement media source plugin for remote video via oEmbed.

For security reasons, the oEmbed system uses an iframe to serve content from a third-party oEmbed provider. By default. the iframe is served from the same domain as the main Drupal site, but this is not secure. Therefore, Media introduced a setting, exposed in a configuration form, which allows site builders/admins to set up an alternate domain from which to serve the iframe.

In order to serve oEmbed content more securely, the iFrame domain needs to point to the Drupal site. This is explained on the form, but not validated in any way.

Proposed resolution

It would be nice to add some sort of validation to ensure that the iFrame domain is actually pointing to the Drupal site, because if it isn't, then almost all oEmbed content on the site will break (404 errors or worse), which scare the pants off of our users.

Remaining tasks

1. Discuss if we should even do this, and if so, how to do it in a way that will please the security team.
2. Write a patch
3. Review it until we're all sick of looking at it
4. Commit it

User interface changes

TBD, but probably none.

API changes

TBD, but probably minimal or none.

Data model changes

None anticipated.

I can no longer save changes to the layout after any kind of form has been added to the layout.

Steps to reproduce:
1. Enable layout builder on a content type, then open layout builder.
2. Add 'Search Form' block to a section
3. Save the layout
4. Re-open the layout builder, and make some change (e.g. reorder blocks)
5. Attempt to save the layout

The page refreshes, but still shows unsaved changes. At this point, it is no longer possible to save the layout until the search form is removed.

Problem/Motivation

To enable more control over the html returned by an oembed provider, more optional contextual information in the theme implementation media_oembed_iframe would be useful (e.g. to enable autoplay for videos from YouTube, if a field formatter setting is set).

Proposed resolution

To enable modules to add contextual information to the iframe, the OEmbedIframeController should pass a "context" query argument to the render array.

To easily add context based on the provider or field formatter, an additional alter hook for the iframe usage should be added.

Remaining tasks

Review of the attached patch.

User interface changes

None

API changes

Additional alter hook for contextual information in the formatter.

Data model changes

None

Problem/Motivation

If a theme hasn't provided base theme setting, the theme will currently fallback to extending stable. This works when there's only one version of stable. However, in Drupal 9, there will be multiple stable versions (Drupal 8 stable aka stable and Drupal 9 stable aka stable9).

When Drupal 9 ships, we can choose to either either:

1. move Drupal 8's Stable (stable) to contrib and a new Stable theme (stable9) to core
2. keep Drupal 8's Stable (stable) in core and add a new Stable theme (stable9 to core

This decision doesn't need to be made in this issue.

In either scenario, themes must explicitly define which version of the Stable theme it wants to extend, by specifying either base theme: stable or base theme: stable9.

Note 1: this solution will also work for Drupal 10! 🥳
Note 2: we could choose to alias stable to stable8 for consistency. But this would merely be a cosmetic nice-to-have.

Proposed resolution

Deprecate the option to omit the base theme property in theme *.info.yml files. Provide warning for themes that haven't configured their base theme and provide them with instructions that if they want their theme behavior remain the same in Drupal 9, they would have to add base theme: stable.

In Drupal 9, the base theme property will be required.

Remaining tasks

• Consensus on approach. See #6 + #7.
• Agree on naming: does Drupal 8's "Stable" theme keep the stable name or do we add stable8 as an alias, for consistency with future Drupal versions'stable9, stable10, et cetera?
• Green patch. See #38 (complicated update to BaseThemeDefaultDeprecationTest) + #41 (removes BaseThemeDefaultDeprecationTest)
• Decide whether we want to extract "app root"-related changes to a separate issue — see #40. That'd make this patch more tightly scoped.
• RTBC.
• Commit.

User interface changes

None.

API changes

base theme is no longer an optional property in a theme *.info.yml file. Change record: https://www.drupal.org/node/3066038

Data model changes

None.

Release notes snippet

Infrastructure added to allow themes written for Drupal 8 to continue to work in Drupal 9.

Problem/Motivation

Currently, when https://oembed.com/providers.json is fetched to generate a list of available oEmbed providers, there is no opportunity to alter the list of providers.

This would be useful in scenarios such as 1) Peer Tube and 2) oEmbed providers that may be organization-specific. Peer Tube is a decentralized video platform. The user can search videos across multiple instance of PeerTube with different domain names. Re custom oEmbed providers, the University of Nebraska-Lincoln hosts its own video service, which will soon support oEmbed. This functionality is needed by this organization.

Proposed resolution

Provide hook_oembed_providers_alter(). (Media already provides hook_oembed_resource_url_alter()).

Remaining tasks

• Submit patch
• Tests pass
• Subsystem maintainer review
• Change record

User interface changes

None.

API changes

Adding hook_oembed_providers_alter().

Data model changes

None.

Release notes snippet

TBD

Original report by phjou

I have studied the code from the oEmbed system in the core and I have discovered that it is based on the list provided by https://oembed.com/providers.json

This is a good list to start but I am wondering how to add a provider to this list?
You can host a new file with the additional services and change the settings "oembed_providers_url". But this method is really complicated to just support more providers.

Moreover I am creating this issue in order to support PeerTube which is a decentralized video platform. The user can search videos across multiple instance of PeerTube with different domain names. Such a complicated way to support a new provider is just impossible for a simple contributor.

In order to resolve this problem:
- The user should be able to override the default URL. The issue already exists for another reason: #2999018
- We should be able to alter the provider list by using a hook. What do you think about adding an alter call in the getAll function from Drupal\media\OEmbed\ProviderRepository?

We have a Media Type "Image" that, unlike its other media type brethren, shows the field "Publisher ID", and it doesn't pre-populate to the current user. Additionally, the other media types show "Authored By" instead, and populate.

Could this be a bug, configuration issue, or a combination of the two?

Problem/Motivation

The oEmbed system requires all non-link resources (i.e., photos, videos, rich text) to have an explicit, non-zero width and a height defined, and throws exceptions (in \Drupal\media\OEmbed\Resource::__construct()) if they don't. This is in keeping with the oEmbed specification. However, some oEmbed providers, like Twitter and Instagram, disobey this rule because the assets they serve contain live text, and may be responsive, and there can't really have a known, predefined height. Therefore, they will be incompatible with our oEmbed implementation.

Proposed resolution

Loosen our adherence to the oEmbed standard in order to support remote assets with dynamic heights.

Remaining tasks

Figure out the best approach, write a patch with tests, and commit it.

User interface changes

None.

API changes

TBD, but likely a couple fewer exceptions will be thrown.

Data model changes

Likely none.

the oEmbedFormatter renders remote video content in an iframe element. This element should contain a title attribute for better accessibility.

https://webaim.org/techniques/frames/#title

Easy fix. Will supply patch shortly.

Updated #65

Problem/Motivation

• Install content_moderation
• Apply the "Editorial" workflow to articles.
• Create an article node in the published workflow state.
• Goto /admin/content
• Use the bulk action "Unpublish content" on the article node.
• You will see this validation error:
  

Proposed resolution

• Remove the publish/unpublish action
• Provide action plugins for every state
• Dynamically create/update/delete action instances when workflow settings are changed.
• Update moderation_content admin view with the state actions.

Remaining tasks

Dynamically create/update/delete action instances when:

• create/update/delete operation happens on a workflow entity by implementing hook workflow (insert|update|delete).
• changes to the workflow entity happen during config import.
• changes to the workflow entity are done hook_update_N.

Update moderation_content admin view with the state actions.
Add functional test for both cases.

User interface changes

moderation_content admin view will have state change actions.

API changes

No API change only addition.
Adds new moderation state change action and deriver.

Data model changes

None.

Problem/Motivation

When the oEmbed provider list cannot be loaded it throws a fatal exception for the entire page. It should log an error message instead.

Proposed resolution

Add the logger trait to the ProviderRepository class

Remaining tasks

Implement logger channel
Inject logger channel
Add tests for logger

User interface changes

None

API changes

None

Data model changes

None

Hello,

I've been studied the media core module, and I have found that it is not possible to support multiple websites with the same provider at the same time. I think it is really annoying that it supports only the centralized approach because the internet is supposed to be decentralized.

The name of the provider is fixed by the service that you use. I will take the example of my usecase to be easier to understand. So my usecase is the PeerTube, a decentralized video platform.

So oEmbed imposes us the provider name as we can see in those examples:

For Youtube:
https://www.youtube.com/oembed?url=https://www.youtube.com/watch?v=xHLes...
provider_name = Youtube

For PeerTube:
https://framatube.org/services/oembed?url=https%3A%2F%2Fframatube.org%2F...
provider_name = PeerTube
https://peertube.cpy.re/services/oembed?url=https%3A%2F%2Fpeertube.cpy.r...
provider_name = PeerTube

Consequently we have something like that in the ressource fetcher yml file pointed by the oembed_providers_url variable from media.settings

[
  0	=> {
    provider_name:	"PeerTube",
    provider_url:	"https://framatube.org",
    endpoint: [
      0 => {	
        url	"https://framatube.org/services/oembed",
        discovery	true
      }
    ]
  },
  1	=> {
    provider_name:	"PeerTube",
    provider_url:	"https://peertube.cpy.re",
    endpoint: [
      0 => {	
        url	"https://peertube.cpy.re/services/oembed",
        discovery	true
      }
    ]
  }
]

After that we arrive to the part that the media core module fails totally.

In Drupal\media\OEmbed\ProviderRepository

$keyed_providers = [];
    foreach ($providers as $provider) {
      try {
        $name = (string) $provider['provider_name'];
        $keyed_providers[$name] = new Provider($provider['provider_name'], $provider['provider_url'], $provider['endpoints']);
      }
      catch (ProviderException $e) {
        // Just skip all the invalid providers.
        // @todo Log the exception message to help with debugging.
      }
    }

If we read that code, it will iterate on every provider and fill the $keyed_providers variable. But of course, it will erase the data from the other websites with the same provider. After that, the core media module will be ok but only for the last website you have declared with the common provider.

I will try to work on that but this will probably be a big change on how the media module uses oEmbed.

For those who want to work on that, the core doesn't allow to override the oembed_providers_url setting yet. So set it manually or there is already an issue on drupal.org: #2999018 Expose oEmbed provider URL setting in the Media configuration form

I have a working module that supports only one website of Peertube because of this issue. If you want to debug this issue, it will give you a usecase to debug: PeerTube

Thank you to those who will help me to resolve this issue.

I tried to play with media_theme_suggestions_media, but it did not work in my case - seems that Drupal is not going into the hook_theme_suggestions_media
for example:

media-oembed-iframe--provider-vimeo.html
media-oembed-iframe--provider-youtube.html

I know I may have something wrong with media__source_ but It's not showing
the suggestion in the twig debug to know the right format

Thanks for all your work on the media module

Opened this issue as a follow up on the following lin
https://www.drupal.org/project/drupal/issues/2998091#comment-13013994
from
#2998091: Remote videos overflow their containing element

/**
 * Implements hook_theme_suggestions_HOOK().
 */
function media_theme_suggestions_media(array $variables) {
  $suggestions = [];
  /** @var \Drupal\media\MediaInterface $media */
  $media = $variables['elements']['#media'];
  $sanitized_view_mode = strtr($variables['elements']['#view_mode'], '.', '_');

  $suggestions[] = 'media__' . $sanitized_view_mode;
  $suggestions[] = 'media__' . $media->bundle();
  $suggestions[] = 'media__' . $media->bundle() . '__' . $sanitized_view_mode;

  // Add suggestions based on the source plugin ID.
  $source = $media->getSource();
  if ($source instanceof DerivativeInspectionInterface) {
    $source_id = $source->getBaseId();
    $derivative_id = $source->getDerivativeId();
    if ($derivative_id) {
      $source_id .= '__derivative_' . $derivative_id;
    }
  }
  else {
    $source_id = $source->getPluginId();
  }
  $suggestions[] = "media__source_$source_id";

  // If the source plugin uses oEmbed, add a suggestion based on the provider
  // name, if available.
  if ($source instanceof OEmbedInterface) {
    $provider_id = $source->getMetadata($media, 'provider_name');
    if ($provider_id) {
      $provider_id = \Drupal::transliteration()->transliterate($provider_id);
      $provider_id = preg_replace('/[^a-z0-9_]+/', '_', mb_strtolower($provider_id));
      $suggestions[] = end($suggestions) . "__provider_$provider_id";
    }
  }

  return $suggestions;
}

This issue is spun off from #2831944-237: Implement media source plugin for remote video via oEmbed, point #1. There are several places in the oEmbed API that handle errors that may occur during the process of fetching or otherwise interacting with oEmbed resources, and these errors are either not logged at all, or logged in very general terms that will not help with troubleshooting. We should revisit these areas and improve the error handling.

The best way I could figure out to do this was with a hack of media-oembed-iframe.html.twig, forcing ?rel=0 into the URL.

Seems like this should be an option in the display settings of the embedded video media type.

If you, like me, want this to work, I did it with twig, this is the contents of my overridden media-oembed-iframe.html.twig file (Also with the

{#
/**
 * @file
 * Default theme implementation to display an oEmbed resource in an iframe.
 *
 * @ingroup themeable
 */
#}
<!DOCTYPE html>
<html>
<head>
  <style>
    iframe {
      position: absolute;
      left: 0;
      top: 0;
      right: 0;
      bottom: 0;
      margin: 0;
      width: 100%;
      height: 100%;
    }
  </style>
</head>
<body style="margin: 0">
{{ media|replace({ "?feature=oembed": "?rel=0&feature=oembed"  })|raw }}
</body>
</html>

Problem/Motivation

When viewing an entity that references an ombed field with caching turned off and no internet connection, a Drupal\media\OEmbed\ProviderException is thrown due to the oembed provider unable to be retrieved. This exception is not caught and causes the site to return a fatal error.

Proposed resolution

• Do not cause a fatal error if caching is turned off and no internet connection is available
• Catch the Drupal\media\OEmbed\ProviderException and log the error
• Provide a unit test

Remaining tasks

• Catch / log exception
• Update documentation to expect to catch an exception when using the Drupal\media\OEmbed\ProviderRepository service.

User interface changes

None.

API changes

None.

Data model changes

None.

Problem/Motivation

There is a mistake in comment documenting function image of the Random class.

See: https://git.drupalcode.org/project/drupal/blob/8.8.x/core/lib/Drupal/Com...

  /**
   * Create a placeholder image.
   *
   * @param string $destination
   *   The absolute file path where the image should be stored.
   * @param int $min_resolution
   * @param int $max_resolution
   *
   * @return string
   *   Path to image file.
   */
  public function image($destination, $min_resolution, $max_resolution) {
    $extension = pathinfo($destination, PATHINFO_EXTENSION);
    $min = explode('x', $min_resolution);
    $max = explode('x', $max_resolution);

$min_resolution, $max_resolution are actually strings in the form of e.g. '400x400' , which becomes obvious after reading the beginning of the image method.

Problem/Motivation

Over in #2831944-187: Implement media source plugin for remote video via oEmbed, @dawhner raised this point:

I'm curious, what would be the advantage of storing the thumbnail locally?

I answered:

If we don't, we're basically hotlinking to an unreliable external resource every time we want to display the thumbnail. I'll open a follow-up to expose a configuration option on the oEmbed source plugin to toggle whether or not the thumbnails are downloaded.

However, it's a legitimate question and worth considering.

Proposed resolution

Possibly add some sort of a switch (plugin configuration option? global config value?) so that site builders can control whether or not thumbnails for oEmbed media items are stored locally.

Remaining tasks

1. Discuss whether we should even do this
2. If we decide to do it, write, review, and commit a patch

User interface changes

TBD.

API changes

TBD.

Data model changes

TBD.

Problem/Motivation

Currently the oEmbed resource fetching in core requires that a specific set of data be returned as a Resource type, discarding data from the provider's return. There are two problems with this:

1. Non-standardized returns cannot use their meta, such as SoundCloud XML using dashes instead of underscores (https://soundcloud.com/oembed?url=https://soundcloud.com/forss/flickermood). I know this is an issue on their end, but is a common one among 3rd parties
2. There is no direct access to the provider meta data in the case the provider has extra meta attached to a response

Proposed resolution

Add a method that supports returning the provider response.

I'm running Drupal 8.6.1 locally in Acquia Dev Desktop and I'm trying to add Remote Video (oEmbed) assets.

I have left the "Thumbnails location" at the default setting of public://oembed_thumbnails

When I attempt to add my first Remote Video media asset ( https://www.youtube.com/watch?v=anAQ794lsBM ) I get this error message that the thumbnail URI is not valid:

The website encountered an unexpected error. Please try again later.</br></br><em class="placeholder">InvalidArgumentException</em>: The URI &#039;public://oembed_thumbnails/6OpXFifNwFUlN09VaoQ-jzI3F_Tac971gZsWRAjnEqM.jpg&#039; is invalid. You must use a valid URI scheme. Use base: for a path, e.g., to a Drupal file that needs the base path. Do not use this for internal paths controlled by Drupal. in <em class="placeholder">Drupal\Core\Utility\UnroutedUrlAssembler-&gt;assemble()</em> (line <em class="placeholder">65</em> of <em class="placeholder">core\lib\Drupal\Core\Utility\UnroutedUrlAssembler.php</em>). <pre class="backtrace">Drupal\Core\Url-&gt;toString() (Line: 180)
Drupal\link\Plugin\Field\FieldFormatter\LinkFormatter-&gt;viewElements(Object, &#039;en&#039;) (Line: 80)
Drupal\Core\Field\FormatterBase-&gt;view(Object, &#039;en&#039;) (Line: 262)
Drupal\Core\Entity\Entity\EntityViewDisplay-&gt;buildMultiple(Array) (Line: 321)
Drupal\Core\Entity\EntityViewBuilder-&gt;buildComponents(Array, Array, Array, &#039;media_library&#039;) (Line: 263)
Drupal\Core\Entity\EntityViewBuilder-&gt;buildMultiple(Array) (Line: 220)
Drupal\Core\Entity\EntityViewBuilder-&gt;build(Array)
call_user_func(Array, Array) (Line: 378)
Drupal\Core\Render\Renderer-&gt;doRender(Array, ) (Line: 195)
Drupal\Core\Render\Renderer-&gt;render(Array) (Line: 1153)
Drupal\views\Plugin\views\field\FieldPluginBase-&gt;advancedRender(Object) (Line: 235)
template_preprocess_views_view_field(Array, &#039;views_view_field&#039;, Array) (Line: 287)
Drupal\Core\Theme\ThemeManager-&gt;render(&#039;views_view_field&#039;, Array) (Line: 437)
Drupal\Core\Render\Renderer-&gt;doRender(Array, ) (Line: 195)
Drupal\Core\Render\Renderer-&gt;render(Array) (Line: 1743)
Drupal\views\Plugin\views\field\FieldPluginBase-&gt;theme(Object) (Line: 761)
Drupal\views\Plugin\views\style\StylePluginBase-&gt;elementPreRenderRow(Array)
call_user_func(Array, Array) (Line: 378)
Drupal\Core\Render\Renderer-&gt;doRender(Array, ) (Line: 195)
Drupal\Core\Render\Renderer-&gt;render(Array) (Line: 709)
Drupal\views\Plugin\views\style\StylePluginBase-&gt;renderFields(Array) (Line: 576)
Drupal\views\Plugin\views\style\StylePluginBase-&gt;renderGrouping(Array, Array, 1) (Line: 468)
Drupal\views\Plugin\views\style\StylePluginBase-&gt;render(Array) (Line: 2111)
Drupal\views\Plugin\views\display\DisplayPluginBase-&gt;render() (Line: 131)
Drupal\webprofiler\Views\TraceableViewExecutable-&gt;render() (Line: 183)
Drupal\views\Plugin\views\display\Page-&gt;execute() (Line: 1630)
Drupal\views\ViewExecutable-&gt;executeDisplay(&#039;page&#039;, Array) (Line: 77)
Drupal\views\Element\View::preRenderViewElement(Array)
call_user_func(Array, Array) (Line: 378)
Drupal\Core\Render\Renderer-&gt;doRender(Array, ) (Line: 195)
Drupal\Core\Render\Renderer-&gt;render(Array, ) (Line: 226)
Drupal\Core\Render\MainContent\HtmlRenderer-&gt;Drupal\Core\Render\MainContent\{closure}() (Line: 582)
Drupal\Core\Render\Renderer-&gt;executeInRenderContext(Object, Object) (Line: 227)
Drupal\Core\Render\MainContent\HtmlRenderer-&gt;prepare(Array, Object, Object) (Line: 117)
Drupal\Core\Render\MainContent\HtmlRenderer-&gt;renderResponse(Array, Object, Object) (Line: 90)
Drupal\Core\EventSubscriber\MainContentViewSubscriber-&gt;onViewRenderArray(Object, &#039;kernel.view&#039;, Object) (Line: 76)
Drupal\webprofiler\EventDispatcher\TraceableEventDispatcher-&gt;dispatch(&#039;kernel.view&#039;, Object) (Line: 156)
Symfony\Component\HttpKernel\HttpKernel-&gt;handleRaw(Object, 1) (Line: 68)
Symfony\Component\HttpKernel\HttpKernel-&gt;handle(Object, 1, 1) (Line: 57)
Drupal\Core\StackMiddleware\Session-&gt;handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\KernelPreHandle-&gt;handle(Object, 1, 1) (Line: 99)
Drupal\page_cache\StackMiddleware\PageCache-&gt;pass(Object, 1, 1) (Line: 78)
Drupal\page_cache\StackMiddleware\PageCache-&gt;handle(Object, 1, 1) (Line: 47)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware-&gt;handle(Object, 1, 1) (Line: 38)
Drupal\webprofiler\StackMiddleware\WebprofilerMiddleware-&gt;handle(Object, 1, 1) (Line: 52)
Drupal\Core\StackMiddleware\NegotiationMiddleware-&gt;handle(Object, 1, 1) (Line: 23)
Stack\StackedHttpKernel-&gt;handle(Object, 1, 1) (Line: 665)
Drupal\Core\DrupalKernel-&gt;handle(Object) (Line: 19)
</pre>

However the folder C:\Users\Nick\Sites\devdesktop\drupal\sites\drupal.dd\files\oembed_thumbnails does get created and an appropriate thumbnail named 6OpXFifNwFUlN09VaoQ-jzI3F_Tac971gZsWRAjnEqM.jpg is placed within it.

I am not sure whether this is a bug or I am using an inappropriate thumbnails location. I tried a few different options but no success. What thumbnails location should I be using in a local Dev Desktop environment?

Problem/Motivation

In #2831944-187: Implement media source plugin for remote video via oEmbed, @dawehner raised this point about the fact that Media will warn site administrators (via the status page), if they are displaying remote oEmbed resources in an iframe served from the same domain as the main Drupal site:

I'm curious whether we could somehow instead of a warning ensure that people at least checked a checkbox to know that they are doing something insecure.

Should site administrators be allowed to permanently acknowledge, and therefore suppress, this legitimate and actionable security warning? If so, how would we go about doing that?

Proposed resolution

TBD.

Remaining tasks

Discuss whether we should do this at all, and possibly implement a patch.

User interface changes

TBD.

API changes

TBD.

Data model changes

TBD.