Problem/Motivation

If you need to load a lot of entities and iterate over them (pretty common), your memory usage could easily go through the roof.

For example, consider updating all nodes for some reason is a post_update hook. You want to load them all and save them, but loading all 10000000000 nodes will kill your machine. You would need to load them in batches to keep memory usage at a sane level.

This could also affect core directly, as we may need to update content (or config) entities in the upgrade path, but have no way of knowing how many items a site will have. This could cause problems and easily balloon over the PHP requirements we set to run Drupal.

Proposed resolution

Introduce an EntityIterator class that uses a generator to break the IDs to load into chunks. Then load just a set amount each time, clearing the static cache before each load so it only contains the items currently being iterated over.

We can get clever and use a generator. This internally creates an iterator so we can implement \IteratorAggregate and just return our generator!!

Remaining tasks

Add some tests

User interface changes

N/A - Developer only

API changes

Addition of new Iterator

Data model changes

N/A

Problem/Motivation

The media library allows a user to upload a new file to create (but not yet save) a new media item. After the media item has been created (but not yet saved), the form presents a 'Remove' button. Pressing the 'Remove' button however, does not actually remove the uploaded file. This is inconsistent with adding a media item from /media/add/image where the Image widget actually removes the file from the file system.

Proposed resolution

Immediately delete a media's file when pressing 'Remove' in the media creation modal.

Remaining tasks

1. Write a patch
2. Review
3. Commit

User interface changes

None.

API changes

None.

Data model changes

None.

The spelling of the word writeable/writable is not consistent in core. Core use ‘writeable’ 22 times and ‘writable’ 154 times. (#9)

Possible resolutions:
- Change all instances of ‘writeable’ to ‘writable’
- Change all instances of ‘writable’ to ‘writeable’
- Leave as-is. Either spelling is correct.

Given that I am not logged in and am trying to login as the administrator to a 8.7.6 site,
when I use the correct username but the wrong password, an error message tells me that the username is not correct.
Shouldn't we modify the error message so that the error message reads "either the username or the password is not correct"?

or

Is this the intended behavior because it is thought that this behavior will help defeat hackers ? (deliberately misrouting them)?

Problem/Motivation

jQuery Joyride was added to support tours. It is used in a Views tour and a welcome tour in Umami. However Joyride has not been maintained for the last 2 years. See https://zurb.com/playground/jquery-joyride-feature-tour-plugin and commit history at https://github.com/zurb/joyride

Proposed resolution

Consider replacing Joyride or remove altogether if we don't think its useful. The Umami team thinks it is useful and uses it.

Remaining tasks

Discuss.

User interface changes

TBD.

API changes

TBD.

Data model changes

TBD.

Release notes snippet

TBD.

Hi, When I try to update Drupal 8.7.0 db br drush updb I get the following error.

>  [notice] Update started: taxonomy_post_update_make_taxonomy_term_revisionable
>  [error]  Drupal\Core\Database\IntegrityConstraintViolationException: SQLSTATE[23000]: Integrity constraint violation: 1048 Column 'name' cannot be null: INSERT INTO {tmp_d305d9taxonomy_term_field_data} (tid, revision_id, vid, langcode, name, description__value, description__format, weight, changed, default_langcode, status, revision_translation_affected) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4, :db_insert_placeholder_5, :db_insert_placeholder_6, :db_insert_placeholder_7, :db_insert_placeholder_8, :db_insert_placeholder_9, :db_insert_placeholder_10, :db_insert_placeholder_11); Array
> (
>     [:db_insert_placeholder_0] => 1
>     [:db_insert_placeholder_1] => 1
>     [:db_insert_placeholder_2] => tags
>     [:db_insert_placeholder_3] => tr
>     [:db_insert_placeholder_4] =>
>     [:db_insert_placeholder_5] =>
>     [:db_insert_placeholder_6] =>
>     [:db_insert_placeholder_7] =>
>     [:db_insert_placeholder_8] =>
>     [:db_insert_placeholder_9] =>
>     [:db_insert_placeholder_10] =>
>     [:db_insert_placeholder_11] => 1
> )
>  in Drupal\Core\Database\Connection->handleQueryException() (line 689 of /var/www/drupal8/web/core/lib/Drupal/Core/Database/Connection.php).
>  [error]  The entity update process failed while processing the entity type taxonomy_term, ID: 1.
>  [error]  Update failed: taxonomy_post_update_make_taxonomy_term_revisionable
 [error]  Update aborted by: taxonomy_post_update_make_taxonomy_term_revisionable
 [error]  Finished performing updates.

Problem/Motivation

In #3074043: Move simpletest.module DB-related functions to TestDatabase, deprecate we're trying to deprecate top-level simpletest module functions by moving them to Drupal\Core\Test\TestDatabase.

It so happens that we can't test many of our changes, because TestDatabase is mostly static and thus can't be mocked.

Proposed resolution

Figure out how to modify TestDatabase so that it can be tested.

Modify.

Test.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

For class names that appear in routing.yml files, core sometimes prefixes these class names with "\" and sometimes doesn't.

For example, in system.routing.yml, observe that the first _controller uses "\Drupal" while the second in just uses "Drupal"

system.cron:
  path: '/cron/{key}'
  defaults:
    _controller: '\Drupal\system\CronController::run'
  options:
    no_cache: TRUE
  requirements:
    _access_system_cron: 'TRUE'

system.admin_compact_page:
  path: '/admin/compact/{mode}'
  defaults:
    _controller: 'Drupal\system\Controller\SystemController::compactPage'
    mode: 'off'
  requirements:
    _permission: 'access administration pages'

I found this to be a problem with class names appearing in _controller, _form, and _title_callback

The attached patch makes all class names start with "\Drupal".

Problem/Motivation

As part of the general clean-up of the File API, file_save_htaccess() and file_ensure_htaccess() need to be deprecated and moved into services.

Proposed Resolution

Move this functionality into a service called htaccess, then turn both of these functions into wrappers around that service.

Problem/Motivation

The content translation handler relies too much on the form state even for stuff that could be read-out directly from the entity. Coupling it so much on the form state makes it really hard for nested entity forms to use it.

Proposed resolution

Instead of determining if the entity is being editing in a new translation by

$new_translation = !empty($source_langcode);

change it to

$new_translation = $entity->isNewTranslation();

and instead of retrieving the source langcode through

$source_langcode = $this->getSourceLangcode($form_state);

retrieve it through

$source_langcode = $metadata->getSource();

but for this adjust
ContentTranslationController::prepareTranslation to set the source language code on the prepared entity - this is actually something we have to do anyway when creating the translation and instead in the content translation entity builder.

Remaining tasks

Review.

User interface changes

None.

API changes

None.

Data model changes

None.

Problem/Motivation

The JSON:API FieldResolver was written prior to a lot of internal JSON:API refactoring. The biggest of which was the introduction of the ResourceType value object which helps map Drupal's entity-based data model into a JSON:API resource object-based data model.

Part of the field resolver has already been refactored to use this value object, but FieldResolver::resolveInternalEntityQueryPath($entity_type_id, $bundle, $external_field_name) has not been.

This shortcoming came up during Decoupled Days 2019 when a lot of interest was expressed for adding the ability to fetch all entities of a particular entity type, not just per-bundle. In other words, JSON:API does not currently have a /jsonapi/node route, only /jsonapi/node/{bundle} routes.

It was agreed that an experiment would be made in contrib to support his new feature. See https://www.drupal.org/project/jsonapi_cross_bundles. This refactoring would unblock that experiment in contrib (currently it requires a core patch/hack) while making the JSON:API module more internally consistent.

Proposed resolution

Refactor this:

FieldResolver::resolveInternalEntityQueryPath($entity_type_id, $bundle, $external_field_name)

into this:

FieldResolver::resolveInternalEntityQueryPath(ResourceType $resource_type, $external_field_name)

which would make that method match the already refactored method on the same class:

FieldResolver::resolveInternalIncludePath(ResourceType $resource_type, array $path_parts, $depth = 0)

User interface changes

None.

API changes

None. The field resolver is @internal.

Data model changes

None.

Release notes snippet

N/A

Problem/Motivation

Automated tests greatly increase the size of a bare Drupal installation on disk. (As of 609912bced (8.8.x, July 15, 2019), they accounted for 43M of 94M (46%).) And they provide no benefit to end users in production. Indeed, they can contribute to security vulnerabilities and other problems. The number and size of files can also be a problem for users of shared hosting (c.f. #2338671: Create a smaller Drupal core download, removing tests, for production sites.), and they needlessly increase download and transfer times and disk usage for everyone, including drupal.org.

Proposed resolution

Use the approach documented in Use gitattributes to keep your tests out of other people’s production to exclude all tests/ directories from release packages. Since the Drupal.org packaging system already uses git-archive to create releases, the change would involve nothing more than the addition of a few lines to our .gitattributes file and would take effect immediately. Note: This would have no effect on Git clones, so neither the testbot nor core contributors would be affected.

Remaining tasks

• Consider possible unintended consequences.
• Review/test (forthcoming) patch.
• Probably write a change notice.
• Decide when to commit/release the change.

User interface changes

If tests can be considered to belong to the user interface in any sense, the change is that they will disappear from tarballs.

Release notes snippet

TBD.

I have a content-type with an address field on it. The field is optional. I cannot send null as a value for that field or I get a 500 internal server error.

This happens on create (POST) or update (PATCH). I would expect to be able to remove such an optional field value from a node by sending a null value in a request.

I can successfully create a node if I pass in a valid address field or do not supply one at all. BUT in the case that I don't supply a value then the serialized response from the POST will contain the field attribute with a value of null. I would expect that if I changed an attribute value in such a result and send the content the server gave me directly back for an update (PATCH) it would succeed (say I just change the title of the node, the server should be able to deserialize what it serialized to me) but I can't because the server result contains null so my client needs to know to delete the field if the property is set and has a null value to avoid the 500 internal server error.

At this point it seems I simply cannot remove an optional address from the node since I cannot send an empty address (the server will complain that is invalid) and I cannot send null to have it completely removed. I haven't yet tried deleting the property on PATCH but this would go against the grain of the verb being a sparse UPDATE.

I.e. per the JSON API specification for PATCH

If a request does not include all of the attributes for a resource, the server MUST interpret the missing attributes as if they were included with their current values. The server MUST NOT interpret missing attributes as null values.

The stack trace:

Error: Call to a member function getClass() on null in /drupal/web/modules/contrib/jsonapi/src/Normalizer/FieldItemNormalizer.php on line 117 #0 /drupal/vendor/symfony/serializer/Serializer.php(182): Drupal\\jsonapi\\Normalizer\\FieldItemNormalizer->denormalize(NULL, 'Drupal\\\\address\\\\...', 'api_json', Array)\n#1 /drupal/web/modules/contrib/jsonapi/src/Serializer/Serializer.php(75): Symfony\\Component\\Serializer\\Serializer->denormalize(NULL, 'Drupal\\\\address\\\\...', 'api_json', Array)\n#2 /drupal/web/modules/contrib/jsonapi/src/Normalizer/FieldNormalizer.php(65): Drupal\\jsonapi\\Serializer\\Serializer->denormalize(NULL, 'Drupal\\\\address\\\\...', 'api_json', Array)\n#3 /drupal/vendor/symfony/serializer/Serializer.php(182): Drupal\\jsonapi\\Normalizer\\FieldNormalizer->denormalize(NULL, '\\\\Drupal\\\\Core\\\\Fi...', 'api_json', Array)\n#4 /drupal/web/modules/contrib/jsonapi/src/Serializer/Serializer.php(75): Symfony\\Component\\Serializer\\Serializer->denormalize(NULL, '\\\\Drupal\\\\Core\\\\Fi...', 'api_json', Array)\n#5 /drupal/web/modules/contrib/jsonapi/src/Normalizer/ContentEntityDenormalizer.php(77): Drupal\\jsonapi\\Serializer\\Serializer->denormalize(NULL, '\\\\Drupal\\\\Core\\\\Fi...', 'api_json', Array)\n#6 /drupal/web/modules/contrib/jsonapi/src/Normalizer/EntityDenormalizerBase.php(99): Drupal\\jsonapi\\Normalizer\\ContentEntityDenormalizer->prepareInput(Array, Object(Drupal\\jsonapi\\ResourceType\\ResourceType), 'api_json', Array)\n#7 /drupal/vendor/symfony/serializer/Serializer.php(182): Drupal\\jsonapi\\Normalizer\\EntityDenormalizerBase->denormalize(Array, 'Drupal\\\\node\\\\Ent...', 'api_json', Array)\n#8 /drupal/web/modules/contrib/jsonapi/src/Serializer/Serializer.php(75): Symfony\\Component\\Serializer\\Serializer->denormalize(Array, 'Drupal\\\\node\\\\Ent...', 'api_json', Array)\n#9 /drupal/web/modules/contrib/jsonapi/src/Normalizer/JsonApiDocumentTopLevelNormalizer.php(169): Drupal\\jsonapi\\Serializer\\Serializer->denormalize(Array, 'Drupal\\\\node\\\\Ent...', 'api_json', Array)\n#10 /drupal/vendor/symfony/serializer/Serializer.php(182): Drupal\\jsonapi\\Normalizer\\JsonApiDocumentTopLevelNormalizer->denormalize(Array, 'Drupal\\\\node\\\\Ent...', 'api_json', Array)\n#11 /drupal/web/modules/contrib/jsonapi/src/Serializer/Serializer.php(75): Symfony\\Component\\Serializer\\Serializer->denormalize(Array, 'Drupal\\\\jsonapi\\\\...', 'api_json', Array)\n#12 /drupal/web/modules/contrib/jsonapi/src/Controller/EntityResource.php(820): Drupal\\jsonapi\\Serializer\\Serializer->denormalize(Array, 'Drupal\\\\jsonapi\\\\...', 'api_json', Array)\n#13 /drupal/web/modules/contrib/jsonapi/src/Controller/EntityResource.php(306): Drupal\\jsonapi\\Controller\\EntityResource->deserialize(Object(Drupal\\jsonapi\\ResourceType\\ResourceType), Object(Symfony\\Component\\HttpFoundation\\Request), 'Drupal\\\\jsonapi\\\\...')\n#14 [internal function]: Drupal\\jsonapi\\Controller\\EntityResource->patchIndividual(Object(Drupal\\jsonapi\\ResourceType\\ResourceType), Object(Drupal\\node\\Entity\\Node), Object(Symfony\\Component\\HttpFoundation\\Request))\n#15 /drupal/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array)\n#16 /drupal/web/core/lib/Drupal/Core/Render/Renderer.php(582): Drupal\\Core\\EventSubscriber\\EarlyRenderingControllerWrapperSubscriber->Drupal\\Core\\EventSubscriber\\{closure}()\n#17 /drupal/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\\Core\\Render\\Renderer->executeInRenderContext(Object(Drupal\\Core\\Render\\RenderContext), Object(Closure))\n#18 /drupal/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\\Core\\EventSubscriber\\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array)\n#19 /drupal/vendor/symfony/http-kernel/HttpKernel.php(151): Drupal\\Core\\EventSubscriber\\EarlyRenderingControllerWrapperSubscriber->Drupal\\Core\\EventSubscriber\\{closure}()\n#20 /drupal/vendor/symfony/http-kernel/HttpKernel.php(68): Symfony\\Component\\HttpKernel\\HttpKernel->handleRaw(Object(Symfony\\Component\\HttpFoundation\\Request), 1)\n#21 /drupal/web/core/lib/Drupal/Core/StackMiddleware/Session.php(57): Symfony\\Component\\HttpKernel\\HttpKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#22 /drupal/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(47): Drupal\\Core\\StackMiddleware\\Session->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#23 /drupal/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(99): Drupal\\Core\\StackMiddleware\\KernelPreHandle->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#24 /drupal/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(78): Drupal\\page_cache\\StackMiddleware\\PageCache->pass(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#25 /drupal/web/modules/contrib/jsonapi/src/StackMiddleware/FormatSetter.php(45): Drupal\\page_cache\\StackMiddleware\\PageCache->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#26 /drupal/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(47): Drupal\\jsonapi\\StackMiddleware\\FormatSetter->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#27 /drupal/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(52): Drupal\\Core\\StackMiddleware\\ReverseProxyMiddleware->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#28 /drupal/vendor/stack/builder/src/Stack/StackedHttpKernel.php(23): Drupal\\Core\\StackMiddleware\\NegotiationMiddleware->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#29 /drupal/web/core/lib/Drupal/Core/DrupalKernel.php(693): Stack\\StackedHttpKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request), 1, true)\n#30 /drupal/web/index.php(19): Drupal\\Core\\DrupalKernel->handle(Object(Symfony\\Component\\HttpFoundation\\Request))

I'm upgrading from 1x and I must say I'm actually running into a lot of 500 internal server errors relating to property definitions within complex fields. So far I've been adding debug code to FieldItemNormalizer to figure out what's causing the server to choke and working around issues like this.

In this case if I add a few dd statements in the offending code:

public function denormalize($data, $class, $format = NULL, array $context = []) {
...
dd($class);
    if (!is_array($data)) {
      $property_value = $data;
      $property_name = $item_definition->getMainPropertyName();
dd($property_name.'='.isset($property_definitions[$property_name]));
      $property_value_class = $property_definitions[$property_name]->getClass();
      return $denormalize_property($property_name, $property_value, $property_value_class, $format, $context);
    }
...
}

The resulting output is:
Drupal\address\Plugin\Field\FieldType\AddressItem
followed by
=
so $property_name is empty as is its value in $property_definitions.

Problem/Motivation

The current way of associating a revision with a workspace has a couple of problems:

- the association history is lost when a workspace is published to Live, because we delete all entries from the workspace_association tables
- when we introduce workspace hierarchies, operations on the workspace_association entities will be either 1) very costly, because we have to update all the associations of a workspace as well as its descendants, or 2) bypass the Entity API completely, which makes the current implementation completely unnecessary overhead :)

Proposed resolution

- track the workspace in which a revision was created in a (revision metadata) base field
- convert the workspace_association entity type to a custom index table

Remaining tasks

Discuss, agree on the proposed resolution, write upgrade path and tests.

User interface changes

Nope.

API changes

The workspace_association will be a service in the container instead of an entity type.

Data model changes

All entity types supported by Workspaces will have an additional revision metadata (reference) field: workspace

Release notes snippet

TBD.

Problem/Motivation

As pointed out in #2784921-112: Add Workspaces experimental module point 1, having a Live (default) workspace entity is not desirable. It's also problematic when uninstalling the Workspace module, as can be seen in #3059824: TypeError: Argument 1 passed to Drupal\workspaces\WorkspaceListBuilder::buildRow() must implement interface Drupal\Core\Entity\EntityInterface, null given.

Proposed resolution

Remove the Live (default) workspace and change the concept model for switching to the Live version of the site by switching out of the current workspace.

Remaining tasks

Reviews.

User interface changes

The workspace switcher block does not have Live as an option anymore, and a Switch to Live action button has been added instead.

Before - in the Live version of the site

Before - in the Stage workspace

After - in the Live version of the site

After - in the Stage workspace

API changes

API additions:

- three new methods on WorkspaceManagerInterface: hasActiveWorkspace(), switchToLive() and executeOutsideWorkspace().
- one new method on WorkspaceNegotiatorInterface: unsetActiveWorkspace()

Data model changes

There is no 'live' workspace entity anymore.

Release notes snippet

There is no concept of a "Live" (default) workspace anymore. Switching to the live version of the site is now accomplished by switching out of the current workspace.

I have extended LayoutDefault to add an 'additional classes' option as per https://www.drupal.org/docs/8/api/layout-api/how-to-register-layouts#usi...
Within validateConfigurationForm any set errors will not appear due to the lack of a status messages section.
I'm thinking should LayoutDefault handle form validation errors?

Problem/Motivation

• https://www.drupal.org/docs/8/system-requirements/browser-requirements lists some extremely outdated browsers, such as Firefox 5 and Safari 5. It's unlikely that any core developers or testers test on these browsers. The browser vendors have long ago abandoned security support for them, so for a core developer or tester to even install one of them potentially opens their computer up to a security vulnerability.
• In Drupal 8.4, we updated to jQuery 3: #2533498: Update jQuery to version 3. https://jquery.com/upgrade-guide/3.0/#browser-support lists what is officially supported by that, which is e.g., only Current and (Current - 1) of Firefox and Safari.

Proposed resolution

1. Update https://www.drupal.org/docs/8/system-requirements/browser-requirements to:
   • Current and current -1 for Google Chrome, Firefox, Safari, Microsoft Edge, and Opera
   • Internet Explorer 11
   • Current for Safari mobile, Chrome for iPhone, Chrome for Android, Android browser, UC browser, and Samsung Internet
2. Clarify what our browser support means.
   • By default, Drupal core is not accepting bug fixes to unsupported browsers since we shouldn't expect committers or other contributors to run unsupported browsers. Committers and the security team will asses the risk case by case basis and make exceptions to this rule when needed.
   • Drupal can start using any given browser feature after all the supported browsers have either native support or a polyfill. Polyfills can be removed after all supported browser support given feature natively.

Remaining tasks

• Agree (or not) on above.

Problem/Motivation

WS-2018-0072 high severity
Vulnerable versions:< 2.2.0
Patched version: 2.2.0
Versions of https-proxy-agent before 2.2.0 are vulnerable to a denial of service. This is due to unsanitized options (proxy.auth) being passed to Buffer().

As per @lauriii this can be fixed publically.

it could be public because it only affects our development tooling, and it’s also already public information since its in a 3rd party dependency

Proposed resolution

Update the yarn package.

Remaining tasks

Create a patch

User interface changes

N/A

API changes

N/A

Data model changes

N/A

Release notes snippet

Updated nightwatch to version 1.2.1
Updated chromedriver to version 75.1.0
Updated stylelint-no-browser-hacks to 1.2.1

Problem/Motivation

Path aliases are currently represented as arrays. While this works OK it also comes with certain limitations. It is harder to document and extend. It is also prone to misuses/hacks.

Proposed resolution

We would like to have path aliases represented as objects with properly defined structure and business logic. It seems that going all the way to full-features content entities takes almost the same amount of effort while giving us even more goodies and flexibility.

Note about performance

Operating with an entity is heavier than just sending arrays around. This patch doesn't change any of the queries that happen in bootstrap phase (which is most sensitive to performance regressions). All the queries (even functions) stay the same. Same is true for URL generation.

Remaining tasks

- review patch

User interface changes

None.

API changes

Path alias arrays are converted to "path_alias" revisionable content entities

Posthumously we have been required to add a boolean "hide from front end" field to our article content type as a number of articles need to be hidden from view yet still available to search engines. We have subsequently found that due to the field not having been set any combination of filtering on this field results in no articles being shown rather than just the ones which have been specifically ticked.

My thinking is that if someone was to choose "Is not equal to True" then this should only hide the fields which have been ticked whereas it is also hiding the articles which have yet to have this field set.

This to me looks to be a bug, so hoping someone can help with this?