Quantcast
Channel: Issues for Drupal core
Viewing all 293641 articles
Browse latest View live

SharedTempStoreFactory and PrivateTempStoreFactory $expire variable should be changeable for each collection

$
0
0

Right now, the $expire variable in PrivateTempStore.php isn't exposed anywhere, except as a parameter in user.services.yml. This gives developers the ability to override the default expiration on private tempstore collections, but it's still a one-size-fits-all value for all collections that are created.

However, there are many cases where one might want different tempstores to have different $expire values. For example, I'm using a private tempstore in a custom module to hold a list of external HTTP_REFERER values for each user who visits the site via an external link. Then, when a user fills out the contact form, those values are submitted via a hidden field so we can track our conversion rates from different sources. Naturally, we want that collection to last several months at least, since a user might find the site via an ad, but not fill out the contact form for some time afterward.

The same site might also use a private tempstore for a multi-step form, in which case it might be useful for the tempstore to only last a few hours. And so on.

What I'd really like to see is a public function setExpire($expire) in PrivateTempStore.php that lets developers manually set the expire value on a per-collection basis in their custom modules. Or perhaps an $expire argument with a default value added to the get() function that's already there.


Deprecate \Drupal\Component\FileSystem\FileSystem::getOsTemporaryDirectory() in favour of php://temp

$
0
0

Problem/Motivation

In #3039026-67: Deprecate file_directory_temp() and move to FileSystem service which adds "fallback" to configurable temporary dir the last fallback to this component which, according to https://www.php.net/manual/en/wrappers.php.php#wrappers.php.memory could be replaced with default stream wrapper
The whole component trying to maintain multi-os compatibility instead of using php defaults
The logic split into component & service could be simplified

Proposed resolution

Deprecate \Drupal\Component\FileSystem component
- replace getOsTemporaryDirectory() usage with php://temp
- find better place for \Drupal\Component\FileSystem\RegexDirectoryIterator which is used by only protected method \Drupal\Component\Discovery\YamlDirectoryDiscovery::getDirectoryIterator()

Remaining tasks

- agree on a way to deprecate
- create patch
- add change record and follow-ups

User interface changes

no

API changes

deprecate \Drupal\Component\FileSystem\FileSystem::getOsTemporaryDirectory()

Data model changes

no

Release notes snippet

tbd

Contact form auto-reply message acts as a spam relay

$
0
0

Problem/Motivation

When a contact form's auto-reply message is enabled, spammers can put their message in the "Subject" field, put a 3rd-party email address in the "Your email address" field, and thereby use your Drupal site to send spam to other people. This is a kind of backscatter attack vector.

Steps to reproduce

  • Create a new Drupal 9.2.2 site using the Standard installation profile
  • On Administration > Structure > Contact forms > Website feedback, fill in some "Auto-reply" text
  • Add a CAPTCHA, honeypot, or other abuse protection (which will block some but not all spam attempts)
  • Deploy the site to production
  • Over time, spammers will discover your contact form and start using your Drupal site to relay spam to 3rd parties. Because Drupal populates the auto-reply email subject with the user-submitted contact form content, spammers will use the subject field to send their message to 3rd parties.
  • 3rd party recipients of the auto-reply spam will then report your Drupal site's email address as spam, which will potentially add it to email blocklists, making it less likely that legitimate email from your site will get through to people who actually want to receive it.
Spammer fills out contact form with their message in the "Subject" field, and a 3rd-party email address in the "Your email address" field3rd party receives unsolicited spam message

Proposed resolution

When the contact form auto-reply is enabled, Drupal should not include user-submitted content in the auto-reply email. Drupal could either use a predefined subject line (e.g. "[contact-form-name] site-name"), or provide a field into which the site maintainer can enter their own subject line. Then, when spammers attempt to use your Drupal site as a spam relay, 3rd parties would still receive unexpected emails, but those emails would not contain the spammer's message — thus removing the incentive for the spammer to exploit your site's contact forms.

Related: If #405338: Contact form: Add token functionaltiy is implemented, it should prevent using tokens containing user-submitted content, or (if tokens containing user-submitted content are permitted and used) show a warning stating that the contact form will potentially be used as a spam relay.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Duplicated joins in entity query

$
0
0

Entity query with condition and sorting by a field from the same table makes duplicated joins.

My investigating:
Class 'Drupal\Core\Entity\Query\Sql\Query' method getTables() always returns new 'Drupal\Core\Entity\Query\Sql\Tables' object.
In this case, the Tables object doesn't have actual state of 'entityTables' property, and adds duplicated joins.
Because ensureEntityTable() method needs correct 'entityTables' property, for building joins.
Method getTables() is called seperatly when sort is added and conditions are compiled (getTables() method can be called several times during conditions' compiling).

My propose:
I think, getTables() method should write Tables object into 'tables' property. And in next call it should return Tebles object from 'tables' property instead of creating new object.

Or I don't understand somethings, and don't see reason why getTables() returns new object.

image styles won't generate a file unless its in an <img> tag.

$
0
0

Problem/Motivation

Trying to use this css code, INSIDE of a TWIG template file.


        {% set large_bgimg = file_url(content.field_background_image|field_target_entity.field_media_image.entity.uri.value|image_style('1440x701')) %}
        {% set small_bgimg = file_url(content.field_background_image|field_target_entity.field_media_image.entity.uri.value|image_style('375x913')) %}

            <style>
                .c-frontpage-header__slide-image--image {
                    background-image:
                        url("{{ small_bgimg }}");
                }

                @media (min-width: 720px) {
                    .c-frontpage-header__slide-image--image {
                        background-image:
                            url("{{ large_bgimg }}");
                    }
                }
            </style>

having the image path dumped in to those styles, IN the DIV, does not generate the actual IMAGE derivation from the image style.

I have to do something ULTRA hacky, like this.

<img width="0" height="0" src="{{ small_bgimg }}" style="display: none; height: 0px; width: 0px;">
        <img width="0" height="0" src="{{ large_bgimg }}" style="display: none; height: 0px; width: 0px;">

To actually get the image to generate.

This will render, and hide the image, and then the background image in the code, renders properly.

Proposed resolution

How do I get derived images to generate for something NOT in an <img> element?

Thank you.

New translations for moderated nodes are not created in the initial workflow state

$
0
0

Problem/Motivation

We have defined a workflow with these states:

  1. Draft
  2. Needs Review
  3. Published

We have two roles:

  • author (Who creates content in draft and sets it to "needs review")
  • admin (Who can transition from "needs review" to "published")

We have applied the patch from comment #7 of issue #3020387: Moderation state is the same for all node's translations in edit page.

Steps to reproduce:

  1. An author has created a node in English, and sets it to "needs review".
  2. Now the author wants to add a german translation for this node. He goes to the Translate tab and clicks on the "Add" Button.
  3. The form where he can edit the node and add the german texts is displayed.
  4. Wen he clicks "Save" he gets the error message "Invalid state transition from Needs review to Needs review".

Logically that is correct, because the author role doesn't have the permission to edit nodes which are in the state "needs review".

But shouldn't new translations be treated as new nodes and therefore be created as "draft"?

Proposed resolution

Find a way to preserve the existing behavior for BC, but either document a way to override or configure this for new translations.

Option A

  • Have a simple toggle for the current behavior vs using the default moderation state.
  • UI is just a checkbox. Have to word it carefully to be clear.
  • Setting is a bool. ID and other details TBD based on UI.

Example UI:

[X] New translations use default moderation state.
(If not checked, new translations will use the current state of the default revision).

Option B

  • Have a simple toggle for the current behavior vs using the default moderation state (same as A)
  • UI is 2 radios.
  • Setting stores a string, ID would be 'translation_default_moderation_state_behavior' or so, with values like 'current' vs. 'default'.
  • In some UIs, explicit radios for different options can be more clear than a checkbox (if each choice requires a little explanation).

Example UI:

Moderation state for new translations:
(*) Default moderation state.
( ) Current moderation state of the default revision.

Option C

  • Store the explicit moderation state for new translations.
  • Setting ID would be 'translation_default_moderation_state' or so.
  • Allow it to have a magic value that means "the current moderation state of the default revision in the original language".
  • UI might be weird / complicated.
  • Would be more powerful for edge cases.

Remaining tasks

  • Decide on a BC method.
  • Implement the new setting.
  • Don't check if the user has access to this transition (because it should be treated as new).
  • Add a DB update for existing sites.
  • Add tests:
    1. what you see when creating a new translation (the bug)
    2. the upgrade path?
    3. the UI for this when creating a workflow?

User interface changes

TBD: A new setting when creating/editing a workflow.

API changes

Probably not.

Data model changes

TBD.

Release notes snippet

TBD.

Highlight non-stable statuses in admin/reports/status page, providing warnings when using non-"stable" extensions

$
0
0

Problem/Motivation

When #3124762: Add 'lifecycle' key to .info.yml files is committed, we will have three new non-stable theme states. At present there is a visual affordance to experimental themes at admin/reports/status. But nothing at present for obsolete and deprecated.

Proposed resolution

Promote the non-stable statuses in admin/reports/status page, optionally even visually

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

Unable to save '/' root path alias

$
0
0

Problem/Motivation

There is a very subtle bug in the alias storage from #2336597: Convert path aliases to full featured entities.

Steps to recreate

  • Create a site with the standard install profile.
  • disable the /node view
  • Create a node with the path alias '/'
  • Got to site settings and set the frontpage path to '/'

In 8.7 and prior you will see all of these succeed and the frontpage and '/' path of your site will be a node.
In 8.8 you will see all of these steps complete without errors but trying to view the node will be broken and the frontpage of your site will be a 404.

Description of the bug

In 8.7 the path validation looks like this which will be important to reference.

    // Trim the submitted value of whitespace and slashes.
    $alias = rtrim(trim($element['alias']['#value']), " \\/");
    if (!empty($alias)) {
      $form_state->setValueForElement($element['alias'], $alias);

      // Validate that the submitted alias does not exist yet.
      $is_exists = \Drupal::service('path.alias_storage')->aliasExists($alias, $element['langcode']['#value'], $element['source']['#value']);
      if ($is_exists) {
        $form_state->setError($element['alias'], t('The alias is already in use.'));
      }
    }

    if ($alias && $alias[0] !== '/') {
      $form_state->setError($element['alias'], t('The alias needs to start with a slash.'));
    }

In 8.8/8.9 the entity has a regex constraint of /^\//i and a preSave that does this:

    // Trim the alias value of whitespace and slashes. Ensure to not trim the
    // slash on the left side.
    $alias = rtrim(trim($this->getAlias()), "\\/");
    $this->setAlias($alias);

The patch converting to entities translated the trim logic and slash enforcement and a cursory comparison might look like the two doing the same thing. There's a subtle difference though when you pass '/' as the value.

In 8.6 we do the trims, and the rtrim() removes the right slash which in this case is also the left slash. But then we do this empty comparison and skip the rest of the validation because it passed form api's empty validation and a empty value means its "/" and everything is good. It doesn't also does not touch the form state, and the '/' gets saved. Inside the empty check for other paths we save the trimmed path to form state and everything is happy. This kind of complicated interaction between the validation and processing is why we like moving away from doing it but it worked.

In 8.8 '/' passes through all the validation but then when get to preSave we always trim the right '/' which leaves and empty string, which then saves as null in the database and... whoops the node's path is broken.

Proposed resolution

Fix the trim logic in presave to correctly handle '/'

Remaining tasks

User interface changes

N/A

API changes

N/A, this fixes a regression.

Data model changes

N/A


Fix Drupal.NamingConventions.ValidVariableName.LowerCamelName

$
0
0

Part of #2571965: [meta] Fix coding standards in core.

This sniff refers to https://www.drupal.org/docs/develop/standards/coding-standards#naming specifically Functions and variables.

This issue is now only doing the Test files, see #15.

Approach

We are testing coding standards with PHP CodeSniffer, using the Drupal coding standards from the Coder module. We need to do a couple of steps in order to download and configure them so we can run a coding standards check.

Step 1: Add the coding standard to the whitelist

Every coding standard is identified by a "sniff". For example, an imaginary coding standard that would require all llamas to be placed inside a square bracket fence would be called the "Drupal.NamingConventions.ValidVariableName.LowerCamelName sniff". There are dozens of such coding standards, and to make the work easier we have started by only whitelisting the sniffs that pass. For the moment all coding standards that are not yet fixed are simply skipped during the test.

Open the file core/phpcs.xml.dist and add a line for the sniff of this ticket. The sniff name is in the issue title. Make sure your patch will include the addition of this line.

Step 2: Install PHP CodeSniffer and the ruleset from the Coder module

$ composer install
$ ./vendor/bin/phpcs --config-set installed_paths ../../drupal/coder/coder_sniffer

Once you have installed the phpcs package, you can list all the sniffs available to you like this:

$ ./vendor/bin/phpcs --standard=Drupal -e

This will give you a big list of sniffs, and the Drupal-based ones should be present.

Step 3: Prepare the phpcs.xml file

To speed up the testing you should make a copy of the file phpcs.xml.dist (in the core/ folder) and save it as phpcs.xml. This is the configuration file for PHP CodeSniffer.

We only want this phpcs.xml file to specify the sniff we're interested in. So we need to remove all the rule items, and add only our own sniff's rule. Rule items look like this:

<rule ref="Drupal.Classes.UnusedUseStatement"/>

Remove all of them, and add only the sniff from this issue title. This will make sure that our tests run quickly, and are not going to contain any output from unrelated sniffs.

Step 4: Run the test

Now you are ready to run the test! From within the core/ folder, run the following command to launch the test:

$ cd core/
$ ../vendor/bin/phpcs -ps

This takes a couple of minutes. The -p flag shows the progress, so you have a bunch of nice dots to look at while it is running. The -s flag shows the sniffs when displaying results.

Step 5: Fix the failures

When the test is complete it will present you a list of all the files that contain violations of your sniff, and the line numbers where the violations occur. You could fix all of these manually, but thankfully phpcbf can fix many of them. You can call phpcbf like this:

$ ../vendor/bin/phpcbf

This will fix the errors in place. You can then make a diff of the changes using git. You can also re-run the test with phpcs and determine if that fixed all of them.

[meeting] Migrate Meeting 2021-07-29

$
0
0

Hello all, it’s time for the weekly migration subsystem meeting. The meeting will take place in slack in various threads
This meeting:
➤ Is for core migrate maintainers and developers and anybody else in the community with an interest in migrations
➤ Usually happens every Thursday and alternates between 1400 and 2100 UTC.
➤ Is done on the #migration channel in Drupal Slack (see www.drupal.org/slack for information).
➤ Happens in threads, which you can follow to be notified of new replies even if you don’t comment in the thread. You may also join the meeting later and participate asynchronously!
➤ Has a public agenda anyone can add to. See the parent issue for an idea of the typical agenda.
➤*Transcript will be exported and posted* to the agenda issue. For anonymous comments, start with a :bust_in_silhouette: emoji. To take a comment or thread off the record, start with a :no_entry_sign: emoji.

Core migration issues

Next video meeting

2021-10-14 - The hope is that most or all of the maintainers will attend. We will try to focus on longer-term goals than in the weekly meeting.

Agenda items

Add items here

[meeting] Migrate Meeting 2021-07-22

$
0
0

Hello all, it’s time for the weekly migration subsystem meeting. The meeting will take place in slack in various threads
This meeting:
➤ Is for core migrate maintainers and developers and anybody else in the community with an interest in migrations
➤ Usually happens every Thursday and alternates between 1400 and 2100 UTC.
➤ Is done on the #migration channel in Drupal Slack (see www.drupal.org/slack for information).
➤ Happens in threads, which you can follow to be notified of new replies even if you don’t comment in the thread. You may also join the meeting later and participate asynchronously!
➤ Has a public agenda anyone can add to. See the parent issue for an idea of the typical agenda.
➤*Transcript will be exported and posted* to the agenda issue. For anonymous comments, start with a :bust_in_silhouette: emoji. To take a comment or thread off the record, start with a :no_entry_sign: emoji.

Core migration issues

Next video meeting

2021-10-14 - The hope is that most or all of the maintainers will attend. We will try to focus on longer-term goals than in the weekly meeting.

Agenda items

Add items here

0️⃣ Who is here today?

Matroskeen:wave: Ivan, UA
benjifisher:wave: Benji: one of the migration maintainers. Massachussetts, USA.
benjifisherUsually I add an "icebreaker" question to this thread, but I have not thought of one yet.
thhafner:wave: Thomas, Chicago
benjifisherHi, @thhafner! Is this your first time at one of these meetings?
thhafneryes
benjifisherWe are glad to have you here! Is there something you want to bring up?
thhafnerRan into an interesting issue and would love to get the migration teams opinion.  #3224907: Uninstalling migrate_drupal_ui Does Not Remove Stored Migration Credentials From the Database
thhafnerWould also love to start contributing back to the Migrate team
benjifisherWhy not comment on 4️⃣ to talk about that?
mikelutz (he/him)Hello all.
quietoneVicki

1️⃣ What should we talk about today? Suggest topics here and I will add threads. I will also check for comments on the issue for today's meeting.

2️⃣ Action items. To be added later.

3️⃣ Statistics

benjifisherFixed since last week's meeting: 1 (not counting the issue for the meeting).
benjifisherRTBC: 1
benjifisherNR: 32, including 2 Major and 7 that have not been updated in more than three months.
benjifisherGoogle sheet for recording stats: https://docs.google.com/spreadsheets/d/1o0Rjlc1vnnLP5bM5P-SMMyGzqn7258hi...

4️⃣ How can I help? Comment in this thread if you are looking for ways to help. Give us some idea of what you would like to do: documentation, code review, testing, project management, ...

5️⃣ Previous minutes.

benjifisherIs there anything to follow up? Anything need to be changes in those minutes?
benjifisherLast week was the quarterly Zoom meeting, so we actually have a recording instead of minutes.
benjifisherFor some reason, I cannot assign credit on the issue. Can one of the other maintainers try? From memory, the following attended the meeting: @mikelutz, @heddn, @quietone, @benjifisher, @marvil07, @dinarcon, @srjosh.
quietoneAdded credit to #3223507: [meeting] Migrate Meeting 2021-07-15 and marked as fixed.

6️⃣ Add extra orderBy to d7_file source plugin to avoid random results

benjifisher#3220155: Add extra orderBy to d7_file source plugin to avoid random results
benjifisher@Matroskeen, you and I seem to be the only ones here today. Is there more to discuss about this issue? I think you already watched the recording of lastr week's meeting.
MatroskeenFirst of all, I suggested more generic approach in the last commits/comments - ordering by source IDs fields.I would like someone to validate before fixing failing tests._Secondly, it was pointed out that adding new order can affect the highwater marks. But what I'm seeing - it's not quite correct for current source plugins that already provides orderBy  In this case, order by highwater is coming 2nd, when the main query is already defined.I mentioned this in the last comment. (edited)
benjifisherI have not had a chance yet to look at the implementation, so I am just replying to your comments.IMO, the extra ORDER BY clause should come after any other sorting. That is the intention of the issue.I have not looked at how you implement it, but if it works as you say then :+1:.As long as it works that way, it should not break highwater marks, nor anything else.I have never looked at the code for highwater marks, but I have seen comments from others that it is buggy. If the ORDER BY clause added for the HW mark comes after other sorts, then that might explain some of the bugs. Have you looked for existing issues related to this?
benjifisher@mikelutz (he/him), are we missing anything here?
mikelutz (he/him)sounds about right. I will say that the implementation of highwater support in SqlBas  is designed for simple querys and having an orderby in a derivitive class will break that implementation. It’s important to note though that highwater is all done in the source plugin, there’s nothing in the general migration or executable classes that touch it, so if you have a more complex query but want to make highwater work, it’s certainly possible to override the initializeiterator and/or the highwater methods to account for more complex queries and keep them from breaking.
benjifisherI had a look at the MR.if the source plugin already contains orderBy by same ID field(s) - nothing changes, the second one will be ignored;Maybe I am reading too much into this claim. If a source plugin extends SqlBase and does the typicalpublic function query() { $query = parent::query(); $query->orderBy(...); return $query;}then it will not work. I thought your comment meant that you had arranged it so that the new orderBy() would come after the one in the child class.
Matroskeen@benjifisher, if the plugin overrides the query() method, then it becomes primary sorting order (if there is no order in parent class yet), or secondary if the order exists.The order by highwater or source IDs (suggested in my approach) is added later - in initializeIterator() method, when the query is already finalized and modified by source plugin. (edited)
MatroskeenAs for the highwater marks, maybe we should be more aggressive and remove existing orderBys if the highwater is specified, but that's another topic to discuss. (edited)
benjifisherMy mistake: I was looking at the MR in the web UI, and I did not click "Show 20 [more] lines" enough times to see what function was being modified.

7️⃣ Uninstalling migrate_drupal_ui Does Not Remove Stored Migration Credentials From the Database

benjifisher#3224907: Uninstalling migrate_drupal_ui Does Not Remove Stored Migration Credentials From the Database
benjifisherFrom the description, that looks like a bug to me.
benjifisherI see some discussion of this problem in this channel yesterday. In addition to what it says on the issue, @Matroskeen said,Try to look at:Drupal\migrate\Plugin\migrate\source\SqlBase::getDatabase()
benjifisherIt should be pretty easy to implement hook_uninstall() and remove the items from the key_value table.
thhafnerI am happy to work towards implementing a fix, just wanted to get the team’s opinion that this was, in fact, a bug.
benjifisherMission accomplished. :wink:
benjifisherWe will also want a test for the issue. I think it will need to be a functional test. Maybe we can add a few lines to an existing test:check that the credentials are storeduninstall migrate_drupal_uicheck that the credentials have been removed.The test cannot repeat the other "Steps to reproduce" from the issue description: Drupal core does not provide a way to run migrations from the CLI. You need Drush to do that.
thhafnerSounds good
dinarconI think this is somewhat related with these discussions...
dinarconhttps://drupal.slack.com/archives/C226VLXBP/p1619705689366300?thread_ts=...
dinarconhttps://drupal.slack.com/archives/C226VLXBP/p1586207291165500?thread_ts=...

8️⃣ Wrap up

benjifisherThanks for participating! I will update 2️⃣. Please continue to add comments in the threads. In 1-7 days, we will post a transcript for today's meeting.
MatroskeenThanks @benjifisher :raised_hands:

Participants:

benjifisher, quietone, Matroskeen, mikelutz (he/him), thhafner, dinarcon

Convert EntityViewsData to use the table mapping

$
0
0

Problem/Motivation

EntityViewsData should use the table mapping which is the generic abstraction from entity types to tables.

Proposed resolution

* Do it
* Change the unit tests

Remaining tasks

User interface changes

API changes

Data model changes

The d7_entity_reference_translation follow-up migration only runs for node translations

$
0
0

Problem/Motivation

This is based on original research by @huzooka.

The goal is to "fix/update" references to a non-default translation of a particular node: it makes them point to the correct node ID, because translations of nodes in Drupal 7 have a different node ID, whereas in Drupal 8|9 all translations have the same node ID. This "fixing/updating" works fine.

However, this fails to "fix/update" references to nodes if the referencing entity is not a node. Any fieldable entity type could have a reference to a node — for example users.

Steps to reproduce

Migrate any entity other than a node that references a translation of a node — that reference will be wrong after migrating.

Proposed resolution

Add a test to d7/FollowUpMigrationsTest.php to confirm that references to nodes in entities that are not nodes, such as users, are migrated correctly. That tests passes, so no other work is needed here.

Note: The deriver for the entity_reference_translation migration generates follow up migration for any content type with a field of 'entity_reference' type and the reference is to a nody type.

Remaining tasks

Commit

User interface changes

N/A

API changes

N/A

Data model changes

N/A

Release notes snippet

N/A

Service tag priority does not work with HTTP Client Middleware

$
0
0

Problem/Motivation

When a priority is added to tags named 'http_client_middleware', the priority is ignored. HTTP client middlewares should use standard service_id_collector so priorities work out of the box.

Steps to reproduce

Create two HTTP middlewares, define them like so:

  test.http_middleware1:
    class: Drupal\test\HttpMiddleware1
    tags:
      - { name: http_client_middleware, priority: -100 }

  test.http_middleware2:
    class: Drupal\test\HttpMiddleware2
    tags:
      - { name: http_client_middleware, priority: 100 }

You'll see the first middleware is executed first, since it's defined first. Instead the second middleware should be executed first since priorities work in highest->lowest order.

Proposed resolution

Deprecate \Drupal\Core\DependencyInjection\Compiler\GuzzleMiddlewarePass and replace with service_id_collector.

Release notes snippet

GuzzleMiddlewarePass has been removed in favor of the generic TaggedHandlersPass, code interacting with GuzzleMiddlewarePass should use TaggedHandlersPass instead.

Fix source plugin documentation

$
0
0

Problem/Motivation

In #3189481: [Meta] Add source plugin documentation to the codebase documentation was added for the migrate source plugins. However, a mistake was made which resulted in the api doc being hard to read. This issue is to correct that.

Proposed resolution

Change * For available configuration keys, refer to the parent classes: to
* For available configuration keys, refer to the parent classes. and add a blank line.

Remaining tasks

Patch
review
commit


Update changed link for /filter/tips

$
0
0

Problem/Motivation

The /filter/tips page has a link to http://www.w3.org/TR/html/ but this page is redirected to https://html.spec.whatwg.org/, resulting in a 301 error.

Steps to reproduce

Run an SEO check on any site with comments enabled. The 301 error should appear.

Proposed resolution

Change the link to https://html.spec.whatwg.org/ to eliminate the 301 error.

Remaining tasks

Change the link to https://html.spec.whatwg.org/

User interface changes

None

API changes

None

Data model changes

None

Release notes snippet

SQL source plugins: allow defining conditions and join in migration yml

$
0
0

Problem/Motivation

For custon migration it's a very common request to filter the migrated items. Some real use cases are:
- migrate nodes older than {date};
- skip blocked users or users with certain emails or users that never accessed the website;
- migrate users with certain roles;

It would be great to be able to use conditions to narrow down your SQL source data for content/config entity types. The resulting functionality would provide much more capabilities to how you can migrate nodes one content type at a time, and the same for taxonomy terms -- but for users (by role), menu links (by menu), and so on.

The functionality would let you narrow your data during the source/extraction phase, instead of during the process/transformation phase (which you can do using skip_on_value/empty + method: row). Benefits would include general tidiness of data :) And! Speed/performance!

Proposed resolution

Introduce new configuration properties in SqlBase base class: conditions, joins and distinct:
conditions - to filter the results;
joins - to join additional tables (required to filter by fields/properties stored in separate tables);
distinct - to avoid duplicated results when joining one-to-many records;

If such configuration is present, it's being applied to the query in prepareQuery method, when the base query is already defined.

Remaining tasks

  1. Review and validate the approach;
  2. Resolve #2833060: SqlBase::prepareQuery() should be called also on count, which is kind of a blocker (the source count is not updated after applying conditions, because the count() ignoring prepareQuery method);
  3. Add more tests;
  4. Write a change record;

TypedData 'Any' can't be normalized to array if it stores an object

$
0
0

Problem/Motivation

Like commerce's order,It use 'any' type data to store object:Adjustment, But the TypedDataNormalizer just return the Adjustment object, It cause it can't be encode, So it can't be serializad

First(laravel): Let's look at laravel's serialization
laravel's serialization has two steps:

  1. toArray() ====> corresponds to druapl's normalize
  2. toJson() ====> corresponds to drupal's encode

laravel's toArray() is recursive,it will recurs to check whether the child value is Arrayable
https://github.com/illuminate/database/blob/5.5/Eloquent/Model.php#L918
https://github.com/illuminate/database/blob/5.5/Eloquent/Concerns/HasAtt...

Second(symfony):
https://github.com/symfony/serializer/blob/3.2/Serializer.php#L138

    public function normalize($data, $format = null, array $context = array())
    {
        // If a normalizer supports the given data, use it
        if ($normalizer = $this->getNormalizer($data, $format)) {
            return $normalizer->normalize($data, $format, $context);
        }
        if (null === $data || is_scalar($data)) {
            return $data;
        }
        if (is_array($data) || $data instanceof \Traversable) {
            $normalized = array();
            foreach ($data as $key => $val) {
                $normalized[$key] = $this->normalize($val, $format, $context);
            }
            return $normalized;
        }
        if (is_object($data)) {
            if (!$this->normalizers) {
                throw new LogicException('You must register at least one normalizer to be able to normalize objects.');
            }
            throw new UnexpectedValueException(sprintf('Could not normalize object of type %s, no supporting normalizer found.', get_class($data)));
        }
        throw new UnexpectedValueException(sprintf('An unexpected value could not be normalized: %s', var_export($data, true)));
    }

symfony's Serializer->normalize() has the ability to recursive normalize object and array too

Both laravel and symfony has the ability to recursive normalize object or array

Then(Drupal):
Let's see TypedDataNormalizer used by normalizing 'any' type data
core/modules/serialization/src/Normalizer/TypedDataNormalizer.php#n21

  public function normalize($object, $format = NULL, array $context = []) {
    return $object->getValue();
  }

We can see TypedDataNormalizer Just return the value
Drupal use symfony's serializer, But it dosn't follow or take full advantage of the symfony's serializer's design

Proposed resolution

  1. make TypedDataNormalizer flow and use symfony's serialize->normalize()'s design. (solve here)
  2. Map type data has similar trouble too: But Map have other more base problem, It can't be normalized when it has no propertydefinition , But if TypedDataNormalizer's problem is solved, Two kind of https://www.drupal.org/node/2895532's the patch(two kind of controversial solution) will work good with no change needed

Remaining tasks

User interface changes

API changes

Data model changes

Use dataprovider for constructor test in ContentEntityTest

Fix 'Drupal.Commenting.DocComment.ShortSingleLine' coding standard

$
0
0

Part of meta-issue #2571965: [meta] Fix coding standards in core

Step 1: Preparation

Open the file core/phpcs.xml.dist and add a line for the sniff of this ticket. The sniff name is in the issue title. Make sure your patch will include the addition of this line.

Step 2: Install & configure PHPCS

Install PHP CodeSniffer and the ruleset from the Coder module:

$ composer install
$ ./vendor/bin/phpcs --config-set installed_paths ../../drupal/coder/coder_sniffer

Once you have installed the phpcs package, you can list all the sniffs available to you like this:

$ ./vendor/bin/phpcs --standard=Drupal -e

This will give you a big list of sniffs, and the Drupal-based ones should be present.

Step 3: Prepare the phpcs.xml file

To speed up the testing you should make a copy of the file phpcs.xml.dist (in the core/ folder) and save it as phpcs.xml. This is the configuration file for PHP CodeSniffer.

We only want this phpcs.xml file to specify the sniff we're interested in. So we need to remove all the rule items, and add only our own sniff's rule. Rule items look like this:

<rule ref="Drupal.Commenting.DocComment.ShortSingleLine"/>

Remove all of them, and add only the sniff from this issue title. This will make sure that our tests run quickly, and are not going to contain any output from unrelated sniffs.

Step 4: Run the test

Now you are ready to run the test! From within the core/ folder, run the following command to launch the test:

$ cd core/
$ ../vendor/bin/phpcs -p

This takes a couple of minutes. The -p flag shows the progress, so you have a bunch of nice dots to look at while it is running.

Step 5: Fix the failures

When the test is complete it will present you a list of all the files that contain violations of your sniff, and the line numbers where the violations occur. You could fix all of these manually, but thankfully phpcbf can fix many of them. You can call phpcbf like this:

$ ../vendor/bin/phpcbf

This will fix the errors in place. You can then make a diff of the changes using git. You can also re-run the test with phpcs and determine if that fixed all of them.

Viewing all 293641 articles
Browse latest View live