Core migration issues
Next video meeting 2025-01-02 (tentative)
Hello all, it’s time for the biweekly migration subsystem meeting. The meeting will take place in slack in various threads. This meeting:
➤ Is for core migrate maintainers and developers and anybody else in the community with an interest in migrations
➤ Usually happens every second Thursday and alternates between 1400 and 2100 UTC.
➤ Is done on the #migration channel in Drupal Slack (see www.drupal.org/slack for information).
➤ Happens in threads, which you can follow to be notified of new replies even if you don’t comment in the thread. You may also join the meeting later and participate asynchronously!
➤ Has a public agenda anyone can add to. #3492076: [meeting] Migrate Meeting 2024-12-05. See the parent issue for an idea of the typical agenda.
➤Transcript will be exported and posted to the agenda issue. For anonymous comments, start with a 👤 emoji. To take a comment or thread off the record, start with a 🚫 emoji.
0️⃣ Who is here today?
| alison | Hello! Just lurking but wanted to say hi. I just got to go to The Lion King with my 4 year old niblings, and it was my first time seeing it (the show not the movie), and, ohmygod. I'm on a high :laughing: |
| benjifisher | My day job calls, but I will check in later. |
| dinarcon | Hello :wave: |
| Dan Davis | Hey |
1️⃣ What should we talk about today? Suggest topics here and I will add threads. I will also check for comments on the issue for today's meeting.
| benjifisher | Convert MigrateSource plugin discovery to attribute[#3421014] |
| benjifisher | A recipe for D7 migrations (vaporware) |
| dinarcon | A bit late, but hoping it is ok to add to the agenda. Source plugin fields... |
2️⃣ Action items. To be added later.
3️⃣ Statistics
4️⃣ Comment in this thread if you are looking for ways to contribute. Give us some idea of what you would like to do: documentation, code review, testing, project management, ...
5️⃣ Previous minutes.
6️⃣ Announcements
7️⃣ Convert MigrateSource plugin discovery to attributes
| Migrate Initiative Meeting | The first step is to move I18nQueryTrait to migrate_drupal. |
| Migrate Initiative Meeting | (Original Request) https://drupal.slack.com/archives/C226VLXBP/p1733434174887399?thread_ts=... (edited) |
| benjifisher | https://www.drupal.org/project/drupal/issues/3258581 |
| benjifisher | One question: how do you test that a trait is not available (since the module that contains it is not enabled)? It seems that automated tests (at least kernel tests) autoload even files from uninstalled modules. !@#$% |
| benjifisher | Second: can we just skip deprecation messages since we will be removing all of migrate_drupal in D12? |
| benjifisher | Third: We want to move the source_module attribute to migrate_drupal. If @godotislate and I promise to follow up on that, can we leave it to a follow-up issue? |
| quietone | Added a Functional test to the MR. It works find locally but the 'test only' tests does not fail where it should. |
8️⃣ A recipe for D7 migrations (vaporware)
| Migrate Initiative Meeting | Should we create such a thing? |
| Migrate Initiative Meeting | (Original Request) https://drupal.slack.com/archives/C226VLXBP/p1733436905750739?thread_ts=... (edited) |
| benjifisher | Now that 3 companies are offering long-term support for D7, it seems likely that we will be doing migrations to modern Drupal for a long time. At least until D11 reaches EOL. |
| benjifisher | Why not create a recipe to help people get started on their migration projects? |
| benjifisher | What modules do we want to install?
migrate_toolsmigrate_plusmigrate_develmigrate_sandboxyaml_editor (for migrate_sandbox)
|
| benjifisher | Can a recipe require drush/drush? |
| dinarcon | How would that be different to using Migrate Upgrade to generate migrations? Will there be any limits as to what is offered? Entity type conversions might be tricky to implement in a generic way. |
| benjifisher | Why the :confused: reaction? Do you think that long-term support means people will continue paying for that support indefinitely and never migrate? |
| benjifisher | We could add that to the list:
|
| dinarcon | Recipes can require any composer package in their own composer.json file. Not sure if requiring non drupal/* packages would be an "anti-pattern", but it is indeed possible. |
| dinarcon | FWIW, https://www.drupal.org/project/migrate_wizard already offers a UI with extra customizations for an upgrade from D7. Maybe expand on that concept? |
| benjifisher | Maybe. |
| benjifisher | I have not looked at that module. |
| benjifisher | If a recipe will help people get started, then I think it is worth it. |
| dinarcon | Benji, have you got any feedback about https://www.drupal.org/project/import_csv ? Or anything you would like to share regarding creating migration related recipes? I know there are plans to ask for user input as part of recipe application. It is already possible via Drush, but not from the UI yet. |
| benjifisher | No feedback so far. |
| benjifisher | BTW, what the import_csv recipe is missing is a way to define migrations in the UI. It is too bad that https://www.drupal.org/project/feeds_migrate never got to a usable state. |
| benjifisher | Technically, an ambitious site builder can use /admin/config/development/configuration/single/import to define migrations. But we want something more like ECA. (edited) |
9️⃣ Source plugin fields
| dinarcon | Wondering why some source plugins do not report all the fields they fetch in their \Drupal\migrate\Plugin\MigrateSourceInterface::fields implementation? (edited) |
| dinarcon | For example, d7_user does not report the changed property in its \Drupal\user\Plugin\migrate\source\d7\User::fields That is available as part of D7's users table and the \Drupal\user\Plugin\migrate\source\d7\User::query implementation fetches all columns in users |
| dinarcon | d7_node does not report nr.vid , nr.log , and nr.timestamp even though those fields are retrieved by \Drupal\node\Plugin\migrate\source\d7\Node::query |
| dinarcon | d7_node_revision which extends d7_node exposes vid, log, and timestamp . While d7_node_revision changes the JOIN condition, these 3 fields are already available in d7_node , but they are not reported by \Drupal\node\Plugin\migrate\source\d7\Node::fields |
| quietone | Speaking for the source plugins I wrote, my attention was on the migration of the data, not the accuracy of the fields method. |
| dinarcon | For context, I noticed this when trying to come up with a field to use as high_water_property in a user migration. I thought that users.changed was a good candidate to include updates to users in an incremental migration. That is when I noticed that changed is not listed in \Drupal\user\Plugin\migrate\source\d7\User::fields nor included in the web/core/modules/user/migrations/d7_user.yml migration plugin. I could add changed: changed to process section of the (derived) upgrade_d7_user migration and it would work to set that property. I was curious if not listing/mapping users.changed was intentional or not. And then I noticed other fields missing from their \Drupal\migrate\Plugin\MigrateSourceInterface::fields implementation. (edited) |
| dinarcon | I know that[#2630732] is doing some introspection to not only list base properties, but also bundle fields, as part of the destination fields report. Not sure if that is possible for source plugins, but that might be out too much? That said, some source plugins that extend \Drupal\migrate_drupal\Plugin\migrate\source\d7\FieldableEntity get field API data in \Drupal\migrate\Plugin\MigrateSourceInterface::prepareRow. Example: https://git.drupalcode.org/project/drupal/-/blob/11.0.9/core/modules/nod... |