Trying to migrate from Drupal 6 to Drupal 8. Running migration through UI.
- Source site runs Pressflow 6.44 LTS
- Destination site runs Drupal 8.5.6
- Source and destination sites are set to a PHP memory limit of 512M
- Source site has currently 123,598 nodes
- Server is a Xeon D-1520 with 2.2/2.7 GHz and 4/8t, 64 GB RAM
- Operating system is Debian 8
The expected behaviour is that the migration system manages to migrate a Drupal 6 site within a reasonable timeframe, let's say max. 1-2 days. Also I'm expecting usable progress indications in the UI. If the migration systemm has not dies, I expect to be able to see any kind of progress.
The experienced behaviour is that migration simply takes too long or does not work at all. I'm getting a progress status with "33%" which does not change for the past three days (!). According to the output at ./batch?id=13&op=start
, it is processing URL aliases and has processed "63 of a total of 188 tasks". Like the statement in %, the tasks feedback has not change withing the past three days. Altogether, this migration is running for ~5 days.
This is an undesirable behaviour. At least every couple of hours something visible needs to indicate if migration is still in progress. If migration does not progress by 1% within a couple of hours, another UI feedback is required, be it 0.1% steps or a progress indication for a subroutine (33% of the whole migration … 78% of migration script foo). If I'm looking for three days at 33% progress, the currently implemented feedback indicator is simply useless or broken.
It's not that migration had completely died; according to the 'mytop' utility, the database server gets a SQL query every 1-2 minutes. This has slowed down significantly within the past days. At the beginning, the databse server was continously running SQL queries on the destination database. I believe this to be another bug as the migration subsystem seems not to be able to process any site with more than a couple of thousend nodes respectively taxonomy terms. It simply does not scale.
The steps required to reproduce the bug(s) are relatively simple - just try to migrate a medium-sized Drupal site (medium sized == number of nodes and/or taxonomy terms > 100,000; a large site would be something like d.o with content of several 100k). You can generate dummy content with devel, if you do not have access to medium to large Drupal sites in the wild. If someone bothers trying to reproduce, you will notice that with increasing amount of content all migrations procedures slow down to a crawl.
I have attempted a couple of migration attempts mit various real-world medium-sized sites, and without exception ran into dead ends like this. It's not a fixed failure at exactly 33% or at "63 of a total of 188 tasks" or at URL alias processing, but migration simply never finishes.
As a side note, I am aware that there is the (theoretical) alternative to run migrations through drush. I have tried this, too, but it also fails (different issue, but it does not work at all, especially it does not work to process separate migrations scripts as described in the docs).