Quantcast
Channel: Issues for Drupal core
Viewing all articles
Browse latest Browse all 299308

Too many open files issue with migrating multiple files using "download" process plugin.

$
0
0

Problem/Motivation

When attempting to migrate image files from an external legacy website, I encountered the following error after processing about 1000 files:

[warning] include(.../docroot/core/modules/migrate/src/MigrateException.php): failed to open stream: Too many open files ClassLoader.php:444

Steps to reproduce

In my implementation, I used the transform method on the "download" process plugin from within another custom process plugin like so:

$uri = $this->downloadPlugin->transform([
$source,
$destination,
], $migrate_executable, $row, $destination_property);

The error should trigger when the Download plugin's transform method is called about 1000 times.

Proposed resolution

It appears the Download process plugin fails to close the stream it opens when it downloads the file. This eventually results in triggering the error from the ClassLoader.

For testing purposes, I tried modifying the transform method within the Download plugin class to close the stream before exiting:

public function transform($value, MigrateExecutableInterface $migrate_executable, Row $row, $destination_property) {

...

@fclose($destination_stream);
return $final_destination;
}

By doing this the error was no longer triggered. As such, I think this issue might be resolved by simply closing the open stream. However, since the stream is saved to the configuration as $this->configuration['guzzle options']['sink'] I'm not sure how this change would effect code, if any, that might later require the stream resource.


Viewing all articles
Browse latest Browse all 299308

Latest Images

Trending Articles



Latest Images

<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>