Quantcast
Channel: Issues for Drupal core
Viewing all articles
Browse latest Browse all 297834

Add a way to preserve queue item data mutations between processing attempts

$
0
0

Problem/Motivation

The queues allow storing any serializable data (serialization is only a limitation for the non-memory queues, e.g. \Drupal\Core\Queue\DatabaseQueue). Here is how the item creation can look:

class QueueItem {
  public function __toString(): string {
    return serialize($this);
  }
}

\Drupal::queue('queue_name')->createItem(new QueueItem());

The issue occurs for the storage-based queues in case of throwing any of the following exceptions during the worker's processItem():

  1. \Drupal\Core\Queue\DelayedRequeueException
  2. \Drupal\Core\Queue\RequeueException
  3. \Drupal\Core\Queue\SuspendQueueException

The in-memory queues do not serialize the instance of QueueItem so it is always the same, can mutate inside the processItem() and preserve the mutated state to the next attempt. The queues like \Drupal\Core\Queue\DatabaseQueue serializes the instance of QueueItem on creation and unserializes on claiming the item meaning the object state can be set only during its instantiation and cannot mutate across multiple attempts.

Steps to reproduce

  1. Implement the class that will play the queue item.
    namespace Drupal\my_module\Component\Queue;
    
    class QueueItem {
      public $processingAttempt = 1;
    
      public function __toString(): string {
        return serialize($this);
      }
    }
    
  2. Implement the queue worker.
    namespace Drupal\my_module\Plugin\QueueWorker;
    
    use Drupal\Core\Queue\DelayedRequeueException;
    use Drupal\my_module\Component\Queue\QueueItem;
    
    /**
     * @QueueWorker(
     *   id = "queue_name",
     *   title = @Translation("The Queue Worker"),
     *   cron = {
     *     "time" = 30,
     *   },
     * )
     *
     * @see \Drupal\Core\Cron::processQueues()
     */
    class TheQueueWorker extends QueueWorkerBase {
      public function processItem($item): void {
        assert($item instanceof QueueItem);
        // The default (initial) value is 1.
        if ($item->processingAttempt === 1) {
          // Increase the value to avoid throwing the exception.
          $item->processingAttempt++;
          throw new DelayedRequeueException(60);
        }
      }
    }
    
  3. Create the queue item.
    \Drupal::queue('queue_name')->createItem(new QueueItem());
    
  4. Run the cron to start processing the queues. The item will disappear from the in-memory queues because its state mutates across attempts and stay forever in the database queues because the processingAttempt change is never committed.

Proposed resolution

Update the data field in the \Drupal\Core\Queue\DatabaseQueue::releaseItem() and \Drupal\Core\Queue\DatabaseQueue::delayItem() (called when one of the mentioned exceptions occurs).

Remaining tasks

  1. #3184170: The `releaseItem()` and `delayItem()` of `Drupal\Core\Queue\DatabaseQueue` violates interfaces return type specifications

Viewing all articles
Browse latest Browse all 297834

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>