Problem/Motivation
The queues allow storing any serializable data (serialization is only a limitation for the non-memory queues, e.g. \Drupal\Core\Queue\DatabaseQueue
). Here is how the item creation can look:
class QueueItem {
public function __toString(): string {
return serialize($this);
}
}
\Drupal::queue('queue_name')->createItem(new QueueItem());
The issue occurs for the storage-based queues in case of throwing any of the following exceptions during the worker's processItem()
:
\Drupal\Core\Queue\DelayedRequeueException
\Drupal\Core\Queue\RequeueException
\Drupal\Core\Queue\SuspendQueueException
The in-memory queues do not serialize the instance of QueueItem
so it is always the same, can mutate inside the processItem()
and preserve the mutated state to the next attempt. The queues like \Drupal\Core\Queue\DatabaseQueue
serializes the instance of QueueItem
on creation and unserializes on claiming the item meaning the object state can be set only during its instantiation and cannot mutate across multiple attempts.
Steps to reproduce
-
Implement the class that will play the queue item.
namespace Drupal\my_module\Component\Queue; class QueueItem { public $processingAttempt = 1; public function __toString(): string { return serialize($this); } }
-
Implement the queue worker.
namespace Drupal\my_module\Plugin\QueueWorker; use Drupal\Core\Queue\DelayedRequeueException; use Drupal\my_module\Component\Queue\QueueItem; /** * @QueueWorker( * id = "queue_name", * title = @Translation("The Queue Worker"), * cron = { * "time" = 30, * }, * ) * * @see \Drupal\Core\Cron::processQueues() */ class TheQueueWorker extends QueueWorkerBase { public function processItem($item): void { assert($item instanceof QueueItem); // The default (initial) value is 1. if ($item->processingAttempt === 1) { // Increase the value to avoid throwing the exception. $item->processingAttempt++; throw new DelayedRequeueException(60); } } }
-
Create the queue item.
\Drupal::queue('queue_name')->createItem(new QueueItem());
-
Run the cron to start processing the queues. The item will disappear from the in-memory queues because its state mutates across attempts and stay forever in the database queues because the
processingAttempt
change is never committed.
Proposed resolution
Update the data
field in the \Drupal\Core\Queue\DatabaseQueue::releaseItem()
and \Drupal\Core\Queue\DatabaseQueue::delayItem()
(called when one of the mentioned exceptions occurs).