Setting Up Multiple RabbitMQ Queues in Symfony with enqueue-bundle

If your Symfony project needs to process messages using different queues in RabbitMQ, the enqueue/enqueue-bundle library offers an elegant and flexible way to do it. This article walks you through the basic setup of multiple queues and how to process them with specific processors.

1. Basic configuration of the enqueue library

In your configuration file (e.g. config/packages/enqueue.yaml), you need to define the transport and the client. To connect to RabbitMQ, we use a DSN stored in an environment variable. We also define the default queue and other client parameters:

enqueue:
    default:
        transport: '%env(resolve:ENQUEUE_DSN)%'
        client:
            app_name: "%enqueue_app_name%"
            default_queue: "%enqueue_default_queue%"
            redelivered_delay_time: 5
            router_topic: default
            router_queue: router
            router_processor: null
        job: true

Note: Set the values of enqueue_app_name and enqueue_default_queue to suit your needs in .env or .env.local.

2. Defining processors and queues

To route messages to different queues correctly, each processor has to implement the static method getSubscribedTopics, in which you define which topics and queues the processor handles. Here is an example:

use Enqueue\Client\TopicSubscriberInterface;

class TestProcessor implements TopicSubscriberInterface
{
    public function __invoke(Message $message, Context $context): string
    {
        // Process the message
        return self::ACK;
    }

    public static function getSubscribedTopics(): array
    {
        return [
            [
                'topic' => 'test.misc',
                'queue' => 'test_queue',
                'prefix_queue' => true,
            ],
        ];
    }
}

This way you can specify which messages should be routed to which queue and which processor will handle them.

3. Initializing queues and starting consumers

Once everything is configured, you need to set up the queues in RabbitMQ and start the consumers that will process the messages. Proceed as follows:

a) Creating all required queues

bin/console enqueue:setup-broker

This command creates all queues, including the routing queue (router), which is responsible for distributing messages to the correct target queues.

b) Starting the router consumer

The router consumer listens on the router queue and routes messages according to the rules defined in getSubscribedTopics() of the individual processors:

bin/console enqueue:consume router --time-limit="now + 300 seconds"

Tip: We recommend running this process separately as a daemon, e.g. using Supervisor or systemd.

c) Starting consumers for individual queues

Start a separate consumer for each specific queue. Example for the test_queue queue:

bin/console enqueue:consume test_queue --time-limit="now + 300 seconds"

This consumer will only process messages routed to the test_queue queue.

Summary

With the enqueue-bundle library, you can easily set up multiple RabbitMQ queues in Symfony, assign specific processors to them and scale the processing of asynchronous tasks efficiently. This approach works especially well for projects with higher message volumes or with different types of processing.