Catalog Service Publication Topologies

This guide describes safe scaling patterns for the Catalog Service publication consumer and explains how to avoid situations where catalog instances "steal" messages from each other.

Purpose and Problem Statement

The Catalog Service consumes publication events (for example media_service.movie.published) and updates its database. In RabbitMQ, message delivery is queue-based:

• An exchange routes by routing key.
• A consumer reads from an inbox queue.
• A logical consumer can run on one or many service instances.

If you operate independent catalog targets but let them all consume from the same queue, messages are distributed across those instances. The result is split datasets and inconsistent catalogs across targets.

Publishing Recap

In the standard flow:

1. The Media Service publishes events to RabbitMQ.
2. The Catalog Service subscribes to these events.
3. Incoming RabbitMQ messages are stored through the transactional inbox pattern.
4. Handlers process inbox messages and apply updates to the Catalog database.

For background on this model, see Messaging.

Option A: Shared Queue Workers (Scale One Catalog Target)

Use this when you need more throughput for a single logical Catalog target (one API + one dataset).

Topology

• Multiple Catalog service instances (pods)
• One shared inbox queue for that logical consumer
• One logical Catalog database target

Behavior

RabbitMQ load-balances queue messages across instances. Each message is handled once for this logical consumer. This is the standard horizontal scaling model.

Requirements

• All instances use the same queue/binding configuration.
• Consumer instances are stateless.
• Message handlers are idempotent to tolerate retries/redeliveries.

Option B: Isolated Catalog Targets with Dedicated Queues

Use this when you need multiple independent publication targets, each with its own complete dataset.

Topology

• One queue binding per target
• One Catalog deployment per target
• One isolated persistence target (database) per target

Behavior

Each target queue receives the full subscribed publication stream independently. No target loses events due to queue sharing with other targets.

Requirements

• Unique queue per target.
• Isolated persistence per target.
• Explicit ownership of bindings and deployment config per target.

Anti-patterns

The following patterns are not recommended:

• Multiple independent Catalog targets sharing one queue (causes message splitting between targets).
• Multiple consumers writing different targets from one shared queue without a deterministic partitioning strategy.
• Assuming retries/redelivery mechanisms can correct topology/configuration mistakes.

Decision Matrix

Goal	Recommended option
Scale one catalog target	Option A: Shared Queue Workers
Maintain multiple independent catalog targets	Option B: Isolated Targets with Dedicated Queues

Operational Checklist

Before and after scaling, check:

1. Queue naming is explicit and target-specific where required.
2. Queue bindings match the intended topology (shared vs isolated).
3. Deployments have correct queue and DB configuration for each target.
4. Published entity counts match expected parity after rollout.
5. Spot-check a sample of entities across targets.
6. Monitor dead-letter queues and retry behavior.

Troubleshooting

Symptom

Target A is missing movies or shows that exist in Target B.

Likely Cause

Independent targets are sharing a single queue, so events were split across targets.

Fix

1. Create dedicated queue bindings per target.
2. Ensure each target has isolated Catalog persistence.
3. Reprocess missing publications (re-publish snapshots) for consistency.