AI Rules for Pub/Sub Patterns

AI Calls Every Service Directly

AI generates inter-service communication with: direct HTTP calls from one service to each consumer (the order service calls the email service, the analytics service, and the inventory service sequentially), tight coupling (adding a new consumer requires modifying the publisher), knowledge of consumers (the publisher must know every service URL and API), sequential processing (each call waits for the previous — total time = sum of all calls), and no filtering (every consumer receives every message, even irrelevant ones). Adding one new consumer means changing the publisher code, deploying, and testing.

Pub/sub solves this by: broadcasting events to a topic (the publisher sends to a topic, not to specific services), fan-out delivery (all subscribers receive the event independently and concurrently), zero publisher coupling (the publisher does not know who subscribes — adding a subscriber requires zero publisher changes), message filtering (subscribers receive only events matching their filter criteria), and independent processing (each subscriber processes at its own pace). AI generates none of these.

These rules cover: topic design and naming, fan-out delivery patterns, message attribute filtering, at-least-once delivery semantics, subscription lifecycle management, and when to choose pub/sub over point-to-point queues.

Rule 1: Topic Design and Event Structure

The rule: 'Organize events into topics by domain: orders (order.created, order.shipped, order.cancelled), payments (payment.processed, payment.refunded), users (user.registered, user.updated). Each topic carries events of one domain. Event structure: { id: uuid, type: "order.created", timestamp: ISO-8601, data: { orderId, userId, items, total }, metadata: { source: "order-service", correlationId, version: 1 } }. The structure is versioned (version field), correlated (correlationId traces the event through the system), and typed (type field for subscriber filtering).'

For topic granularity: 'One topic per domain entity, not one topic per event type. Topic: orders carries order.created, order.shipped, and order.cancelled. Subscribers filter by event type: email service subscribes to orders with filter type = order.shipped (sends shipping confirmation). Analytics subscribes to orders with no filter (processes all order events). One topic per entity is manageable (10 topics for 10 domains). One topic per event type explodes (50 event types = 50 topics to manage).'

AI generates: one global events topic for all event types from all domains. 100 event types in one topic: every subscriber receives every event and must filter locally (wasting bandwidth and processing). Or: direct calls with no topics at all. Per-domain topics: clean separation, subscriber filtering, and manageable topic count. 10 domains = 10 topics, not 1 mega-topic or 100 micro-topics.

• One topic per domain: orders, payments, users — not one per event type
• Event structure: id, type, timestamp, data, metadata (source, correlationId, version)
• Versioned events: version field for backward-compatible evolution
• correlationId: trace the event through the entire system — debugging across services
• type field enables subscriber filtering: receive only events you care about

Rule 2: Fan-Out to Independent Subscribers

The rule: 'When an event is published to a topic, every active subscriber receives a copy. order.created published: the email service sends a confirmation, the inventory service reserves stock, the analytics service records the order, and the loyalty service awards points — all concurrently, all independently. One service failing does not affect the others. Adding a fifth subscriber (fraud detection): subscribe to the orders topic. Zero changes to the publisher or any existing subscriber.'

For subscriber independence: 'Each subscriber has: its own processing speed (email sends in 100ms, analytics aggregation takes 2 seconds — neither waits for the other), its own retry policy (email retries 3 times, analytics retries 10 times), its own DLQ (email DLQ separate from analytics DLQ), and its own failure handling (email failure does not block inventory reservation). Subscribers are completely isolated — they share only the event data, not processing state, errors, or lifecycle.'

AI generates: await emailService.sendConfirmation(order); await inventoryService.reserve(order.items); await analyticsService.record(order); — three sequential calls. Email service is slow (3 seconds): user waits 3 extra seconds. Analytics is down: the entire handler fails. Adding loyalty points: modify the handler, add a fourth call. With pub/sub: publish once, 4 subscribers process concurrently and independently. Total time: max(individual times), not sum. Adding loyalty: one subscription, zero publisher changes.

Rule 3: Message Attribute Filtering

The rule: 'Subscribers filter messages by attributes to receive only relevant events. AWS SNS: filter policy on the subscription: { "eventType": ["order.shipped"] } — this subscription receives only shipped orders. Google Cloud Pub/Sub: filter on message attributes: attributes.eventType = "order.shipped". Redis Pub/Sub: channel-based filtering (subscribe to orders:shipped channel). Filtering at the broker level: the irrelevant messages are never delivered to the subscriber. Zero wasted bandwidth, zero wasted processing.'

For filter granularity: 'Filter by: event type (order.shipped vs order.cancelled), priority (high vs low — process high-priority events on a faster consumer), region (us-east events to US processor, eu-west events to EU processor), and customer tier (enterprise events trigger premium support workflows). Multiple filters combine: eventType = order.shipped AND customerTier = enterprise. The broker evaluates filters — the subscriber receives only matching messages. No client-side filtering, no wasted delivery.'

AI generates: every subscriber receives every event and filters locally. 10,000 events per minute, subscriber cares about 100: 9,900 events received, parsed, evaluated, and discarded. With broker-level filtering: 100 events delivered. 99% less network traffic, 99% less processing. The filter is one configuration on the subscription, not application code in the consumer.

• Broker-level filtering: subscriber receives only matching events, not all events
• Filter by: event type, priority, region, customer tier — combinable
• AWS SNS filter policies, Google Pub/Sub attribute filters, Redis channel patterns
• 99% less traffic when subscribers care about a subset of events
• Filter is subscription config, not application code — change without redeploying

Rule 4: At-Least-Once Delivery and Idempotent Subscribers

The rule: 'Most pub/sub systems provide at-least-once delivery: the same message may be delivered multiple times (redelivery after acknowledgment timeout, network retry, or broker failover). Subscribers must be idempotent: processing the same event twice produces the same result. Implementation: track processed event IDs (SET event:{eventId} EX 86400 in Redis — if already processed, skip). The combination of at-least-once delivery + idempotent subscribers = effectively exactly-once processing.'

For acknowledgment patterns: 'After successfully processing a message: acknowledge it (ack). The broker removes it from the subscription. If the subscriber crashes before acknowledging: the broker redelivers after a visibility timeout (SQS: 30 seconds default, configurable). If processing takes longer than the visibility timeout: extend the timeout before it expires (heartbeat pattern). Never acknowledge before processing is complete — if the subscriber crashes after ack but before processing: the message is lost.'

AI generates: subscribe, receive message, process, no acknowledgment tracking. The message is redelivered: processed again (duplicate email sent, duplicate analytics recorded, duplicate inventory reserved). With idempotent subscribers: the event ID is checked, the duplicate is detected, the message is acknowledged without re-processing. The user receives one email, not three. At-least-once delivery + idempotency = reliable exactly-once semantics.

Rule 5: Pub/Sub vs Point-to-Point Queues

The rule: 'Pub/sub and queues solve different problems. Pub/sub (fan-out): one message, multiple independent subscribers. Use for: event notifications (order.created notifies email, analytics, inventory), state changes (user.updated syncs to search index, cache, and reporting), and cross-domain communication (payment domain notifies order domain). Point-to-point queue: one message, one consumer. Use for: task processing (resize this image, send this email), work distribution (load-balance jobs across workers), and ordered processing (FIFO queue for sequential work).'

For the combination pattern: 'Many systems combine both: pub/sub for event distribution + queue for task processing. order.created is published to a topic (pub/sub). The email subscriber receives the event and enqueues a send-email job in BullMQ (queue). The analytics subscriber receives the event and processes inline (no queue needed — fast, idempotent). Each subscriber decides whether to process inline or enqueue based on its own needs. The pub/sub distributes events; queues manage workloads.'

AI generates: either direct calls (tight coupling, no distribution) or queues for everything (point-to-point, no fan-out). The notification pattern (one event, many listeners) needs pub/sub. The task pattern (one job, one worker) needs a queue. Using a queue for notifications: each consumer polls its own queue, the publisher must publish to N queues. Using pub/sub for tasks: message delivered to all workers, must coordinate who processes it. Each pattern for its purpose.

Complete Pub/Sub Patterns Rules Template

Consolidated rules for pub/sub patterns.

• One topic per domain: orders, payments, users — event types as message attributes
• Fan-out: one publish, N independent subscribers — add subscribers without changing publisher
• Broker-level filtering: subscribers receive only matching events, 99% less traffic
• At-least-once delivery: subscribers must be idempotent — track processed event IDs in Redis
• Acknowledge after processing: never before — crash before ack = safe redelivery
• Pub/sub for event notification (fan-out), queues for task processing (point-to-point)
• Combine both: pub/sub distributes events, subscribers enqueue tasks in their own queues
• correlationId in every event: trace through the entire system for debugging