Event-driven architecture: events, brokers, and Kafka vs RabbitMQ

What does event-driven actually mean?

Martin Fowler's 2017 note on the term records that "when people talk about 'events', they actually mean some quite different things". He separates four patterns:

• Event notification. A system sends a small message saying something changed, "often just some id information and a link back to the sender", and a consumer that needs more calls back for the details. A webhook delivers the same kind of notification over plain HTTP, with no broker in between.
• Event-carried state transfer. The event carries the changed data, so consumers keep their own copy and "don't need to contact the source system in order to do further work". The cost is more copies and more receiver state; the gain is resilience when the source is down and no remote call to read the data.
• Event sourcing. Every state change is recorded as an event, and the system state is purely derived from that log. Fowler's example is version control: the commits are the event store and the working copy is the state. It does not have to be asynchronous.
• CQRS. Command Query Responsibility Segregation is "the notion of having separate data structures for reading and writing information", and "Strictly CQRS isn't really about events".

For connecting systems, the first two matter most. Event sourcing and CQRS are choices about how one system stores and reads its own data; adopt either, both or neither without changing how systems talk to each other.

The four patterns do not settle events versus commands. An event is a fact in the past tense with no intended recipient ("order 1042 was paid"). A command is an instruction aimed at one recipient ("charge order 1042"). Fowler warns about the "passive-aggressive command": the source system "expects the recipient to carry out an action" but styles the message as an event, which hides a real dependency inside a flow that looks decoupled. A quick test: if the flow breaks when one specific consumer is offline, that message was a command.

Queue or log: how should the broker hold your events?

The brokers below hold messages in one of two ways.

A queue hands each message to one of several competing consumers and removes it once the consumer is done; pub/sub instead lets several subscribers each receive a copy. RabbitMQ's streams documentation calls this "destructive consume behaviour": messages are deleted from the queue when a consumer is finished with them. In Amazon SQS the consumer deletes the message after processing; if it does not within the visibility timeout (30 seconds by default), the message becomes visible again for another consumer. SQS standard queues promise at-least-once delivery and best-effort ordering.

A log appends each record to a durable, ordered sequence and keeps it for a retention window. In Kafka's design documentation, a topic is "a set of totally ordered partitions, each of which is consumed by exactly one consumer within each subscribing consumer group at any given time", so a consumer's position is "just a single integer", its offset. A consumer can "deliberately rewind back to an old offset and re-consume data", which "violates the common contract of a queue, but turns out to be an essential feature for many consumers". Records stay until retention expires: log.retention.hours defaults to 168, seven days. RabbitMQ streams bring the same non-destructive model into RabbitMQ, and Pub/Sub's seek replays acknowledged messages when you have configured retention.

The model answers questions you have not asked yet:

Decide per flow whether you need replay before you pick a product.

Which broker should carry your events?

Kafka and RabbitMQ both changed recently, and this section follows their 4.3 documentation (October 2026). Kafka 4.0 (March 2025) was "the first major release to operate entirely without Apache ZooKeeper". Kafka 4.2 (February 2026) declared share groups ("Kafka Queues") production-ready: consumers in a share group can outnumber partitions and acknowledge records individually, so Kafka now has a queue mode beside its log. RabbitMQ 4.0 removed classic queue mirroring, leaving quorum queues and streams as its replicated types.

"Not documented" means the cited pages do not describe the feature. Three patterns matter more than any cell.

Ordering is scoped, not global. Each ordering guarantee in the table holds within a unit: a Kafka partition, a RabbitMQ queue, an SNS FIFO message group, a Service Bus session or a Pub/Sub ordering key. In Kafka the key you choose (customer, order, device) decides which partition a record lands in, and so both what stays in order and how many consumers in a group can work in parallel.

Replay separates the families, and the gap is narrowing. If a consumer bug corrupts derived data, a log lets you fix the code and rewind. SNS FIFO topics and EventBridge offer archive and replay too, and AWS says new applications should start with the Custom Event Bus, which "retains events for replay".

Operations are the price. Running Kafka or RabbitMQ yourself means owning upgrades, storage and capacity. The cloud services replace that with limits you design around, such as up to five targets per rule on EventBridge's default and Classic buses. If your consumers are functions rather than long-running services, our serverless computing guide explains how events, instances and scale to zero fit together.

What does exactly-once actually promise?

Kafka's design documentation lists three semantics: at most once ("Messages may be lost but are never redelivered"), at least once ("Messages are never lost but may be redelivered") and exactly once ("Each message is processed once and only once"). Then it warns: "Many systems claim to provide 'exactly-once' delivery semantics, but it is important to read the fine print."

Since 0.11.0.0 a producer can send messages atomically to several partitions using transactions, so that "either all messages are successfully written or none of them are". A consumer set to read_committed returns only committed transactions, where the default, read_uncommitted, also returns aborted ones. Together they give exactly-once "when reading, processing and writing data on Kafka topics", and the documentation calls Kafka Streams the simplest way to get it. For any other destination, exactly-once "generally requires cooperation with such systems". The documentation's answer for an external store is to let the consumer "store its offset in the same place as its output", so the data and the position change together.

The producer defaults are already the safe ones: Kafka 4.3's producer configuration enables idempotence by default and defaults acks to all. An exactly-once loop adds a transactional ID and two consumer settings:

# Kafka 4.3 producer defaults: idempotent, durable writes
enable.idempotence=true
acks=all
max.in.flight.requests.per.connection=5

# Also required for an exactly-once loop inside Kafka
# producer (the id is an example)
transactional.id=orders-enricher-1
# consumer
isolation.level=read_committed
enable.auto.commit=false

Everything past that boundary is yours. RabbitMQ's reliability guide states: "Use of acknowledgements guarantees at least once delivery." It adds that consumers "must be prepared to handle deliveries they have seen in the past". Publishers get duplicates too: a broker "might have sent a confirmation that never reached the producer", so a retried publish can arrive twice. Kafka, SQS standard queues, Event Grid, Pub/Sub and Service Bus in peek-lock mode also document at-least-once delivery.

So the consumer must be idempotent: handling an event twice leaves the same result as handling it once. Record each processed event ID in the same database transaction as the state change, and a replayed event is recognized and skipped. Our dead letter queue guide covers that pattern, the retries in front of it and the replay after it. The managed services state their own limits. Pub/Sub's exactly-once delivery applies "within a cloud region" and only to pull subscriptions, and combined with ordering it limits throughput to "an order of thousands of messages per second".

How do you keep events readable as they change?

A request-driven API has a contract. An event stream needs one even more, because the producer may never meet its consumers. Three pieces do the work.

A common envelope. CloudEvents, which graduated from the CNCF in January 2024, standardizes the metadata around the payload. Four attributes are required: id, source, specversion and type. Producers must keep source plus id unique for each distinct event, and a re-sent duplicate may reuse its id, so consumers "MAY assume that Events with identical source and id are duplicates". That pair is your idempotency key.

{
  "specversion": "1.0",
  "type": "com.example.order.paid",
  "source": "/orders",
  "id": "order-1042-paid-1",
  "time": "2026-10-09T17:58:00Z",
  "datacontenttype": "application/json",
  "traceparent": "00-0af7651916cd43dd8448eb211c80319c-b7ad6b7169203331-01",
  "data": {
    "orderId": "1042",
    "total": 129.0,
    "currency": "CAD"
  }
}

A schema registry. Events outlive the code that wrote them, so the payload shape needs versions of its own. Confluent Schema Registry provides "a REST service for validating, storing, and retrieving Avro, JSON Schema, and Protobuf schemas", and it checks compatibility before accepting a new version. The default rule is BACKWARD: consumers on the new schema can read data written with the old one. Confluent's reason suits logs: it is preferred for Kafka "so that you can rewind consumers to the beginning of the topic". The rule also sets the upgrade order: with BACKWARD, "upgrade all consumers before you start producing new events"; FORWARD reverses it.

Three habits keep you out of trouble:

1. Add new fields as optional; Confluent lists "add optional fields" among the changes BACKWARD allows. Treat a changed meaning as a new field.
2. When a change cannot be compatible, Confluent suggests you "create a brand-new topic and start migrating applications" instead of mixing incompatible versions in one topic.
3. Use transitive checks, which compare against all previous versions and not only the last, on any topic a consumer might replay from the start.

Documentation. AsyncAPI is a specification for defining the interfaces of an async API, "much like OpenAPI (aka Swagger) does for REST APIs", so a consumer team can see what a stream carries without reading your code. Its current version, 3.1.0, was released in January 2026. We apply the same contract-first discipline to request APIs in our enterprise API guide; here the contract also has to survive replays of old events.

Choreography or orchestration: who runs the workflow?

Once several services react to each other's events, somebody owns the overall flow, or nobody does. The Azure Architecture Center describes choreography as letting "each service decide when and how to process a business operation, instead of depending on a central orchestrator". Orchestration is the inverse: one component calls the steps and tracks where each transaction stands.

Fowler's warning lands on the choreography column: a logical flow spread over event notifications is "not explicit in any program text. Often the only way to figure out this flow is from monitoring a live system." Choreography is not free coupling; it is coupling you cannot grep.

When a business operation spans services that each own their data, one database transaction cannot cover it, and the saga pattern fills the gap. In their 1987 paper Sagas, Garcia-Molina and Salem describe a long-lived transaction written as a sequence of transactions, where "either all the transactions in a saga are successfully completed or compensating transactions are run to amend a partial execution". A compensating transaction "undoes, from a semantic point of view" what the original did, but "does not necessarily return the database to the state that existed" before. You do not un-charge a card by deleting a row; you issue a refund.

An illustrative order flow: reserve stock, charge the card, book the shipment. If the carrier rejects the address, the saga refunds the charge, then releases the reservation. Each compensating action is itself an event and must be idempotent, because it is retried until it succeeds.

Debugging needs a thread to follow, and Azure's guidance is that a choreographed system must "consistently use distributed tracing and correlation identifiers". Put a correlation ID on the triggering request and carry it through every event and call. The standard format is W3C Trace Context (a W3C Recommendation of 23 November 2021): a traceparent header with four fields (version, trace ID, parent ID, trace flags) plus tracestate for additional vendor-specific trace information. The CloudEvents distributed tracing extension puts the same values on the event, as in the JSON above, so that traces can "span an event-driven system"; there traceparent is required and tracestate optional.

What breaks in production?

Start with the failure that stops a whole lane. A consumer that must process a partition in order cannot move past a record it cannot handle. A Kafka consumer group tracks one offset per partition, so a poison message holds up everything behind it until the application sets it aside. RabbitMQ quorum queues drop or dead-letter a message after a delivery limit (default 20), and Kafka's share groups count delivery attempts, which enables automated handling of unprocessable records. Whatever the broker, decide how many attempts a message gets before it is parked; our dead letter queue guide covers the pattern.

The other usual failures fit in one table:

Saving a record and publishing its event are two writes to two systems, and either can succeed alone. AWS Prescriptive Guidance describes the fix: use an outbox table or change data capture. With an outbox, the event is written in the same transaction as the state change and a separate process publishes it, in the order the database was updated. That process might send duplicates, so the guidance recommends an idempotent consumer that tracks processed messages.

When should you not go event-driven?

• The flow is simple CRUD. One writer, one reader, no reactions worth decoupling: a function call is a dependency you can see, step through and refactor. Microsoft says broker overhead is not justified for straightforward interactions.
• The caller needs the answer now. A payment authorization, a login, a stock check at checkout: an event says something happened, and it cannot answer "did it work?". Make the call synchronously, then publish the event about what you did. Request-reply over queues can imitate a call, but it effectively turns the flow synchronous, with a broker in the middle.
• A scheduled batch is enough. Where trading partners already exchange documents on a schedule (the territory of EDI), a batch job may be all you need.
• The team cannot operate a broker. Someone must own upgrades, storage, capacity and on-call. If nobody can, start with a cloud queue or topic. Microsoft lists teams without experience operating distributed asynchronous systems as a poor fit.
• Half-finished states must stay invisible. Sagas compensate; they do not isolate. The 1987 paper notes that "other transactions might see the effects of a partial saga execution". If a business rule forbids anyone seeing a charged but unshipped order, keep the steps in one database transaction. Microsoft likewise flags business transactions that need strong consistency across services.

A migration plan you can start this month

1. Pick one flow that hurts. Minutes of latency are acceptable, failures currently page someone, and volume is real but modest, such as orders to accounting.
2. Write the contract before the code. A CloudEvents envelope, a schema registered with BACKWARD compatibility, one named owner per event type, and a sample payload reviewed by the people who own the data.
3. Publish through an outbox. The event is committed in the same transaction as the state change, so one cannot be saved without the other. If the source is a legacy database nobody may touch, attach change data capture instead.
4. Wrap the consumer before traffic arrives. An idempotent handler, bounded retries with backoff, a dead-letter queue with an alarm at one message, and a redrive procedure someone has actually run. Our DLQ guide is the checklist.
5. Thread a correlation ID end to end and prove you can trace one order through every hop, in logs or traces, before you add a second flow.
6. Cut over gradually. Run the event flow beside the existing integration, compare outputs against the old path daily, and retire it only after a clean month, in the spirit of the strangler fig pattern. A legacy Java modernization used the same discipline, comparing each slice's screens, database writes and report files with the old system before accepting it.
7. Rehearse the bad day. Replay a day of events into a test environment, dead-letter a poison message on purpose, and write down what happened.

This is the shape of the work in our integrations service: contracts, idempotent consumers, dead-letter queues and a runbook, whether the broker is Kafka, RabbitMQ or a managed service. When the event-driven parts live inside a system we are building for you, that is custom software development.