What is the difference between a message queue and an event stream?

How to think about it

A message queue distributes work: one competing consumer takes responsibility for a message, acknowledges successful processing, and usually causes that message to leave the queue. An event stream retains a history of events: consumers track their own positions, and several independent consumer groups can read the same event and replay it while it is retained.

The distinction an interviewer is testing

Start with the meaning of the message.

A command or job says, “Please do this.” Examples include “resize this image,” “send this email,” or “charge this order.” A queue is usually the natural fit because one worker should perform the job.

An event says, “This already happened.” Examples include “order 8472 was created” or “customer 19 changed their address.” A stream is usually the natural fit because billing, analytics, search indexing and fraud detection may all need to react independently.

That gives us this practical comparison:

Concern	Message queue	Event stream
Primary purpose	Distribute work	Retain and distribute facts
Ownership	One competing consumer handles a delivery	Each consumer group has its own position
After successful processing	Message is acknowledged, settled or deleted	Record remains until retention expires
Replay	Usually limited or awkward	Normal while data is retained
Ordering	Often none unless a special mode is used	Usually ordered within a partition or shard
Fan-out	Usually one queue per downstream subscriber	Multiple consumer groups read one stream
Main backlog signal	Queue depth and oldest-message age	Consumer lag, meaning distance behind the stream

A consumer group is a named set of application instances that share one logical subscription. Within a group, the instances divide the work. Different groups are independent subscribers.

The important word is “usually.” These are semantics, not rigid product categories. A queue can have retention and dead-letter storage. A stream can use acknowledgements. The name printed on the product brochure is not the design.

How the mechanisms differ

When a worker receives a queue message, the broker commonly hides it for a lease called a visibility timeout. If the worker finishes, it acknowledges or deletes the message. If the worker crashes, the lease expires and the message becomes available again.

Amazon SQS Standard is a familiar example. Its delivery model is at-least-once, meaning a message will normally be retried rather than silently discarded, but the same message can arrive more than once. RabbitMQ has a similar acknowledgement and redelivery pattern, although its configuration and failure behaviour differ.

That duplicate is not an edge case. A worker can successfully charge a card and then crash one millisecond before acknowledging the message. The broker has no way to know that the card charge succeeded. It retries.

An idempotent operation is one where repeating the same request has the same final effect as doing it once. Payment processing might use order_id as an idempotency key so a retry returns the existing charge instead of creating a second one.

A stream works differently. The producer appends a record to a durable log. Each record has a position, commonly called an offset, which is simply its numbered location in that log. Reading a record does not delete it. A consumer processes the record and periodically stores its latest completed offset.

Every consumer group stores its own offset. If the billing group has reached position 184,921 and the analytics group has reached position 179,400, both can continue at their own speed. A new group can often start at the oldest retained record, subject to the stream’s retention policy.

Streams usually divide records into partitions, which are independent lanes of the log. Ordering is normally guaranteed within one partition, not across the entire stream. If records are partitioned by order_id, events for one order can remain ordered while unrelated orders are processed in parallel.

Concrete example: an online shop

Suppose an online shop receives 10,000 orders per hour, or about 2.78 orders per second. Every order.created event needs three kinds of work:

1. Billing should charge the customer.
2. The warehouse should begin fulfilment.
3. Analytics should update its dashboards.

Imagine putting every order.created message into one queue called orders, with eight workers attached.

A queue uses competing-consumer semantics. A particular message goes to one worker. If worker 3 receives order 8472, workers 1 through 2 and 4 through 8 do not also receive that same delivery. That is excellent if the workers are interchangeable billing workers. It is wrong if billing, warehouse and analytics are three different applications. Only whichever application won the delivery sees the message.

The usual queue design is therefore a fan-out arrangement: publish the work to three separate queues, such as billing-orders, warehouse-orders and analytics-orders. Each downstream application gets its own queue and can have its own worker pool, retry policy and dead-letter queue.

A stream makes the three subscriptions more direct. Put the event in an order-events topic with six partitions and seven days of retention. Use three consumer groups:

• billing reads every order event for billing.
• warehouse reads every order event for fulfilment.
• analytics reads every order event for reporting.

Each group sees order 8472. If analytics is deployed at noon and is configured to start from the earliest retained position, it can rebuild its seven-day dashboard from the event history. The billing group does not rewind merely because analytics did.

Within one group, the six partitions provide at most six useful parallel lanes. Running 20 analytics instances does not create 20-way parallelism for a six-partition topic. Fourteen instances will have nothing to process until partitions become available.

The processing shape looks like this in pseudocode:

# Queue: one worker owns this delivery
message = receive("billing-orders")
charge_once(message.order_id)
ack(message)

# Stream: this group tracks its own position
for record in read("order-events", group="billing"):
    charge_once(record.order_id)
    commit_offset(record)

The names are illustrative, not a particular SDK. The important ordering is not. A worker should perform the side effect before acknowledging the queue message or committing the stream offset. That ordering can still produce a duplicate after a crash, so charge_once needs an idempotency mechanism.

The senior-level nuance

The common trap is saying, “Streams broadcast, while queues do not.” That is too simple.

Kafka can behave like a work queue when several consumers use the same consumer group: each record is assigned to one group member. The same Kafka topic can also behave like a broadcast source when billing, warehouse and analytics use different groups. Redis Streams supports consumer groups and acknowledgements. A queue system can also fan out messages into several queues.

The real questions are:

• Should one worker own a job, or should several applications independently observe a fact?
• How long must the data remain available?
• Must a new consumer replay old records?
• What ordering scope is required?
• What happens when processing fails halfway through?
• How much operational complexity can the team afford?

Use a queue for transient jobs, isolated retry policies and straightforward backpressure. A queue depth of 500,000 tells you work is waiting. A rising oldest-message age tells you the system is falling behind. Queues are a good fit for image processing, email delivery and one-owner workflow steps.

Use a stream when retention, fan-out, auditability or rebuilding state matters. The cost is real: you must manage partitions, consumer lag, offset commits, rebalancing and retention limits. A seven-day stream is not an archive. After seven days, the records may be gone.

Neither system gives end-to-end exactly-once side effects for free. A Kafka transaction can make certain Kafka-to-Kafka writes atomic, but it cannot magically make an external email provider or payment gateway participate in that transaction. In practice, use at-least-once delivery with idempotent consumers, unique database constraints, an inbox table or an outbox pattern.

An outbox solves a common dual-write failure. The order service writes both the order and an outbox event in one database transaction. A relay later publishes the outbox event to the stream. Without that pattern, the service can commit the order and crash before publishing, leaving the database correct but downstream systems unaware.

A failure you should recognise at 3 a.m.

Suppose billing, warehouse and analytics all consume the same queue. The first symptom may be that the broker looks healthy and the queue drains normally, but the analytics dashboard contains roughly one-third of the orders. Nothing was necessarily lost. The three applications competed for the same deliveries, so each saw only part of the workload.

The fix is separate queues behind a fan-out step, or a stream with one consumer group per application.

Another common failure occurs when a new stream consumer starts from the earliest retained offset and replays six weeks of events into a non-idempotent side effect. The first symptom is a sudden spike in duplicate notifications, refunds or database writes. Choose the start position deliberately, and make replay safe before treating it as a feature.

What they’ll ask next

Can Kafka be used as a message queue?

Yes. Put the workers in the same consumer group, and Kafka distributes partitions among them. The difference is that Kafka still retains records according to its policy, so offsets and replay remain part of the design. If all you need is a simple transient job with per-message visibility timeouts, a queue may be easier.

Can either system guarantee exactly-once processing?

A broker alone generally cannot guarantee exactly-once external effects. A crash can happen after the side effect and before the acknowledgement or offset commit. Use idempotency keys and transactional techniques where the system boundary supports them. Treat “exactly once” as an end-to-end application property, not a switch on the broker.

How do you preserve ordering in a stream?

Partition by the key whose events must stay ordered, such as order_id. All events for one order then use the same partition and are read in sequence within that partition. You do not get global ordering across six partitions without accepting a serious throughput and availability constraint.

Say this in the interview: “A queue assigns a job to one competing consumer, while a stream retains an ordered event history that independent consumer groups can read and replay; I choose between them based on ownership, retention, fan-out, ordering and delivery semantics.”