Keeping messages in order

Queues, Kafka and event-driven systems Mid-level Python 3.13+ PostgreSQL 17+ Reviewed By Best Latest Apps

Predict the output

Events arrive out of order. The consumer ignores older versions. What's the final state?

Python 3.13
state = ("new", 0)
events = [("shipped", 3), ("paid", 2), ("created", 1)]
for status, v in events:
    if v > state[1]:
        state = (status, v)
print(state)

Options
Hint 1 of 1

Which events have v greater than what's applied?

Choose an answer, then open Show answer below to check it.

Show answer

Answer: A ('shipped', 3)

Correct. Versions 2 and 1 are older than 3, so they're ignored.

Comparing each event's version with the stored one makes the update monotonic: only a newer version applies. Version 3 arrives first, so the older 2 and 1 are skipped.

Why not the others?
  • B ('created', 1). That's last-arrival-wins, the bug versions prevent.
  • C ('paid', 2). 2 isn't newer than 3.
  • D ('new', 0). Version 3 is newer than 0, so it applies.

30-second answer

Route all messages for one entity to the same ordered channel (same Kafka key, or SQS FIFO message group) and process each channel with one consumer at a time. Then avoid things that reorder inside it: parallel processing of one partition, and retries that push a failed message to the back. Where you can't guarantee order, carry a version or sequence number and have consumers ignore or buffer events older than what they've applied.

Practise saying it · 30 s. No recording, just a timer

How do you keep order?

Say your answer out loud, then check which of these points you covered:

  • Same key / message group
  • One consumer per partition
  • No reordering retries
  • Version checks

Deep dive

Producer settings: Kafka's idempotent producer keeps order within a partition even with retries.

Sending a failed message to a retry topic breaks order for that key; pause the key or block the partition instead if order matters.

Version checks in the consumer: UPDATE … WHERE version < $new.

Global ordering across all entities needs a single partition, which caps throughput.

Common mistakes

  • Thread pools processing messages from one partition in parallel.
  • Retry topics for ordered workflows.
  • Assuming wall-clock timestamps from different producers give an order.

Senior take & follow-ups

Same key, single consumer per partition, no reordering retries, and versions as the safety net.

Follow-up questions

Think of your answer first, then open the hint.

  1. How do you parallelise and keep order?

    Show hint

    Parallelise across keys, not within one key.

  2. What if events arrive out of order anyway?

    Show hint

    Use versions and drop or buffer older ones.

Related questions

Suggest a fix if something here is wrong or out of date.