Consumer groups and rebalancing

Queues, Kafka and event-driven systems Mid-level Python 3.13+ PostgreSQL 17+ Reviewed 31 Oct 2026 By Best Latest Apps

Predict the output

Six partitions, a group of four consumers. With an even assignment, how many partitions does the busiest consumer own?

Python 3.13

partitions, consumers = 6, 4
busiest = -(-partitions // consumers)
print(busiest)

Options2161.5

Hint 1 of 1

Ceiling of 6 / 4.

Choose an answer, then open Show answer below to check it.

Show answer

Answer: A 2

Correct. 6 over 4 means two consumers own 2 and two own 1.

Within a group, each partition goes to exactly one consumer. Six partitions over four consumers means two get 2 each and two get 1; the code works out the ceiling of 6/4.

Why not the others?

B 1. Six partitions can't fit one per consumer with four consumers.
C 6. Partitions are shared within a group.
D 1.5. Partitions aren't split; the code uses ceiling division.

30-second answer

The group coordinator assigns each partition to exactly one consumer in the group. When membership changes, or a consumer stops polling for longer than max.poll.interval.ms, the group rebalances and partitions move. The new owner resumes from the last committed offset, so anything processed but not committed is processed again. Cooperative rebalancing moves only the partitions that need to move instead of pausing everyone.

Practise saying it · 35 s. No recording, just a timer

Explain consumer group rebalancing.

Say your answer out loud, then check which of these points you covered:

One owner per partition
Join, leave, stall trigger it
Resume from committed offset
Cooperative / KIP-848
Keep poll loop fast

Deep dive

Slow processing inside the poll loop is the usual cause of surprise rebalances; process faster, lower max.poll.records, or raise the interval.

Static membership (group.instance.id) avoids rebalances on quick restarts.

Kafka 4 makes the new consumer rebalance protocol (KIP-848) generally available, with the broker driving assignment.

Commit offsets for work you've finished; on revoke, commit before giving partitions up.

Common mistakes

Doing a 10-minute job per message inside poll() with default settings.
Running more consumers than partitions and expecting more throughput.
Auto-commit with asynchronous processing, losing messages on rebalance.

Senior take & follow-ups

One owner per partition, rebalance triggers, resume from committed offset, and how to avoid churn.

Follow-up questions

Think of your answer first, then open the hint.

Why are messages processed twice after a rebalance?
Show hint
The new owner restarts from the last committed offset.
What triggers a rebalance besides join and leave?
Show hint
Missing heartbeats or exceeding max.poll.interval.ms.

Suggest a fix if something here is wrong or out of date.