Load balancing strategies

Performance and scaling Mid-level Python 3.13+ PostgreSQL 17+ Reviewed 31 Oct 2026 By Best Latest Apps

Predict the output

Round robin over three servers for 9 requests. What's printed?

Python 3.13

from itertools import cycle

servers = cycle(["A", "B", "C"])
sent = {"A": 0, "B": 0, "C": 0}
for _ in range(9):
    sent[next(servers)] += 1
print(sent)

Options{'A': 3, 'B': 3, 'C': 3}{'A': 9, 'B': 0, 'C': 0}{'A': 4, 'B': 3, 'C': 2}A random split

Hint 1 of 1

What does cycle do?

Choose an answer, then open Show answer below to check it.

Show answer

Answer: A {'A': 3, 'B': 3, 'C': 3}

Correct. Round robin ignores load: even a slow server gets a third.

itertools.cycle gives A, B, C, A, B, C forever, so 9 requests split 3 each. Round robin counts requests, not load, so a slow server still gets its full share.

Why not the others?

B {'A': 9, 'B': 0, 'C': 0}. cycle moves to the next server each time.
C {'A': 4, 'B': 3, 'C': 2}. 9 divides evenly by 3.
D A random split. Round robin is deterministic.

30-second answer

Round robin sends requests to servers in turn, so a slow server still gets its full share and its queue grows. Least-connections (or least outstanding requests) sends new work to the server with the fewest in flight, so a slow server naturally gets less. Health checks remove servers that fail outright, but a slow server that still answers usually passes them. Layer 4 routes TCP connections; layer 7 sees HTTP and can route by path or header.

Practise saying it · 35 s. No recording, just a timer

Explain load balancing.

Say your answer out loud, then check which of these points you covered:

Round robin vs least-connections
Slow server problem
Health checks vs outliers
L4 vs L7
Avoid stickiness

Deep dive

Power of two choices: pick two servers at random and send to the less loaded one; nearly as good as least-connections with less coordination.

Long-lived HTTP/2 or gRPC connections defeat connection-level balancing; balance per request (L7) instead.

Sticky sessions pin users to servers; prefer stateless servers with shared session storage.

Passive outlier detection (eject servers whose error rate or latency spikes) catches what active health checks miss.

Strategy	Requests to the slow server	What the health check does
Round robin	Its full share (1 in 3 with 3 servers); its queue grows	Nothing while it still answers in time
Least connections	Fewer: requests pile up there, so new ones go elsewhere	Nothing while it still answers in time
Either, with outlier ejection	None for a while after its latency or errors spike	Removes it only if it fails or times out

Round robin ignores how busy a server is Ran on Python 3.13.5 · mypy --strict

Python 3.13

from itertools import cycle

servers = cycle(["A", "B", "C"])
sent = {"A": 0, "B": 0, "C": 0}
for _ in range(9):
    sent[next(servers)] += 1
print(sent)

Common mistakes

Assuming health checks catch slow servers.
Sticky sessions that make scaling and deploys uneven.
One gRPC connection carrying all traffic to one backend.

Senior take & follow-ups

Compare round robin with least-connections on a slow server, then health checks, L4 vs L7 and outlier detection.

Follow-up questions

Think of your answer first, then open the hint.

Why is gRPC hard to balance at L4?
Show hint
One long-lived connection carries many requests to one server.
What's power of two choices?
Show hint
Sample two servers, pick the less loaded; avoids herding.

Suggest a fix if something here is wrong or out of date.