Limiting concurrency with semaphores

Concurrency in Python services Mid-level Python 3.13+ PostgreSQL 17+ Reviewed By Best Latest Apps

Predict the output

Twenty tasks share a semaphore of 3. What's the highest number running at once?

Python 3.13
import asyncio

running = peak = 0


async def call(sem: asyncio.Semaphore) -> None:
    global running, peak
    async with sem:
        running += 1
        peak = max(peak, running)
        await asyncio.sleep(0.01)
        running -= 1


async def main() -> None:
    sem = asyncio.Semaphore(3)
    async with asyncio.TaskGroup() as tg:
        for _ in range(20):
            tg.create_task(call(sem))
    print(peak)


asyncio.run(main())

Options
Hint 1 of 1

What does Semaphore(3) admit?

Choose an answer, then open Show answer below to check it.

Show answer

Answer: A 3

Correct. The semaphore admits three at a time; the rest wait.

A semaphore is a counter of free slots: async with takes one and gives it back on exit. With 3 slots, a fourth task waits until one of the first three finishes, so the peak is 3.

Why not the others?
  • B 20. That's unbounded fan-out.
  • C 1. The semaphore allows three.
  • D 17. Waiting tasks don't run.

30-second answer

Wrap each outbound call in an asyncio.Semaphore(10): at most ten coroutines hold it at once and the rest wait their turn. The same idea bounds database connections (the pool size), CPU-heavy work and file handles. Unbounded fan-out with gather over thousands of items exhausts sockets, memory and the remote service's patience.

Practise saying it · 30 s. No recording, just a timer

Bound concurrency.

Say your answer out loud, then check which of these points you covered:

  • Shared Semaphore(N)
  • Timeouts
  • Not a rate limit
  • Align with pools

Deep dive

Pair the semaphore with timeouts so a stuck call doesn't hold a slot forever.

Semaphores limit concurrency, not rate; combine with a token bucket for requests per second.

HTTP clients have their own connection limits (httpx Limits); align them.

Across processes, a semaphore must live in shared storage (Redis) if the limit is global.

Common mistakes

  • gather(*[call(x) for x in items]) over 10,000 items.
  • A per-request semaphore (each request gets its own, so nothing is limited).
  • Confusing a concurrency limit with a rate limit.

Senior take & follow-ups

Semaphore around the call, shared, with timeouts; distinguish from rate limiting.

Follow-up questions

Think of your answer first, then open the hint.

  1. Concurrency vs rate limits?

    Show hint

    How many at once vs how many per unit time.

  2. Where should the semaphore live?

    Show hint

    Shared by all callers of that dependency, usually module or app level.

Suggest a fix if something here is wrong or out of date.