Concurrency in Python services cheat sheet

The GIL, threads and processes

The global interpreter lock lets only one thread run Python bytecode at a time in standard CPython. Threads still help for I/O-bound work, because the lock is released while waiting on sockets and files. CPU-bound Python code doesn't get faster with threads; use processes (multiprocessing, ProcessPoolExecutor), native extensions that release the GIL, or move the work out of the request path. Python 3.13 added an experimental free-threaded build without the GIL; 3.14 made it officially supported (PEP 779), but still as a separate, optional build.

from concurrent.futures import ProcessPoolExecutor


def work(n: int) -> int:
    return sum(i * i for i in range(n))


if __name__ == "__main__":
    with ProcessPoolExecutor(max_workers=2) as pool:
        print(list(pool.map(work, [10, 100, 1000])))

How asyncio works

asyncio runs coroutines on an event loop. When a coroutine awaits I/O, it yields control, the loop registers interest with the OS (epoll or kqueue), and runs other ready coroutines. When data arrives the waiting coroutine resumes. One thread can juggle thousands of mostly idle connections because waiting costs almost nothing. Anything that doesn't await, like CPU work or a blocking library call, stalls every coroutine on that loop.

Blocking calls inside async code

A synchronous call inside a coroutine blocks the event loop thread, so every other request on that loop waits until it returns. Fix it by using an async client, or by moving the call to a thread with await asyncio.to_thread(fn, ...) (or loop.run_in_executor), which frees the loop while the thread waits. CPU-heavy work should go to a process pool instead. asyncio's debug mode reports callbacks that take longer than 100 ms.

Race conditions in async code

Yes. A coroutine can be paused at any await, and another can change shared state before it resumes. A read, an await, then a write based on the old read loses updates exactly like threads do. Keep read-modify-write sequences free of awaits, protect them with asyncio.Lock, or push the operation into the database or Redis as one atomic statement.

Sharing state between threads safely

Treat shared mutable state as unsafe unless one operation is guaranteed atomic. d[k] += 1 is a read, an add and a write, and another thread can run between them. Use a threading.Lock around compound updates, prefer passing messages through queue.Queue (which is thread-safe), or keep state per thread and merge at the end. Better still, keep per-request state local and shared state in a store with atomic operations.

Limiting concurrency with semaphores

Wrap each outbound call in an asyncio.Semaphore(10): at most ten coroutines hold it at once and the rest wait their turn. The same idea bounds database connections (the pool size), CPU-heavy work and file handles. Unbounded fan-out with gather over thousands of items exhausts sockets, memory and the remote service's patience.

Cancellation and cleanup in asyncio

Frameworks typically cancel the request's task, which raises CancelledError at the coroutine's current await. Use try/finally or async with so connections, locks and temp files are released. Don't swallow CancelledError; if you catch it to clean up, re-raise it. Work that must finish even if the client leaves (charging a card already in flight) belongs in a separate task or a job queue, not the request coroutine.

WSGI vs ASGI servers

WSGI is the synchronous Python web interface: each request occupies a worker (process or thread) until it returns, as with Flask or Django under gunicorn. ASGI is the asynchronous one: an event loop per worker handles many requests concurrently and supports WebSockets, as with FastAPI or Starlette under uvicorn. Either way you run several worker processes per machine for CPU parallelism, often around one per core for async workers, more for sync ones with I/O waits.

Request-scoped context with contextvars

Store it in a contextvars.ContextVar. Each asyncio task runs in a copy of the context, so setting the request id at the start of a request affects only that request's code and the tasks it creates, even while thousands of requests interleave on one thread. A logging filter or structured logger reads the variable and adds it to every record. Thread-locals don't work for this in async code.

Producer-consumer with asyncio.Queue

Create a bounded asyncio.Queue, start N worker tasks that loop on await queue.get(), process, and call task_done(), and have the producer await queue.put() items. The bound gives backpressure: the producer waits when workers fall behind. await queue.join() waits until every item is processed, then you cancel the workers. Exceptions in a worker should be caught per item so one bad item doesn't kill the worker.

Deadlocks with application locks

Path one takes lock A then waits for B; path two holds B and waits for A. Neither can proceed: a deadlock. Prevent it by always acquiring locks in one global order (for example by sorted resource id), holding as few locks as possible for as short as possible, and using timeouts on acquisition so a stuck path fails loudly instead of hanging. In databases the engine detects cycles; in application code nothing does.

TaskGroup vs gather

Both run coroutines concurrently. With gather, if one fails the exception propagates but the others keep running in the background unless you cancel them, and return_exceptions=True mixes errors into results. TaskGroup (Python 3.11+) is structured: when one task fails, it cancels the rest, waits for them, and raises an ExceptionGroup with every failure. No task outlives the async with block, so nothing leaks.