Memory growth in Python services

Performance and scaling Mid-level Python 3.13+ PostgreSQL 17+ Reviewed By Best Latest Apps

Spot the bug

Which line makes memory grow without limit?

  1. from collections import OrderedDict
  2. ​
  3. _seen: dict[str, bytes] = {}
  4. _bounded: OrderedDict[str, bytes] = OrderedDict()
  5. ​
  6. ​
  7. def remember(key: str, value: bytes) -> None:
  8. _seen[key] = value
  9. _bounded[key] = value
  10. if len(_bounded) > 1_000:
  11. _bounded.popitem(last=False)
Hint 1 of 1

Which dict ever removes items?

Choose an answer, then open Show answer below to check it.

Show answer
  • Answer: line 8 _seen[key] = value

Line 8 adds every key to _seen forever. _bounded evicts its oldest entry past 1,000 items; _seen never shrinks.

30-second answer

First confirm it's a leak and not a cache warming up or fragmentation: watch RSS over hours under steady load. Common causes are unbounded caches and module-level dicts, objects kept alive by references in globals or closures, growing per-request lists, and C extensions. Use tracemalloc to compare snapshots and see which lines allocate the growth, then bound the structure (an LRU with a max size) or fix the reference.

Practise saying it · 30 s. No recording, just a timer

Investigate memory growth.

Say your answer out loud, then check which of these points you covered:

  • Confirm leak vs warm-up
  • tracemalloc snapshots
  • Unbounded caches, globals
  • Bound or fix
  • Recycling is a band-aid

Deep dive

functools.lru_cache(maxsize=None) and functools.cache are unbounded; the default lru_cache keeps 128 entries.

Worker recycling (max requests per worker) is a mitigation, not a fix.

Large allocations and fragmentation can keep RSS high even after objects are freed.

Set container memory limits and alert before the kill.

Common mistakes

  • Restarting nightly and calling it fixed.
  • Unbounded in-process caches.
  • Storing per-request data in globals.

Senior take & follow-ups

Confirm, snapshot with tracemalloc, find the growing structure, bound or fix it.

Follow-up questions

Think of your answer first, then open the hint.

  1. Which tool shows where memory is allocated?

    Show hint

    tracemalloc snapshots compared over time.

  2. Is lru_cache bounded by default?

    Show hint

    Yes, maxsize defaults to 128; maxsize=None makes it unbounded.

Related questions

Suggest a fix if something here is wrong or out of date.