Autoscaling services

Performance and scaling Mid-level Python 3.13+ PostgreSQL 17+ Reviewed By Best Latest Apps

Pick the fix

Workers read from a queue; CPU stays at 15% while the backlog grows to 50,000. What should drive scaling?

Options
Hint 1 of 1

Which number grows with the problem?

Choose an answer, then open Show answer below to check it.

Show answer

Answer: A Queue depth relative to per-worker throughput

Correct. It measures the work waiting.

Scale on the signal that tracks the work. These workers wait on I/O, so CPU stays low while the backlog grows; queue depth divided by per-worker throughput says how many workers you need.

Why not the others?
  • B CPU above 70%. The workers are I/O-bound; CPU never gets there.
  • C Memory use. Doesn't track backlog.
  • D Time of day only. Misses unexpected spikes.

30-second answer

Scale on a signal that tracks load and reacts early: CPU for CPU-bound services, in-flight requests or queue depth for I/O-bound ones and workers. Keep a minimum that handles normal peaks, since new instances take time to boot and warm caches. Add cooldowns so it doesn't flap, cap the maximum so you don't overwhelm the database, and pre-scale for known events. Autoscaling doesn't help when the bottleneck is a shared dependency.

Practise saying it · 30 s. No recording, just a timer

Autoscaling.

Say your answer out loud, then check which of these points you covered:

  • Right signal
  • Min for normal peaks
  • Max protects dependencies
  • Warm-up, cooldowns
  • Pre-scale for events

Deep dive

Queue-based workers scale well on backlog divided by per-worker throughput.

Readiness probes stop traffic before an instance is warm.

Scale-down should be slower than scale-up.

Load-test the scaling policy, not just the service.

Common mistakes

  • Scaling on CPU for an I/O-bound service.
  • No maximum, so new instances exhaust database connections.
  • Minimum of 1 instance for a production API.

Senior take & follow-ups

Right signal, sensible min/max, warm-up, cooldowns, and the dependency ceiling.

Follow-up questions

Think of your answer first, then open the hint.

  1. What limits scaling out?

    Show hint

    Shared dependencies: the database, connection limits, rate-limited APIs.

  2. Why scale down slowly?

    Show hint

    To avoid flapping during bursty traffic.

Related questions

Suggest a fix if something here is wrong or out of date.