Uploading large files

API design: REST, HTTP, GraphQL and gRPC Mid-level Python 3.13+ PostgreSQL 17+ Reviewed By Best Latest Apps

Order the steps

Order the steps of a direct-to-storage upload.

  • Client uploads parts directly to storage, retrying failed parts
  • Client completes the multipart upload
  • Storage event triggers a worker to verify and process
  • Client asks the API to start an upload
  • API returns presigned part URLs that expire soon

Choose an answer, then open Show answer below to check it.

Show answer

Answer: D E A B C

  1. Client asks the API to start an upload API checks auth, size and type limits.
  2. API returns presigned part URLs that expire soon No file bytes pass through the API.
  3. Client uploads parts directly to storage, retrying failed parts A dropped connection only repeats one part.
  4. Client completes the multipart upload Storage assembles the object.
  5. Storage event triggers a worker to verify and process Scan, transcode, then mark available.

30-second answer

Don't stream large files through your API servers. Have the API authorise the upload and return a presigned URL to object storage (S3, GCS) with a short expiry, size limit and content type; the client uploads directly, in parts for big files so a dropped connection resumes instead of restarting. Storage then notifies your backend (event or callback) to validate, scan and process the file in a worker.

Practise saying it · 35 s. No recording, just a timer

Design a large upload.

Say your answer out loud, then check which of these points you covered:

  • Presigned URL, short expiry
  • Direct to storage
  • Multipart, resumable
  • Validate bytes server-side
  • Async processing

Deep dive

Multipart uploads: S3 parts are 5 MiB to 5 GiB each, up to 10,000 parts.

Never trust the client's content type; check the bytes (magic numbers) server-side.

Store files under random keys, not user-supplied names.

Mark uploads pending until processed; clean up abandoned ones with lifecycle rules.

Common mistakes

  • Proxying gigabytes through app servers with request size limits and timeouts.
  • Long-lived or unrestricted presigned URLs.
  • Serving user uploads from your main domain (stored XSS risk).

Senior take & follow-ups

Presigned direct upload, multipart for resume, then async validation and processing.

Follow-up questions

Think of your answer first, then open the hint.

  1. How do you resume an upload?

    Show hint

    Multipart or tus-style chunks; retry only the failed parts.

  2. Where do you scan for malware?

    Show hint

    In the post-upload worker before marking the file available.

Similar on iOS: Resilient large downloads The iOS page is the download side on a phone; here the server hands out presigned upload URLs.

Related questions

Suggest a fix if something here is wrong or out of date.