How would you design a recommendation system for an online store that needs to respond in under a hundred milliseconds?

What the interviewer is looking for

• This question tests ml system design. Designs end-to-end systems covering data, candidate generation, ranking, serving and latency budgets, with clear trade-offs.
• Strong answers: Candidate retrieval plus ranking stages, precomputed embeddings, caching, feature freshness, cold start handling and online evaluation.
• Likely follow-up: How would you handle a brand-new product with no history?

Another way it may be asked: When a shopper opens a product page, we need to show ten relevant items almost instantly. How would you build that?

A structure to follow: Requirements, sketch, deep-dive, trade-offs

1. Step 1

   Requirements

   Ask who uses it, what it must do, and which qualities matter most: speed, reliability, cost, consistency.

2. Step 2

   Sketch

   Draw the main parts and the path one request takes through them before adding detail.

3. Step 3

   Deep-dive

   Pick the riskiest part (storage, queueing, failure handling) and go deeper there.

4. Step 4

   Trade-offs

   Name what you gave up for each choice, and what you would do first if it needed to grow.

This is guidance on shape, not a script. Use your own real experience and facts; never invent details to fill a gap. See the STAR guide and answer frameworks.

Common slips

• Naming tools before establishing requirements.
• Drawing every box at the same depth instead of going deep where it matters.
• Presenting one design as the only answer, with no trade-offs.

How to practise it

Time yourself: two minutes on requirements, five on the sketch, five on one deep-dive, two on trade-offs.

Related questions

• Why do we split data into training, validation and test sets, and what goes wrong if information leaks between them?
• How would you turn a notebook a data scientist gave you into code that's safe to run in production?
• What does it mean for features to be computed the same way in training and in production, and why does that matter?
• Walk me through how you've deployed a model. How was it packaged, served and rolled out safely?
• Tell me about a time a model's performance dropped in production. How did you notice and what did you do?

Browse all questions