Why do we split data into training, validation and test sets, and what goes wrong if information leaks between them?

What the interviewer is looking for

• This question tests model training & evaluation. Explains data splits, leakage, metrics and ways to speed up training with specific techniques and measured results.
• Strong answers: Separate sets for fitting, tuning and final checks, and leakage inflating scores, including time-based and duplicate-entity leaks.
• Likely follow-up: How would you split data for a forecasting problem?

Another way it may be asked: If the same customers show up in both your training and test data, why might your model look better than it really is?

A structure to follow: Answer, explain, example, limits

1. Step 1

   Answer

   Give your direct answer in the first sentence so the interviewer knows where you are going.

2. Step 2

   Explain

   Say how it works or why you would do it that way, in plain words before jargon.

3. Step 3

   Example

   Anchor it in something you have actually done or seen, with the detail that shows hands-on experience.

4. Step 4

   Limits

   Say when your approach would not be the right one. It shows judgement.

This is guidance on shape, not a script. Use your own real experience and facts; never invent details to fill a gap. See the STAR guide and answer frameworks.

Common slips

• Reciting a definition without showing you have applied it.
• Using terms you cannot explain if asked a level deeper.
• Claiming experience beyond what you can discuss in detail.

How to practise it

Say the answer in 90 seconds, then answer the follow-up as if it were a new question.

Related questions

• Your model training takes twelve hours on a GPU and the team needs to iterate faster. What would you try?
• How would you turn a notebook a data scientist gave you into code that's safe to run in production?
• What does it mean for features to be computed the same way in training and in production, and why does that matter?
• Walk me through how you've deployed a model. How was it packaged, served and rolled out safely?
• Tell me about a time a model's performance dropped in production. How did you notice and what did you do?

Browse all questions