Data Scientist
You're building a fraud model where only one in a thousand transactions is fraud. How would you evaluate whether it's any good?
What the interviewer is looking for
- This question tests model evaluation. Picks metrics that match business cost, handles imbalance and leakage, and plans for monitoring after launch.
- Strong answers: Rejects accuracy, uses precision, recall and precision recall curves, sets thresholds from the cost of misses versus false alarms.
- Likely follow-up: How would you pick the decision threshold with the fraud team?
Another way it may be asked: A model that always says not fraud would be ninety nine point nine percent accurate. So how would you actually judge this model?
A structure to follow: Answer, explain, example, limits
- Step 1
Answer
Give your direct answer in the first sentence so the interviewer knows where you are going.
- Step 2
Explain
Say how it works or why you would do it that way, in plain words before jargon.
- Step 3
Example
Anchor it in something you have actually done or seen, with the detail that shows hands-on experience.
- Step 4
Limits
Say when your approach would not be the right one. It shows judgement.
This is guidance on shape, not a script. Use your own real experience and facts; never invent details to fill a gap. See the STAR guide and answer frameworks.
Common slips
- Reciting a definition without showing you have applied it.
- Using terms you cannot explain if asked a level deeper.
- Claiming experience beyond what you can discuss in detail.
How to practise it
Say the answer in 90 seconds, then answer the follow-up as if it were a new question.
Related questions
- After a model goes live, how would you know it's starting to get worse?
- How would you explain a p-value to a product manager who has never taken a statistics course?
- You get a dataset where about a fifth of the values in an important column are missing. How do you decide what to do?
- What is overfitting, and how would you spot it in a model you just trained?
- Tell me about a split test or experiment you designed or analysed. How did you choose the success metric and the sample size?