Prompt and context
Just before launch, an experiment or data check shows that a key assumption is false. The team has already invested time, and the owner is under delivery pressure. Use a real example to explain how you separated facts from inferences, contained the impact, and corrected direction together instead of immediately looking for someone to blame.
What the interviewer evaluates
- Updating your judgment quickly when new evidence appears.
- Turning debate into impact, unknowns, options, and stop conditions.
- Keeping information flowing while decisions, owners, and actions remain traceable.
- Converting a failed assumption into a process, metric, or validation improvement.
Questions to clarify before answering
- Which assumption failed, and what are the evidence confidence and time window?
- Is the impact a reversible experience regression or an irreversible data, safety, or compliance risk?
- How many users are exposed, and what is the smallest containment action?
- Who owns the final decision, and what escalation or rollback mechanisms exist?
- What judgment or action were you personally responsible for?
30-second answer framework
I would recheck the data and separate facts, inferences, and unknowns, then show the team the impact and stop conditions. I would propose reversible actions such as limiting traffic, delaying the launch, or running a smaller validation, and record the decision and owner. Once the impact was contained, I would identify the missing validation, turn it into a check or metric, and explain how my judgment changed.
Step-by-step deep dive
Step 1: Verify that the assumption failed
Check definitions, samples, experiment conditions, and timing with a relevant teammate. Write the failure as a falsifiable statement so one anomaly is not treated as a conclusion.
Step 2: Reduce the blast radius first
Choose a response that fits the risk: pause, reduce traffic, disable the risky path, or return to a known-safe version. State duration, success metric, and reversal method for each action.
Step 3: Move the discussion from blame to decision
Use a timeline to show what was visible then, what evidence arrived now, and what remains unknown. Describe system conditions and decisions, not a person’s ability or motive; defer accountability analysis until facts are stable.
Step 4: Enable an informed decision
Give the decision owner options, costs, risks, and stop conditions. For safety, compliance, or irreversible loss, follow the existing escalation path rather than relying on personal seniority.
Step 5: Publicly update your own judgment
State what you believed, why you believed it, and which evidence changed the conclusion. Owning an error keeps responsibility visible and makes it safer for others to add information.
Step 6: Close the learning loop
Record impact, timeline, decisions, signals, and action items with owners, dates, and verification. Turn recurring assumptions into pre-launch checks, experiment gates, or automated monitoring.
High-quality sample answer
Before a pricing launch, we assumed new users would follow the existing trial path. Gray-release data showed that most users dropped at step two. I checked the instrumentation and sample to rule out a measurement issue, then proposed stopping the traffic increase, keeping the old path, and interviewing a small user group. In the decision meeting I put the facts, unknowns, and three options on one page for the product owner. We found that copy and entry placement jointly caused the confusion. I acknowledged that I had treated the old path as a stable assumption too early, helped redesign the experiment, and added key-path validation to the launch checklist. The review focused on available information and system conditions; every action had an owner and acceptance metric.
Common mistakes
- Saying only “I was wrong” without evidence, containment, or a decision.
- Treating a failed assumption as proof that one person lacks ability.
- Expanding exposure after discovery without a stop condition.
- Describing team results without your own judgment, action, and boundary.
- Ending the review with slogans instead of an owner, date, and metric.
Follow-ups and responses
Follow-up 1: How do you distinguish a wrong assumption from poor execution?
Compare the expected premise, evidence, and execution record. If the premise was false, it is an assumption problem; if the premise held but the agreed action was not done, it is execution. Both can coexist and need separate actions.
Follow-up 2: What if the owner insists on launching?
Present impact, reversible options, and stop conditions, then ask the owner to state the accepted risk. Use the team escalation path and preserve the record when a safety, compliance, or irreversible threshold is crossed.
Follow-up 3: How do you keep the discussion from becoming blame?
Use a timeline and observable facts, separating information available then from hindsight. Contain the impact first; discuss process and decision conditions in the review.
Follow-up 4: What do you record?
Record the assumption, validation method, sample, impact, stop condition, decision owner, and outcome metric so another person can reproduce the reasoning.
Follow-up 5: What process did you change?
Name the new experiment gate, launch checklist, monitoring, or second review, and give a later result that shows the change was used.
Follow-up 6: What if users were already affected?
Stop further exposure, restore service, and notify affected parties with a clear timeline and remediation owner. Then run a blameless review for system improvements instead of delaying recovery for an argument about fault.