Prompt and context
You are two weeks from launching an important feature. The interviewer asks you to run a pre-mortem, convert possible failures into validation work and accountable owners, then decide whether to expand after launch. This tests whether you can connect discovery, launch operations, and measurable decisions instead of listing generic risks.
What the interviewer is testing
The core ability is risk-to-decision translation. A strong answer distinguishes assumptions from evidence, assigns a named owner to each material risk, defines guardrails before exposure, and sets a reversible rollout path. It also shows that user value and operational safety are evaluated together.
Clarifying questions to ask first
Ask who the target users are, what problem the feature solves, and what “important” means in business terms. Clarify whether the launch is a limited beta or a broad release, what systems or commitments can be affected, and which teams own product, engineering, support, legal, and operations. Ask what baseline metrics and rollback capability already exist.
A 30-second answer framework
Say: “I would start with a pre-mortem that imagines the launch failed, group the failure modes by user value, adoption, reliability, and compliance, and rank them by impact and evidence. For each high-severity risk I would assign an owner, a falsifiable validation, a leading signal, a guardrail, and a rollback action. I would launch to a representative slice with an explicit duration, review absolute guardrails and user-centered metrics, then make a documented Go/No-Go decision before expanding.”
Step-by-step deep analysis
1. Imagine failure and state assumptions
Write a concrete failure statement, such as “After seven days, activation is flat, support contacts doubled, and p95 latency breaches the service objective.” Separate assumptions about user behavior, data quality, capacity, and policy from facts already measured. This prevents the workshop from becoming a free-form brainstorm.
2. Rank risks and assign owners
Use impact, likelihood, reversibility, and evidence strength to rank risks. A high-impact irreversible failure belongs in the first validation wave. Assign one directly responsible owner, a due date, and the decision they can unblock; list collaborators separately so accountability is unambiguous.
3. Define signals, metrics, and guardrails
For each assumption, define a user-centered goal, observable signal, and metric. HEART is a useful structure: Happiness, Engagement, Adoption, Retention, and Task success. Pair target metrics with absolute guardrails such as error rate, latency, complaint rate, or opt-out rate. A target is a reason to expand; a guardrail breach is a reason to pause or roll back.
4. Design a reversible exposure plan
Use a canary or staged rollout: choose a representative population, exposure percentage, duration, time window, and on-call owner before enabling it. Start with a small slice that limits blast radius. Define the stop condition and rollback command in advance, and record who can invoke it without waiting for a large meeting.
5. Decide Go/No-Go and learn
At the review point, compare the pre-registered thresholds with observed data. Go means target metrics meet the bar and no absolute guardrail is breached. No-Go means pause, roll back, or keep the feature at its current exposure while the owner tests a named hypothesis. Record the decision, evidence, and next checkpoint so the team can revisit it without rewriting history.
6. Make follow-up work explicit
Turn every unresolved risk into a dated action with a single owner and a measurable exit condition. Keep the exposure level, evidence, and decision log together so the next review can compare like with like and avoid reopening settled assumptions.
High-quality sample answer
“I would invite product, engineering, support, and compliance owners to a 45-minute pre-mortem. We would imagine the launch failed after one week and write specific failures. Suppose our top risks are low task success, a 20% increase in support contacts, and a latency regression. I would assign one owner to each, validate task success with five moderated sessions plus an instrumented funnel, and verify capacity with a load test. Before launch I would register a target activation metric and absolute guardrails for p95 latency, error rate, and support contacts. I would canary 5% of representative users for 24 hours during staffed coverage, with rollback ready. If activation improves while guardrails hold, I would expand to 25% and review again. If a guardrail breaches, I would pause or roll back, publish the evidence, and let the owner test the corresponding fix before the next Go/No-Go review.”
Common mistakes and improvements
- Brainstorming without decisions: Convert every serious risk into an owner, validation, threshold, and action.
- Only measuring conversion: Add user-centered task success and happiness signals plus reliability and support guardrails.
- Launching to everyone first: Use staged, representative exposure with a defined duration and blast-radius limit.
- Vague rollback language: Name the command, owner, trigger, and maximum response time before launch.
Follow-up questions and responses
What if stakeholders reject a pre-mortem as slowing delivery?
Time-box it, focus on the three highest-impact assumptions, and show the decision each validation enables. A short pre-mortem can remove rework without becoming a launch approval committee.
How do you choose the first canary population?
Choose a slice that represents the intended workload and user mix while limiting blast radius. Exclude known incompatibilities, document the sampling rule, and avoid selecting only friendly internal users unless that is the explicit test objective.
What if target metrics improve but a guardrail worsens slightly?
Do not average away an absolute safety threshold. Pause at the pre-registered boundary, investigate segmentation and causality, and expand only after the owner demonstrates a safe fix or the threshold is formally re-approved.
How do you communicate a No-Go decision?
State the threshold, observed evidence, owner, immediate action, and next checkpoint in writing. Frame No-Go as a controlled learning decision, then return with new evidence rather than defending the original launch date.