Skip to content
- Clear hypothesis, target metric, and minimum detectable effect (MDE).
- Event tracking implemented and QA’d.
- Define hypothesis and success criteria; pre-register analysis plan.
- Estimate sample size using baseline and MDE; set duration.
- Randomize traffic and ensure mutual exclusion with other tests.
- Monitor data quality; avoid peeking and stopping early.
- Analyze with appropriate statistical test; report lift and confidence.
- Balanced cohorts, no tracking gaps, assumptions met.
- Insufficient power: extend duration or increase MDE.
- Sample ratio mismatch (SRM): check bucketing and filters.
- 1–3 weeks typical. Improves decision quality and learning speed.
