FastTool
Start toolDataAll tools

EXPERIMENT DECISION FIREWALL

A/B Test False Discovery Firewall

Before shipping a winning A/B test, check whether the win survives sample-ratio mismatch, false-discovery correction, practical effect thresholds and guardrail harm.

Reviewed 2026-07-08

Data
SRM p-valueBH false discovery controlGuardrail blocker checkDecision receipt

Find experiment wins that should not ship yet.

Use the sample, randomize an experiment, or paste aggregate metric rows. The tool separates raw significance from corrected, practical and guardrail-safe decisions.

Result appears here after you run the false-discovery firewall.

WHY THIS IS DIFFERENT

It blocks the tempting but unsafe “p-value win.”

A real ship decision needs more than raw p<0.05. This tool checks allocation health, multiple comparisons, practical size and whether a guardrail metric moved the wrong way.