Statistical Power Statistical Power Calculator Estimate the power of a two-group experiment or A/B test from effect size, sample size, and alpha. Supports tests on means and on proportions. 1. Choose test type Difference in means (two-sample, equal n) Difference in proportions (two-sample, equal n) 2A. Inputs for means Example: control mean = 10, treatment mean = 12 → effect size = 2. SD is the common standard deviation. Mean (group A) Mean (group B) Standard deviation (common) Sample size per group 2B. Inputs for proportions Example: baseline conversion p1 = 0.10, new variant p2 = 0.13. Proportion p1 (group A) Proportion p2 (group B) Sample size per group 3. Test settings Alpha (significance) Tails Two-tailed One-tailed Calculate power 4. Results Estimated power — Effect size (absolute) — Indicative n per group for 80% power — What is statistical power? Statistical power is the probability that your test will detect an effect if the effect is really there. Low power → high chance of false negatives. Key determinants of power Effect size : bigger differences are easier to detect. Sample size : more data → narrower standard error → higher power. Alpha : a higher alpha (e.g. 0.1) makes it easier to reject H0, increasing power. Variability : lower standard deviation → higher power. Formula idea (z-approximation) For a two-sample test on means (equal n), the test statistic roughly follows z = (μ₂ − μ₁)

Calculators in Statistical Power Statistical Power Calculator Estimate the power of a two-group experiment or A/B test from effect size, sample size, and alpha. Supports tests on means and on proportions. 1. Choose test type Difference in means (two-sample, equal n) Difference in proportions (two-sample, equal n) 2A. Inputs for means Example: control mean = 10, treatment mean = 12 → effect size = 2. SD is the common standard deviation. Mean (group A) Mean (group B) Standard deviation (common) Sample size per group 2B. Inputs for proportions Example: baseline conversion p1 = 0.10, new variant p2 = 0.13. Proportion p1 (group A) Proportion p2 (group B) Sample size per group 3. Test settings Alpha (significance) Tails Two-tailed One-tailed Calculate power 4. Results Estimated power — Effect size (absolute) — Indicative n per group for 80% power — What is statistical power? Statistical power is the probability that your test will detect an effect if the effect is really there. Low power → high chance of false negatives. Key determinants of power Effect size : bigger differences are easier to detect. Sample size : more data → narrower standard error → higher power. Alpha : a higher alpha (e.g. 0.1) makes it easier to reject H0, increasing power. Variability : lower standard deviation → higher power. Formula idea (z-approximation) For a two-sample test on means (equal n), the test statistic roughly follows z = (μ₂ − μ₁).