A/B Test Calculator
Is that lift real, or did you just stop looking at a lucky moment?
IntermediateWeb appData viz8–18hAI-buildable 4/5no setupworth shipping
A significance calculator with sample size planning and a plain-language verdict. Its real job is to push back on the most common mistake in experimentation: peeking at results and stopping the moment they look good.
UI
Logic
Useful
Hosting · Free forever
Deploys free on GitHub Pages, Cloudflare Pages or Vercel Hobby.
Deploy it and share a link.
Core features · 7
- Enter visitors and conversions per variant
- Significance and confidence interval are calculated
- A plain-language verdict explains what the numbers mean
- Required sample size calculates from a target effect size
- A warning appears when the sample is below the planned size
- The distribution of both variants is plotted with overlap shown
- The scenario encodes into a shareable URL
The interesting part
- Implementing the statistics correctly, including the normal approximation's limits
- Explaining significance without reinforcing common misinterpretations
- Discouraging peeking without making the tool feel obstructive
Once it works
- Add Bayesian analysis alongside frequentist
- Add multiple variant comparison with correction
- Add sequential testing that permits valid early stopping
Suggested stack
reacttypescripttailwindrecharts