This repository hosts the source code for ABTestCalculator.com, a visual A/B test significance calculator.
First, clone this repository and ensure you have npm installed.
Next, install the project's dependencies by running npm install.
Finally, simply run gulp from the command line and then open build/index.html in your browser.
Because this uses ECMAScript 2015, you'll need to ensure you run gulp with the --harmony flag. You can do this with the following alias:
alias gulp='node --harmony `which gulp`'
For an explanation of the math behind A/B testing significance calculations, checkout this excellent series by Aaron O'Connell:
- The Math of Split Testing Part 1: Statistical Sampling Uncertainty
- The Math of Split Testing Part 2: Chance of Being Better
- The Math of Split Testing Part 3: The Chance of being Similar
This tool uses the Wald method to estimate the distribution of the sample proportions and tests are considered significant at the p = 0.10 confidence level.
If you have any suggestions, find a bug, or just want to say hey drop me a note at @mhmazur on Twitter or by email at matthew.h.mazur@gmail.com.
MIT © Matt Mazur