Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add target condition or percentage "lift" to A/B test success criteria #23

Open
roborourke opened this issue Apr 14, 2020 · 0 comments
Open

Comments

@roborourke
Copy link
Collaborator

roborourke commented Apr 14, 2020

Right now just achieving a statistically significant increase in a conversion rate doesn't really account for the desired amount of "lift" we might want.

A small increase in a value may not actually be enough to want to make changes on the back of.

This target lift should give us enough information to calculate a target sample size before the test starts if we're able to query the current baseline via elastic. This sample size should allow us to estimate a recommended amount of time to run the test for.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants