-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
André Ekeberg
committed
Apr 28, 2020
0 parents
commit 06fdf29
Showing
16 changed files
with
3,173 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,7 @@ | ||
# OS files | ||
.DS_Store | ||
Thumbs.db | ||
|
||
# Composer | ||
vendor | ||
composer.lock |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,9 @@ | ||
# Changelog | ||
|
||
All notable changes to this project will be documented in this file. | ||
|
||
## [1.0.0] - 2020-04-29 | ||
|
||
Initial release | ||
|
||
[1.0.0]: https://github.com/andreekeberg/abby/releases/tag/1.0.0 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,21 @@ | ||
MIT License | ||
|
||
Copyright (c) 2020 André Ekeberg | ||
|
||
Permission is hereby granted, free of charge, to any person obtaining a copy | ||
of this software and associated documentation files (the "Software"), to deal | ||
in the Software without restriction, including without limitation the rights | ||
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell | ||
copies of the Software, and to permit persons to whom the Software is | ||
furnished to do so, subject to the following conditions: | ||
|
||
The above copyright notice and this permission notice shall be included in all | ||
copies or substantial portions of the Software. | ||
|
||
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR | ||
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, | ||
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE | ||
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER | ||
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, | ||
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE | ||
SOFTWARE. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,198 @@ | ||
# 🙋♀️ Abby | ||
|
||
Abby is a simple but powerful A/B testing library. | ||
|
||
The library lets you easily setup your **experiments** and their **control** and **variation** groups, **track** your visitors and assign them to a group, get detailed statistics including recommended **sample sizes** and determining the confidence of your results, including when an experiment have achieved **statistical significance**. | ||
|
||
The confidence is calculated using the [z-score](https://en.wikipedia.org/wiki/Standard_score) and [p-value](https://en.wikipedia.org/wiki/P-value) of your results, to see if the [null hypothesis](http://en.wikipedia.org/wiki/Null_hypothesis) can be rejected. An accompanying minimum [sample size](https://en.wikipedia.org/wiki/Sample_size_determination) is also calculated using a [two-tailed test](https://en.wikipedia.org/wiki/One-_and_two-tailed_tests) to control the [false discovery rate](https://en.wikipedia.org/wiki/False_discovery_rate). | ||
|
||
Abby is dependency free, and completely database agnostic, meaning it simply works with data you provide it with, and exposes a variety of methods for you to store the result in your own storage of choice. | ||
|
||
## Requirements | ||
|
||
- PHP 5.4.0 or higher | ||
|
||
## Installation | ||
|
||
``` | ||
composer require andreekeberg/abby | ||
``` | ||
|
||
## Basic usage | ||
|
||
### Tracking a user | ||
|
||
```php | ||
// Setup a new Token instance | ||
$token = new Abby\Token(); | ||
|
||
// If we can't find an existing token cookie, generate one and set tracking cookie | ||
if (!$token->getValue()) { | ||
$token->generate()->setCookie(); | ||
} | ||
|
||
// Setup a User instance | ||
$user = new Abby\User(); | ||
|
||
// Associate the token with our user | ||
$user->setToken($token); | ||
``` | ||
|
||
### Adding existing user experiments to a user instance | ||
|
||
```php | ||
// List of experiments associated with a tracking token | ||
$data = [ | ||
[ | ||
'id' => 1, | ||
'group' => 1, | ||
'converted' => false | ||
] | ||
]; | ||
|
||
// Loop through users existing experiments and add them to our user instance | ||
foreach ($data as $item) { | ||
// Setup experiment instance based on an existing experiment | ||
$experiment = new Abby\Experiment([ | ||
'id' => $item['id'] | ||
]); | ||
|
||
// Setup a group instance based on stored data | ||
$group = new Abby\Group([ | ||
'type' => $item['group'] | ||
]); | ||
|
||
// Add the experiment (including their group and whether they have | ||
// already converted) to our user instance | ||
$user->addExperiment($experiment, $group, $item['converted']); | ||
} | ||
``` | ||
|
||
### Including the using in new experiments | ||
```php | ||
// Experiment data | ||
$data = [ | ||
'id' => 2 | ||
]; | ||
|
||
// Make sure the experiment isn't already in the users list | ||
if (!$user->hasExperiment($data['id'])) { | ||
// Setup a new experiment instance | ||
$experiment = new Abby\Experiment([ | ||
'id' => $data['id'] | ||
]); | ||
|
||
// Assign the user to either control or variation in the experiment | ||
$group = $user->assignGroup($experiment); | ||
|
||
// Add the experiment (including assigned group) to our user instance | ||
$user->addExperiment($experiment, $group); | ||
} | ||
|
||
// Getting updated user experiment list | ||
$user->getExperiments(); | ||
|
||
// Store updated experiment list for our user | ||
``` | ||
|
||
### Delivering a custom experience based on group participation | ||
|
||
```php | ||
// Experiment data | ||
$data = [ | ||
'id' => 1 | ||
]; | ||
|
||
// If the user is part of the variation in our experiment | ||
if ($user->inVariation($data['id'])) { | ||
// Apply a custom class to an element, load a script, etc. | ||
} | ||
``` | ||
|
||
### Defining a user conversion in an experiment | ||
|
||
```php | ||
// Experiment data | ||
$data = [ | ||
'id' => 1 | ||
]; | ||
|
||
// On a custom experiment goal, check if user is a participant and define a conversion | ||
if ($user->isParticipant($data['id'])) { | ||
$user->setConverted($data['id']); | ||
} | ||
|
||
// Getting updated user experiment data | ||
$user->getExperiments(); | ||
|
||
// Store updated data for our user | ||
``` | ||
|
||
### Getting experiment results | ||
|
||
```php | ||
// Setup experiment instance with stored results | ||
$experiment = new Abby\Experiment([ | ||
'groups' => [ | ||
[ | ||
'name' => 'Control', | ||
'size' => 3000, | ||
'conversions' => 300 | ||
], | ||
[ | ||
'name' => 'Variation', | ||
'size' => 3000, | ||
'conversions' => 364 | ||
] | ||
] | ||
]); | ||
|
||
// Retrieve the results | ||
$result = $experiment->getResult(); | ||
|
||
// Get the winner | ||
$winner = $result->getWinner(); | ||
|
||
/** | ||
* Get whether we can be confident of the result (even if we haven't | ||
* reached the minimum group size for each variant) | ||
*/ | ||
|
||
$confident = $result->isConfident(); | ||
|
||
/** | ||
* Get the minimum sample size required for each group to reach statistical | ||
* significance, given the control groups current conversion rate (based on | ||
* the configured minimumDetectableEffect) | ||
*/ | ||
|
||
$minimum = $result->getMinimumGroupSize(); | ||
|
||
/** | ||
* Get whether the results are statistically significant | ||
*/ | ||
|
||
$significant = $result->isSignificant(); | ||
|
||
/** | ||
* Get complete experiment result | ||
*/ | ||
|
||
$summary = $result->getAll(); | ||
``` | ||
|
||
## Documentation | ||
|
||
* [Experiment](docs/Experiment.md) | ||
* [Token](docs/Token.md) | ||
* [User](docs/User.md) | ||
* [Group](docs/Group.md) | ||
* [Result](docs/Result.md) | ||
|
||
## Changelog | ||
|
||
Refer to the [changelog](CHANGELOG.md) for a full history of the project. | ||
|
||
## License | ||
|
||
Abby is licensed under the [MIT license](LICENSE). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,50 @@ | ||
{ | ||
"name": "andreekeberg/abby", | ||
"type": "library", | ||
"description": "🙋♀️ Minimal A/B Testing Library", | ||
"keywords": [ | ||
"ab test", | ||
"ab testing", | ||
"conficence", | ||
"conversion rate", | ||
"conversion ratio", | ||
"conversions", | ||
"false discovery rate", | ||
"p-value", | ||
"probability value", | ||
"probability", | ||
"ratio", | ||
"sample size", | ||
"sample", | ||
"sequential testing", | ||
"significance", | ||
"split test", | ||
"split testing", | ||
"split-run", | ||
"standard score", | ||
"statistics", | ||
"testing", | ||
"two-sample", | ||
"two-tailed", | ||
"two-tailed test", | ||
"z-score" | ||
], | ||
"homepage": "https://github.com/andreekeberg/abby", | ||
"license": "MIT", | ||
"authors": [ | ||
{ | ||
"name": "André Ekeberg", | ||
"email": "andre@pocketsize.se", | ||
"homepage": "https://github.com/andreekeberg", | ||
"role": "Developer" | ||
} | ||
], | ||
"autoload": { | ||
"psr-4": { | ||
"Abby\\": "src/" | ||
} | ||
}, | ||
"require": { | ||
"php": ">=5.4.0" | ||
} | ||
} |
Oops, something went wrong.