Real Startup Book
A/B Testing
A/B testing (also known as split testing and bucket testing) is a randomized method of comparing two versions of an element (A and B) against each other to determine which one performs better using a metric to define success.
Blog Posts & Articles
Validation
Content
A/B Testing
A/B testing is not so good for testing:
- New things (such as change of version or novelty effect).
- Too many changes, as the results will not be conclusive.
- If something is missing (such as feature, style, information).
Field Tips
- Keep your A/B Testing variations to a minimum to ensure meaningful results - @sircastel
- Define your metrics and minimum success rate before running A/B Testing - @sircastel
- Got a tip? Add a tweetable quote by emailing us: [email protected]
Case Studies
- VWO - Website Redesign Increased Conversions
- VWO - SaaS Pricing A/B Testing
- Airbnb - Experiments at Airbnb
- Grab - How Grab Experimented with Chat to Drive Down booking Cancellations
- Got a case study? Add a link by emailing us: [email protected]
Tools
- Optimizely - A/B Testing
- Got a tool to recommend? Add a link by emailing us: [email protected]
References
- Wikipedia - A/B Testing
- Udacity - Online A/B Testing Course
- Evan Miller - A/B Testing Sample Size Calculator Tool
- MailChimp - How Long Should You Run Your A/B Test?
- Got a reference? Add a link by emailing us: [email protected]