Tag Archives: A/B Testing

Size Matters, particularly in prediction

A follow-up on my previous post on statistical significance and hypothesis-testing: Let’s say we pull a number of samples, as in the previous post, from both a control group and a test group. Let’s also say that for the samples from … Continue reading

Posted in A/B Testing, Bayes, Data Analytics, Data Driven Management, Probability, Simulation, Statistics | Tagged , , , , , | Leave a comment

Lies,damn lies, statistics,predictions – the world is more random than you might think…

For a number of years ago, John Ionnidis published a paper claiming to prove that most research papers are in fact wrong.  That is, the findings of many/most research papers can actually not be reproduced by other, independent teams. According … Continue reading

Posted in A/B Testing, Bayes, Data Driven Management, Numpy, Probability, Python, Research, Simulation, Statistics | Tagged , , , , , , , , , | Leave a comment

A/B-testing on trendline data using Bayesian Inference

Asume we are in the process of doing evidence based testing of two competing strategies, “A” and “B”, and we want to evaluate these competing strategies over several days, weeks, months or whatever timespan we have decided upon. That is, … Continue reading

Posted in A/B Testing, Bayes, Data Analytics, Data Driven Management, Numpy, Probability, Python, Simulation, Statistics | Tagged , , , , | Leave a comment

Bayesian A/B-testing, part III

This final part on Bayesian A/B-testing will continue looking at the various assumptions, implicit or explicit, that always are in play when building statistical models.  In part II, we looked at what impact larger data sets have on our inferences, … Continue reading

Posted in A/B Testing, Bayes, Data Analytics, Data Driven Management, Math, Numpy, performance, Probability, Pystan, Python, Simulation, STAN, Statistics | Tagged , , , , , , , , , | Leave a comment

Bayesian A/B-testing, part II

Continuing my example by examining how the different assumptions – yes, in any model there are always assumptions, explicit or implicit – of the model impact the end result, that is, the prediction of the sought after signup-rate, a.k.a our posterior … Continue reading

Posted in A/B Testing, Bayes, Behavioral Economics, Data Analytics, Data Driven Management, Math, Numpy, Probability, Pystan, Python, Simulation, STAN, Statistics | Tagged , , , , , , , , | Leave a comment

A/B-testing with Bayesian Inference

The example Inspired by Rasmus Bååth’s lectures on Bayesian Inference, I’ve implemented a simple Python example demostrating how Bayesian Inference can be used for A/B-testing, that is, evidence based testing. This methodology, i.e. A/B-testing, is useful in most domains, e.g. … Continue reading

Posted in A/B Testing, Bayes, Behavioral Economics, Big Data, Data Analytics, Data Driven Management, Math, Numpy, performance, Probability, Pystan, Python, Simulation, STAN, Statistics | Tagged , , , , , , , , , , , | Leave a comment