A/B tests are often used as one-shot experiments for improving a product. In our paper (…), we describe how we use an AI technique called Bayesian optimization to adaptively design rounds of A/B tests based on the results of prior tests. Compared to a grid search or manual tuning, Bayesian optimization allows us to jointly tune more parameters with fewer experiments and find better values.
The post goes into examples and shares data from usage Facebook’s just-published methodology. Important.