A/B testing is the gold standard for online experimentation used by most companies to test out their product features. Whereas A/B test experimentation works just fine in most settings, it is particularly susceptible to interference bias, particularly in the case of online marketplaces or social networks. In this article, we aim to look at the situations with interference bias and some potential ways to mitigate its effect on evaluation.

SUTVA, the Fundamental Assumption of A/B Testing and Its Violations

One of the fundamental assumptions of A/B Testing is SUTVA — Stable Unit Treatment Value Assumption. The potential outcome of treatment in the randomization unit depends only on the treatment they receive and not on the treatments assigned to other subjects.