Definition
A randomized controlled experiment that exposes comparable groups to different variants and compares a predefined outcome.
Distinguish it from nearby terms
An offline benchmark compares systems on a fixed dataset. A/B testing estimates the effect of variants in an actual user or operational setting through randomized assignment.
Check your understanding
Choose the outcome, guardrails, population, and stopping rule before inspecting results to reduce biased conclusions.