What's the Difference Between "Traffic Allocation" and "A/B Split" in A/B Testing? A Guide to Getting the Setup Right
- ab-testing
- cro
- how-to
Summary
- Traffic allocation = what percentage of all visitors are included in the experiment
- A/B split = within that experiment group, what ratio of visitors see A versus B
- These are two different things. Keeping them separate lets you test with less risk
When setting up an A/B test, it's easy to get confused because two different "percentage" settings show up: traffic allocation (the share included in the experiment) and the A/B split. These two settings mean completely different things. Understanding them separately lets you run experiments safely.
What Is "Traffic Allocation"? How Many People Get Pulled Into the Experiment
Traffic allocation is the percentage of all visitors who are included in the experiment.
- Set it to 100%, and every visitor sees either A or B.
- Set it to 20%, and only one-fifth of visitors are included in the experiment, while the remaining 80% see the normal (unchanged) page.
The purpose of limiting traffic allocation is risk management. Rolling out a change with an unknown effect to every single visitor right away can be risky. Instead, you can try it on a small group first and expand once you're confident it's working well.
What Is the "A/B Split"? The Ratio Between A and B
The A/B split is the ratio at which A and B are shown to the visitors who are included in the experiment.
- 50/50 is the standard choice. It's a fair comparison, and it lets you collect the data you need faster.
- If you're unsure about the change, you can set A (the original) to a larger share and B (the new version) to a smaller share, rolling it out more cautiously.
What Happens When You Combine the Two?
For example, with "20% traffic allocation, 50:50 A/B split," the breakdown looks like this:
| Visitors | Percentage |
|---|---|
| Not in the experiment (normal page = unchanged) | 80% |
| In the experiment, seeing A (Control) | 10% |
| In the experiment, seeing B (Variant B) | 10% |
The key point is that you can independently decide "how many people to include" (traffic allocation) and "how to show it to them within that group" (A/B split).
Why Is Separating the Two a Good Idea?
If traffic allocation and the A/B split are bundled into a single setting, two distinct goals â "keeping risk low" and "comparing fairly" â get mixed together, making the setup confusing.
By keeping them separate, you can:
- Use traffic allocation to control the size of the risk, and
- Use the A/B split to maintain fairness in the comparison,
so you can configure each setting according to its own purpose. This is also the approach used by leading A/B testing tools.
How HeatMapX Handles This
In HeatMapX's A/B testing, you can likewise set "the percentage included in the experiment (traffic allocation)" and "the A/B split" separately. You can start with a small traffic allocation to test safely, then expand it as you gain confidence in the results.
Summary
Traffic allocation determines how many people are pulled into the experiment; the A/B split determines how A and B are shown to that group. Keeping these two concepts separate is the key to testing correctly while keeping risk under control.