# What is a holdout test?

> A holdout test keeps a random group from seeing your ads, then compares their sales with people who did. It shows what your ads truly caused.
- **Author**: Marxx
- **Published**: 2026-10-26
- **Category**: Measurement
- **URL**: https://marxx.ai/library-glossary/holdout-test

---

A holdout test keeps a random group of people from seeing your ads, then compares their results with a group that did. The gap between the two groups is what your ads caused.

## How it works

1. Pick an audience, such as all past website visitors.
2. Split it at random. Keep the holdout small, such as 10%, so most people still see your ads.
3. Show ads only to the test group.
4. After two to four weeks, compare purchase rates.

## A worked example

You run a retargeting campaign to 50,000 past visitors.

- Test group: 45,000 people see ads. 1,350 buy, a 3.0% rate.
- Holdout group: 5,000 people see no ads. 125 buy, a 2.5% rate.

The ads lifted the purchase rate by half a point. So in the test group, about 0.5% x 45,000 = **225 sales** came from the ads. The other 1,125 would have happened anyway.

If the campaign cost INR 90,000, the cost per extra sale was INR 400. Ads Manager would have shown INR 67 per sale (90,000 / 1,350). The test shows the true cost is six times higher.

## When to run one

- Before you scale retargeting, which often looks better than it is
- When you doubt a channel's reported numbers
- Once a year on your biggest campaign, as a health check

## Tips

- Keep the holdout untouched: no emails or offers that the test group does not also get.
- Run long enough to collect at least a few hundred purchases in total.
- Write down what you expect before you look, so you do not explain away the result.

## FAQ

### Does Meta run holdout tests for me?

Meta offers conversion lift tests on eligible accounts, which compare a test group with a control group that does not see your ads. You can create one in the Experiments tool or ask your Meta account representative. As a guide, Meta asks for a campaign from the past year with $5,000 or more in spend and 500 conversions (checked 2026-10-20).

### Can I run one without a big budget?

Yes, using cities instead of people. Turn ads off in a few matched cities and compare. That is a [geo lift test](/glossary/geo-lift-test).

---
- [More Measurement articles](https://marxx.ai/library-glossary/category/measurement)
- [All articles](https://marxx.ai/library-glossary)