# How to run a geo holdout test > Pause ads in a few matched states, keep them on elsewhere and compare orders. Six steps to run a geo holdout test, a worked rupee example and the mistakes to avoid. - **Author**: Marxx - **Published**: 2026-11-20 - **Category**: Testing - **URL**: https://marxx.ai/library-guides/how-to-run-a-geo-holdout-test --- To run a geo holdout test, keep your ads on in most regions and pause them in a few matched ones for three to four weeks. Then compare orders in both groups, using your own store data. The extra orders in the regions that kept ads are what your ads caused. A [geo holdout test](/glossary/holdout-test) answers one question: how many sales would I lose if I stopped these ads? ## Step 1: choose the metric Pick orders or revenue per region from your own store or CRM, not from Meta. Meta credits sales to ads. Your store data shows what happened. ## Step 2: choose matched regions Take the last 8 to 12 weeks of orders by state. Put states with similar size and trend into two groups, a test group that keeps ads and a holdout group that pauses them. Use states that sell enough each week to show a change of a few per cent. Avoid splitting a single city, since people cross the line. ## Step 3: set locations in Ads Manager At the ad set level, go to **Audience**, then **Locations**. Add only the test states as locations, or add the whole country and exclude the holdout states (checked 2026-10-20, Meta help). Meta says who sees an ad depends on current and home location. A person who lives in a shown state may see your ad while visiting a held-out one. Meta also says accuracy cannot be guaranteed and some impressions can land outside your location settings. Expect a little leakage and note it. ## Step 4: run it clean - Run for three to four whole weeks. - Change nothing else. No new offers or price cuts. - Keep the budget in test states steady. - Log any event that hits one group, such as a local festival or a stock-out. ## Step 5: measure Use the growth of the test group as the guide for what the holdout would have done. Expected holdout orders without the pause equal their old orders times the test group's growth. The gap between expected and actual is your lift. ## Step 6: turn it into a decision Multiply the extra orders by your average order value, then divide by the ad spend you paused. Compare the result with your [break-even ROAS](/glossary/break-even-roas). ## A worked example Say you run ads in 12 states. You pick 8 as the test group and 4 as the holdout, matched on 10 weeks of orders. For four weeks you pause ads in the 4 holdout states. Average order value is INR 1,500. | | Weekly orders before | Weekly orders during | Change | |---|---|---|---| | Test states (ads on) | 1,200 | 1,230 | Up 2.5% | | Holdout states (ads paused) | 600 | 540 | Down 10% | - Expected holdout orders at the test group's growth: 600 x 1.025 = 615. - Orders lost to the pause: 615 minus 540 = 75 a week, so 300 over four weeks. - Sales lost: 300 x INR 1,500 = INR 4,50,000. - Ad spend you paused: INR 2,00,000. - Incremental return: INR 4,50,000 / INR 2,00,000 = 2.25. If your gross margin is 40%, break-even ROAS is 1 / 0.40 = 2.5. At 2.25, ads in those states pay back less than they cost at the margin. Before you cut, repeat the test and look at repeat purchases. Say Meta had reported 3.5 in those states. The test shows how far platform credit and real lift can differ. ## Tools Meta's open-source [GeoLift package](https://github.com/facebookincubator/GeoLift) is written in R under the MIT licence. Its readme says it helps pick test markets with power calculators and measures lift with synthetic control methods (checked 2026-10-20). Paid tools are listed in [the best incrementality testing tools](/best/best-incrementality-testing-tools). For a lighter route, see [how to measure incrementality on a small budget](/guides/how-to-measure-incrementality-on-a-small-budget). ## Mistakes to avoid - Choosing regions by gut, not by past sales - Running one week, which is mostly noise - Starting a sale in only one group - Reading Meta's numbers instead of your store's - Treating one test as final ## FAQ ### How many regions do I need? At least two or three in the holdout group, and more where you can. One region is hard to trust, because a local event can swamp the result. ### How long should the test run? Three to four whole weeks. Longer helps if your buyers take time to decide. ### Is a geo holdout test the same as a geo-lift test? They are close. A [geo-lift test](/glossary/geo-lift-test) usually adds statistical modelling to pick regions and judge the gap. The basic idea is the same. --- - [More Testing articles](https://marxx.ai/library-guides/category/testing) - [All articles](https://marxx.ai/library-guides)