Growth Systems Library
Incrementality Testing
Incrementality testing is the method of measuring whether your advertising actually causes conversions — or whether those customers would have bought anyway. Without it, every attribution model — last-click, multi-touch, or platform-reported — tells you how credit was assigned, not whether that spending created any additional revenue.
Incrementality testing works by creating two groups: one that sees your advertising as normal (the exposed group) and one that does not (the holdout group). The difference in conversion rate between the two groups is the incremental lift — the portion of revenue that your media actually caused.
The test eliminates the most common distortion in growth measurement: correlation mistaken for causation. A customer who sees your retargeting ad and converts may have purchased regardless — because they already had high intent. Attributing that conversion to the ad inflates its measured efficiency and leads to over-investment in channels that are observing demand rather than creating it.
Exactius deploys incrementality testing as a foundational input into the Capital Allocation Loop — the data layer of the Growth Operating System that determines how media budget is allocated across channels.
Most companies discover through their first incrementality test that 20–50% of the conversions attributed to paid media are not incremental. They would have happened without any spend. This means a significant portion of their paid media budget is not generating new revenue — it is capturing revenue that organic, brand, or direct would have captured anyway.
The downstream effect on unit economics is significant. When non-incremental conversions inflate your channel's reported ROAS, your blended CAC looks better than it is. You scale a channel based on false signal, contribution margin erodes, and the business grows revenue while losing efficiency.
For any company optimising toward LTV:CAC — the ratio Exactius uses as its primary growth signal — incrementality testing is not optional. It is the mechanism that ensures the LTV:CAC signal is real, not a measurement artefact.
The core formula:
Incremental lift = (Conversion rate, exposed group) − (Conversion rate, holdout group)
Incremental conversions = Lift × Total exposed audience size
True incremental CPA = Total media spend ÷ Incremental conversions
Three common test designs
1. Geo holdout test — Split markets geographically — run media in some regions, suppress it in others. Compare conversion rates between regions, controlling for baseline differences. Best for upper-funnel channels (TV, YouTube, display) where user-level holdouts are difficult.
2. User-level holdout (Ghost bids) — Use platform tools (Meta Conversion Lift, Google Brand Lift) to randomly exclude a percentage of your target audience from seeing ads. Compare conversion rates between exposed and unexposed groups. Most precise for lower-funnel paid social.
3. Time-based (hold-out period) — Pause a channel entirely for a defined period and compare performance against a comparable period. Simplest to run, but hardest to control for external variables (seasonality, competitive activity).
What breaks the test
- Holdout group too small (under 10% of audience) — results are not statistically significant
- Testing during anomalous periods (promotional events, competitor launches)
- Not accounting for view-through conversions contaminating the holdout
- Platform-reported lift studies that use the platform's own attribution — always use independent measurement where possible
Benchmark: A well-run incrementality test across a mid-size DTC business typically finds that 25–40% of attributed conversions are non-incremental. Retargeting campaigns consistently show the highest non-incrementality rates.
Most brands have never run an incrementality test. They are flying on attribution data that tells them which channel got credit — not which channel created demand. The result is predictable: over-investment in retargeting and bottom-of-funnel channels that harvest intent rather than generate it, and under-investment in channels that actually move the market.
Exactius builds incrementality testing into the Growth Infrastructure layer from the first month of engagement. The outputs feed directly into the Capital Allocation Loop — the mechanism that governs where the next dollar of media spend goes. When you know which channels are genuinely driving new demand, the capital allocation decision becomes empirical rather than directional.
The Growth Operating System, developed by David Manela, treats incrementality data as a prerequisite for any meaningful LTV:CAC signal. An LTV:CAC ratio built on non-incremental attribution is not an efficiency metric — it is a confidence interval around a fiction.
Exactius embeds growth squads that operate incrementality testing as a continuous practice, not a one-time audit. Results are refreshed quarterly as audience composition, creative mix, and market conditions shift.
What is incrementality testing in digital marketing?
Incrementality testing measures whether your advertising is the actual cause of a conversion, or whether the customer would have converted anyway without seeing the ad. It works by comparing conversion rates between an exposed group (saw your ads) and a holdout group (did not see your ads). The difference in conversion rate is the incremental lift — the revenue your media actually created. Exactius uses incrementality testing as the foundational input into media budget allocation decisions.
How is incrementality testing different from attribution?
Attribution assigns credit for a conversion to one or more touchpoints in the customer journey. Incrementality testing measures whether any of that credit represents real causal impact. A customer can touch five channels before converting, and attribution models will distribute credit across them — but if that customer had a 95% probability of converting regardless, none of those channels were meaningfully incremental. Incrementality answers the question attribution cannot: did this spend create demand that would not have existed otherwise?
How do you run an incrementality test without a large budget?
The minimum viable incrementality test is a geo holdout: pause spending in one or two comparable markets for 2–4 weeks and compare conversion rates against markets where you continued spending. You need enough volume in each geography to reach statistical significance — typically 100+ conversions per cell per week. Platforms like Meta and Google also offer built-in conversion lift studies that require no minimum spend, though their results use the platform's own attribution methodology, which introduces some bias. For an independent read, use your own analytics or a third-party measurement tool.
Exactius Growth
Your growth system is either compounding or degrading.
Book a diagnostic call. We'll identify where your growth system is breaking.
Book a call