Incrementality Testing vs Attribution: Measuring What Actually Causes Conversions

Introduction –

Marketing measurement has become increasingly sophisticated, but one fundamental question remains difficult for many organizations to answer: what actually caused a customer to convert? Businesses can now track impressions, clicks, website visits, email interactions, social media engagement, lead submissions, purchases, and numerous other customer activities. However, simply knowing what happened before a conversion does not necessarily tell us what caused that conversion.

This distinction has made the debate around Incrementality Testing vs Attribution increasingly important. Attribution attempts to distribute credit among marketing touchpoints that appear in a customer’s journey, while incrementality testing focuses on determining whether a marketing activity created additional conversions that would not have happened otherwise. The difference may appear subtle, but it can significantly influence how businesses evaluate campaigns and allocate marketing budgets.

For example, imagine that a customer sees a retargeting advertisement and later purchases a product. An attribution system may give credit to the retargeting campaign because the advertisement appeared before the purchase. But what if the customer had already decided to buy the product before seeing that advertisement? In that situation, the campaign may receive attribution credit without actually causing the conversion.

This is why modern marketing teams increasingly need to move beyond the question of “Which channel gets credit?” and start asking “What changed because we invested in this channel?”

What Is Marketing Attribution?

Marketing attribution is a measurement approach that attempts to determine which marketing interactions contributed to a customer’s conversion. A customer may interact with several channels before making a purchase, such as paid search, organic search, social media, email, display advertising, webinars, content, or direct visits. Attribution models attempt to distribute credit across these interactions according to predefined rules or statistical models.

For example, a customer may first discover a company through a Google search, later download an ebook after seeing a social media post, receive several marketing emails, attend a webinar, and eventually speak with a salesperson before becoming a customer. An attribution model attempts to determine how much credit each interaction should receive for the eventual conversion.

Different attribution models produce different results. First-touch attribution gives most or all of the credit to the first interaction, while last-touch attribution gives credit to the final measurable interaction before conversion. Linear attribution distributes credit across multiple touchpoints, while more advanced data-driven approaches use statistical techniques to estimate the contribution of different interactions.

Attribution can therefore be extremely useful for understanding customer journeys and marketing touchpoints. However, it has an important limitation: it generally tells marketers what happened around a conversion, not necessarily what would have happened if the marketing activity had never occurred.

What Is Incrementality Testing?

Incrementality testing takes a different approach. Instead of asking which marketing touchpoint should receive credit, it asks whether the marketing activity actually changed customer behavior.

The fundamental question is simple: Would this customer have converted if they had not been exposed to the marketing activity? Answering this question requires some form of comparison between customers who received the marketing intervention and customers who did not.

A typical incrementality experiment divides an eligible audience into a treatment group and a control group. The treatment group receives the campaign, while the control group is intentionally excluded from it. If the treatment group produces a meaningfully higher conversion rate than the control group, the difference can provide evidence that the marketing activity generated incremental conversions.

For example, suppose 10,000 customers are included in a campaign experiment. Five thousand customers receive the campaign, while another 5,000 comparable customers do not. If 6% of the treatment group converts while only 4% of the control group converts, the two-percentage-point difference represents the estimated incremental lift, assuming the experiment has been properly designed and the result is statistically reliable.

Incrementality Testing vs Attribution: The Fundamental Difference –

The most important distinction between Incrementality Testing vs Attribution is the difference between correlation and causation. Attribution identifies relationships between marketing interactions and conversions, while incrementality attempts to determine whether the marketing intervention actually caused additional behavior.

Consider a customer who has visited a product page three times, searched for the brand by name, read several reviews, and added a product to their shopping cart. Shortly afterward, that customer sees a retargeting advertisement and completes the purchase. A last-touch attribution model may assign the conversion to the retargeting advertisement.

However, it is entirely possible that the customer was already going to purchase. The advertisement may have played little or no causal role in the final decision.

Incrementality testing attempts to answer this problem by comparing the behavior of customers who received the advertisement with similar customers who did not. If both groups convert at almost the same rate, the campaign may have generated substantial attributed conversions but little incremental business.

Why Attribution Can Overstate Marketing Performance –

One of the biggest problems with attribution is that customers who are already highly likely to convert can make campaigns appear more effective than they actually are. This is especially common with branded search, retargeting, email remarketing, and other channels that frequently interact with customers who are already near the bottom of the funnel.

Imagine a customer who has already decided to purchase a product. They search for the company’s brand, click a paid search advertisement, visit the website, and complete the purchase. A last-click attribution system may report that paid search generated the conversion.

From an attribution perspective, the result makes sense because paid search was the final measurable interaction. From a causal perspective, however, the conclusion is less certain. The customer may have purchased even if the advertisement had never appeared.

This is why marketers should be careful when interpreting statements such as “This campaign generated 10,000 conversions.” The data may actually show that 10,000 customers converted after interacting with the campaign. Those two statements are not necessarily equivalent.

Why High-Intent Customers Create a Measurement Problem –

High-intent customers create one of the biggest challenges for marketing measurement because they are already more likely to convert than the average customer. When these customers are exposed to marketing, their subsequent purchases can easily be attributed to the campaign even when the campaign had limited influence on their decision.

Retargeting provides a simple example. A customer visits an ecommerce website and looks at a particular product. Later, the customer sees an advertisement for that same product and eventually purchases it. The retargeting campaign may receive conversion credit, but the customer’s original website visit may have been a much stronger indication of purchase intent.

Incrementality testing can help separate existing demand from demand that was actually created or influenced by marketing. This makes it particularly valuable when companies are deciding whether additional spending on a channel will genuinely produce additional revenue.

Attribution Is Still Valuable –

The limitations of attribution do not mean that companies should abandon it. Attribution remains valuable because it provides visibility into customer journeys and helps marketing teams understand how different channels interact with prospects and customers.

For example, attribution can help a marketing team discover that customers who eventually become high-value accounts frequently interact with webinars, technical content, email campaigns, and sales outreach. That information can be useful when designing future customer journeys and content strategies.

The key is to understand what attribution can and cannot tell you. Attribution is excellent for understanding which interactions occurred around conversions. Incrementality is better suited to understanding whether marketing activity caused additional conversions.

The two approaches should therefore be viewed as complementary rather than mutually exclusive.

How Incrementality Testing Measures Lift –

Incrementality testing typically focuses on incremental lift, which represents the difference in outcomes between a treatment group and an appropriate control group.

For example, assume that customers exposed to a campaign have a conversion rate of 7%, while comparable customers in the control group have a conversion rate of 5%. The absolute incremental lift is 2 percentage points.

The relative lift would be calculated by comparing the difference with the control group’s baseline performance. In this example, the campaign generated a 40% relative improvement over the control conversion rate.

However, the numbers alone are not enough. Marketing teams also need to consider sample size, statistical significance, experiment duration, audience quality, seasonality, and other variables that can influence the results.

How Incrementality Testing Works –

A well-designed incrementality test starts with a clearly defined business question. Instead of simply asking whether a campaign produced conversions, the marketing team should identify exactly what causal question it wants to answer.

The eligible audience is then divided into appropriate treatment and control groups. The treatment group receives the marketing activity, while the control group remains unexposed or receives the normal baseline experience.

Both groups are monitored during the experiment, and the resulting business outcomes are compared. Depending on the campaign, the measurement could focus on purchases, leads, revenue, pipeline, retention, subscriptions, or another meaningful business metric.

The quality of the experiment depends heavily on the quality of the control group. If the treatment and control populations are fundamentally different, the resulting difference may reflect audience characteristics rather than marketing impact.

Common Types of Incrementality Tests –

• Randomized Controlled Tests –

Randomized controlled experiments are one of the strongest approaches to measuring incremental impact. Customers are randomly assigned to treatment and control groups, which helps reduce selection bias and creates more comparable populations.

These experiments can be particularly useful for advertising, promotional campaigns, email marketing, and other situations where individual-level testing is technically possible.

• Holdout Tests –

Holdout testing involves intentionally excluding a portion of an eligible audience from a marketing campaign. The behavior of the holdout group can then be compared with customers who received the campaign.

This approach is often useful for retention campaigns, email programs, customer lifecycle marketing, and advertising experiments.

• Geo-Based Testing –

When individual-level experimentation is difficult, marketers can use geographic regions as treatment and control groups. For example, a company might increase advertising investment in selected cities while maintaining normal marketing activity in other comparable locations.

The resulting differences in sales, leads, or other outcomes can then be analyzed to estimate incremental impact.

Incrementality in B2B Marketing –

Incrementality is particularly valuable for B2B organizations because B2B buying journeys are often long, complex, and influenced by multiple stakeholders. A potential customer may interact with advertising, content, webinars, sales representatives, events, product demonstrations, and multiple members of the buying committee before an opportunity is created.

Traditional attribution may assign credit across those touchpoints, but it may not reveal which activities actually increased the probability of creating or progressing the opportunity.

For example, a B2B company could test whether exposing target accounts to a particular content campaign increases the rate at which those accounts become sales opportunities. Another experiment could determine whether a specific advertising strategy increases pipeline creation compared with a similar group of accounts that was not exposed.

This allows B2B marketing teams to connect marketing activity with actual changes in account behavior rather than simply counting interactions.

Attribution and Incrementality Should Work Together –

The most effective measurement strategy does not necessarily require companies to choose between attribution and incrementality. Instead, organizations can use attribution to understand customer journeys and incrementality to validate causal impact.

For example, attribution might reveal that paid social frequently appears in the customer journey before conversions. That observation can be useful, but it does not prove that paid social caused those conversions.

The marketing team can then conduct an incrementality experiment by comparing similar audiences that are and are not exposed to the campaign. If exposed customers demonstrate a statistically meaningful increase in conversion, the organization has stronger evidence that paid social is generating incremental value.

This combination creates a more reliable measurement framework because attribution provides descriptive insight, while experimentation provides causal evidence.

The Future of Marketing Measurement –

The future of marketing measurement is likely to place greater emphasis on causal analysis. Privacy changes, fragmented customer journeys, reduced availability of individual-level tracking, and increasingly complex advertising ecosystems are making traditional measurement more difficult.

Artificial intelligence can help marketers analyze large amounts of data, identify patterns, predict customer behavior, and optimize campaigns. However, prediction should not be confused with causation. An AI model may identify that customers exposed to a particular campaign are more likely to convert, but that does not necessarily prove that the campaign caused the additional conversions.

Experiments and well-designed measurement frameworks remain essential for answering causal questions. AI can make these systems more efficient, but the fundamental question remains the same: What would have happened if we had not taken this marketing action?

Building a Better Conversion Measurement Strategy –

Companies should also establish clear baselines and use control groups whenever practical. Measuring performance without a counterfactual makes it difficult to determine what would have happened naturally.

Finally, marketing teams should connect measurement to meaningful business outcomes. Clicks and impressions can be useful diagnostic metrics, but the ultimate evaluation should consider outcomes such as incremental revenue, profitable customer acquisition, retention, pipeline creation, and customer lifetime value.

Common Mistakes Companies Make –

One common mistake is treating attribution as proof of causation. A customer interacting with an advertisement before converting does not automatically mean the advertisement caused the purchase.

Another mistake is failing to account for existing customer intent. Customers who are already highly likely to buy can make campaigns appear extremely successful when measured only through attributed conversions.

Companies can also make mistakes by running experiments that are too small, too short, or poorly controlled. Seasonality, competing campaigns, pricing changes, and external events can all affect the results.

The most important principle is to design measurement around the question the organization actually needs to answer rather than simply using the metrics that are easiest to collect.

Conclusion –

Incrementality Testing vs Attribution is ultimately a question about how seriously an organization wants to understand the true impact of its marketing investment. Attribution provides valuable information about customer journeys and helps marketers understand which touchpoints are associated with conversions. But association alone does not establish causation.

Incrementality testing takes the analysis further by asking what would have happened without the marketing activity. Through treatment and control groups, holdouts, geographic experiments, and other methodologies, organizations can estimate the additional conversions, revenue, or pipeline actually created by marketing.

The strongest marketing organizations will not necessarily abandon attribution. Instead, they will use attribution to understand the journey and incrementality to validate the impact. This combination provides a more complete view of marketing effectiveness.

Ultimately, the goal should not be to give every marketing channel as much credit as possible. The goal should be to understand which investments actually change customer behavior and create additional business value.

That is the difference between measuring conversions and understanding what actually causes them.

Frequently Asked Questions –