X-Factor: 1 — Incrementality: 0

“Stay curious.” It’s a simple idiom rooted in great thought.

“So what?” An equally valuable sentiment, rooted in the same great thought.

Incrementality, the sort-of-new-kid-on-the-block in advertising measurement, is neither curious or explanatory. There are real-world limits on its utility.

Incrementality attempts to isolate the change in a result, or the impact of a channel or campaign or tactic, by asking ‘what if’ that spend were non-existent. It’s a good first-level thought, but almost always needs to be placed in context. The notion makes sense: If we can isolate both the spend and the result, and identify a correlation between the two, we can say “if a, then b.”

But, that logic reaches its limits quickly, when second-level questions remain unasked, or context is lacking.

So, here’s the next question to ask: Does the conclusion (explanation) stick out above the noise? To see if it does, try an X-Factor challenge:

If we isolate 1 percent of the results, and 1 percent of the spend, we might identify a strong correlation between the two. It may look overwhelming, when we compare the 1 percent of results with the 1 percent of the spend.

But, so what? What if there’s just 1 percent noise or uncertainty in the overall data set?

What if an X Factor of just 1 percent of the total is included in the results? Do the results still stick out? Do they extend above or below the noise?

What if we assume — as we should — that the data has some inherent variability and volatility?

What if we are concerned about the quality of the data, or its independence and bias?

A problem with incrementality is that it does not challenge itself. If the process finds a simple answer, it stops there, and then extends that logic, untested, beyond its knowledge.

Incrementality is looking for change on change: That might actually occur, but some sense of confidence and context is necessary before that insight is extended beyond the test. If the identified result can be explained just as easily with a 1 percent X-Factor, confidence should be pretty low.

So, incrementality is a useful tool in an advanced measurement set. But only with context, curiosity and pragmatic, genuine skepticism.