How not to measure creative

We've all heard the "creative is X% of marketing impact" stat. Stop citing it. You'll never prove it for your own brand, and trying will cost you more than it teaches you. 

Just say creative is important and move on. Why?

1. 50% of what? CPA? ROAS? Volume? How are you going to report regularly on how well the creative drove ROAS or leads vs. the media? You won't. The idea that you'd give your creative team a goal of driving some defined amount of ROAS and the media team the balance is ludicrous. 

2. Even reasonably estimating creative impact in a test would mean keeping all other variables - media, notably - constant, and you won't do that. Are you really going to stop optimizing digital channels? And that doesn't even start to address other variables like competitive media spend, product changes, consumer sentiment, seasonality, etc. 

3. MMMs are deeply unlikely to be able to isolate creative impact. Believe me, I've tried. In theory, you could test creative performance variables from various tools and platforms, but they'd all need to be harmonized into a single "creative performance" variable across channels, with at least weekly frequency, to be vaguely useful in a model. It's hard to imagine that project ever making above the cutline.

None of this means creative isn't very important. Quite the opposite.

Creative impact measurement is likely the least sophisticated aspect of marketing right now. Every channel platform reports something different (often many different things at once). There are also third-party tools that report yet another long list of different metrics - often proprietary to that tool. And then there are surveys with an even longer list of metrics, none of which overlap with the first two examples.

You end up with a long deck of tables of often cherry-picked metrics woven into a hazy narrative that is unsatisfying for everyone.

There's a better way, but it requires the same level of triangulation and discipline that general media spend receives.