Trusting the numbers
The spat between the NFL and Nielsen is a timely reminder: never trust a number you didn't measure yourself
The news that the NFL and Nielsen are in dispute around precisely how to measure the number of people watching the Green Bay Packers on any given Sunday should hardly come as a surprise. Estimating TV audiences was never a settled science, and even if it was, a changed and fractured broadcast environment has brought with it new ways of measuring and new arguments about which approach is best.
I am not an expert in this area and I would not pretend to be. My interest in this news stems more from the fact that it illustrates what I consider to be an almost universal law of marketing, namely:
Never trust a number you didn’t measure yourself.
To be a little more specific, there are broadly two kinds of metrics out there:
Ones that you can measure for yourself. New leads. Pipeline. Total unique web visitors (although there are some major caveats about this one). Even followers if that’s what you’re into.
Ones that you have to take someone’s word for. “Reach” or “impressions” being prime examples.
It is awfully tempting to believe in, and report on, the latter group. The numbers are big, they sound impressive, and I am sorry to say that they will often be taken at face value by people who really should know better. I have seen CMOs, CEOs, investors and more willing to take the fact that 4 billion people saw our LinkedIn advert on trust.
But for all sorts of reasons I almost never report them, and don’t believe you should either. Here are three of the more important ones:
In digital media, ‘impressions’ is almost meaningless. In some cases it simply means an ad was downloaded. Whether it appeared on the screen or not is irrelevant. Even when impressions ARE related to genuine display the threshold is low (I think the rule is something like at least 50% of the advert on the screen for anything more than a second). And anyway, who is to say a user noticed it, or indeed appreciated it?
The sheer volume of bots out there means even if you could trust the numbers, you’d be right to wonder about whether they are humans or machines. This isn’t a knock on any individual platform or publisher, but the truth is that the space is so corrupted that assuming a certain amount of bot traffic is just a fact of life
Reach and impressions, other than for a very small subset of organisations, aren’t really what we are trying to achieve with our content or advertising anyway. If you start reporting for them, you start optimising for them. When you start optimising for them, you are spending time and money moving a number that doesn’t really have any relationship to the success of your business.
The joys of the real world
Fortunately there is a better way, and that is to focus on numbers that matter, and that we can trust (because we measured them ourselves). I’ve spoken about attribution previously, mostly to elaborate on just how complex an issue it is. But one thing is certainly true: always do it yourself rather than taking somebody else’s word for it.
Or to put that another way, even leaving aside impressions, don’t let LinkedIn tell you how many leads they delivered to your business. Work it out for yourself. Of course when you do so, you are also connecting that lead to the opportunity it becomes down the line, and the revenue it delivers in the future - all going well. A useful reminder that we are looking to measure and optimise what matters (revenue), not just whatever we can.
Like I say, I am not an expert when it comes to the economics of TV advertising, but it is surely no coincidence that organisations that DO advertise on TV are increasingly looking to track real world outcomes to that spend. I am assuming that is what explains the QR codes we see on ads today, or the invitation to download an app or even just search for a brand name. These are all things that can be measured, and tied to down-the-line behaviour.
In some cases, like the QR code the BBC show for paying the TV license, they may be spectacularly optimistic, but the intention is there. Those of us who are lucky to operate in environments where we can measure the things that matter, should make sure we make the most of that good fortune.
