Ad recall, one of marketing’s most commonly used brand lift metrics, is wrong 80 per cent of the time, according to new research published by On Device. B&T chatted with Mark Griffiths, ANZ commercial director and Allan Breiland, head of research at On Device, about the data and its implications for advertisers in the Australian market.
The global study analysed more than 2,000 brand lift campaigns worldwide, comparing traditional ad recall results with passively measured ad exposure. It found that recall-based measurement routinely misidentifies who has or hasn’t seen an ad, inflating uplift figures and distorting campaign results.
The findings cut right to the heart of how the industry measures effectiveness, and why advertisers are moving towards passive methodologies that deliver more trustworthy insights.
Breiland says it’s “beyond” him why some marketers still use ad recall.
“Ad recall has never been a reliable metric; it’s a flawed approach. It gives you misleading data – it doesn’t relate to whether you were exposed to activity or not,” he said.
Griffiths added: “Memory is messy and error-prone. People recall ads they didn’t see, forget ads they did, and sometimes confuse one brand for another. When your exposed and control groups are built on these shaky foundations, your measurement is fundamentally flawed.
Our analysis proves that ad recall is not fit for purpose as a proxy for exposure. If advertisers want measurement they can trust to guide budgets, passive methodologies are by far the most reliable solution.”
The study also uncovered the impact of customer selection bias. Analysis of 15 passive-measurement studies found ad recall for existing customers was on average 3.1 times higher than for non-customers, despite both groups having comparable exposure to the campaigns. In practice, this means that in an ad recall exposed group, seven in ten respondents are likely to be existing customers, meaning ad recall often measures pre-existing brand relationships rather than genuine lift.
It also found that cross-media campaigns are more effective at driving brand lift than single media campaigns. When you look at digital or video on its own, and then look at them as part of a cross-media campaign, they’re outperforming single-media channels, with a 2.8-times improvement in brand consideration, and 2.3-times in intent.
“When we measure campaigns, we look at the whole market and funnel. Awareness, consideration, and intent, people’s attitude to creative. Often, the media is great but then we find that the creative is not performing very well. You start to draw those conclusions. Looking at that multi-channel approach, recognising that each one has its own function. It’s about clients recognising the channels that are right for them,” Griffiths said.
The Risks of Relying on Ad Recall
Part of the reason why ad recall might have stuck around is due to its simplified approach, and oftentimes, positive results.
“A lot of the time, people want to believe it works. If they get a good number and ad recall, you might ask someone who loves your brand anyway because it’s validating. It’s been set in people’s ways in the past, especially when TV was dominant,” Griffiths said.
“People are so keen nowadays on having instant results. They want to make decisions straight away, which probably worked five or six years ago when people were spending more on performance. That’s shifted now, and brand has become important again post-COVID.
In the performance field, you want to make quick decisions and change them 24 hours later. The whole point of a brand campaign, though, is that it takes time to build memory so that people are passively and subconsciously building that brand consideration. If you’ve only waited a week and you’re making decisions based on data from so early on, that’s quite crazy.”
Griffiths advised marketers to have good sample sizes and analyse how a campaign is performing over time, before making any decisions.
“When you start to look at how all the channels are working together, you might start to see patterns among them and make decisions that way – dialling up money in one channel, and down in another. If you’re making decisions on unreliable data, you could be costing businesses millions of data,” he added.
Breiland said marketers risk drawing the wrong conclusions and spending their money where it’s ineffective.
Investing In The Right Measurements
A lot of people are moving away from ad recall and moving to more reliable and robust approaches, such as passive measurement. This is based on factual consumer behaviour tracking, rather than consumer claims. It uses methods such as live GPS tracking for the real-time location of the consumer and real-time audio signals that recognise the ad being consumed.
“Passive measurement is where it’s going, and ideally is across all touch points. This approach is very viable in digital, outdoor, and media, and even TV now since they’re all connected. This is where we see the industry moving in terms of the best approach measurement,” Breiland said.
Griffiths said that that paying to set up passive measurement, where people give up their permissions for privacy, location data and device IDs may cost more than ad recall studies, but pays off in the long run.
“It might be a little more than what you were paying in the past, but the reality is you’re getting a fuller picture – you’re getting to understand how your different media channels are working. There’s a balance there. I’m looking at it from a sales and commercial perspective, and I would say that even from five years ago, it’s a lot cheaper now to do these kinds of studies and getting the results. The learnings from these are so much more sophisticated, and it allows the clients to implement them into their next campaigns,” he said.
“Ultimately, any client should be doing this. Market mix modelling is all the rage at the moment, and there’s lots of stuff going on with that, but brand lift is still very important, because it allows people to see how those campaigns are performing. It allows them to dial in, dial out, change components of a campaign and bring those learnings into that bigger business picture.”
With Australian digital ad spend reaching $17.2 billion in FY25, up 10.6 per cent year on year, accuracy in measurement has never been more important. Digital video continues to lead that growth, accounting for 29 per cent of total online ad spend and increasing 21.9 per cent to $5 billion.
“Australian marketers are increasingly demanding accuracy over ease. Choosing recall-based methods because they’re cheaper or simpler only creates more problems down the track. Clean, passive measurement not only gives marketers confidence in their results but also clarity on what truly drives performance and long-term growth,” Griffiths said.
