In Defence Of The Humble Ad Pre-Test

In this guest post, Visit Victoria’s head of research and insights, Julian Major (main photo), weighs in on the oft-loathed ad pre-test and says the practice doesn’t deserve the negative press it often receives…
Creatives don’t like ad-testing. A big generalisation, I know. But I can empathise. Someone works on a solution to a brief, only to be met with a poor response by consumers in an artificial and contrived environment. Recently, I’ve seen pre-testing described as ‘snake oil’. Is this actually the case?
Many criticisms around ad-testing are valid. The big question of whether performance in ad-testing is predictive of in-market success is incredibly difficult to prove. Many brands will not pre-test at all. Those who do may not launch an ad if it tests poorly, or significantly change it before launch. So, the final sample of ads in-market is biased. And that is before getting to the question of what defines in-market effectiveness. To prove that ad testing works is to prove an ad works in-market and while not impossible, will be difficult for many brands.
Research is an artificial task, and this is often referenced with focus groups. Being in a room with eight people, discussing in depth something you would pay scant attention to real life is unusual. So is answering questions in an online survey on an ad that someone has forced you to watch probably more than once. Some will also strap machines to you to get biometric responses, which is just a tad unusual.
So, I think we are right to be sceptical of any grand claims made by research agencies on the power of ad testing and it is not wrong to reference weaknesses of a methodology.
But does that mean there is no value in pre-testing? I do not think so. After all, we know the alternative is not great. Nicole Hartnett and others from the Ehrenberg Bass Institute found that marketers’ predictions around sales effectiveness of advertisements ‘were correct no more often than random chance’. Marketers are good at what they do but they are not advertising clairvoyants and we should be sceptical around people professing that expertise trumps all else.
Instead, we need to think more humbly about the value pre-testing provides.
While we know there is not a magic formula to create the perfect sales-effective ad, there are some basics that most would agree with. And pre-testing can help with these.
- An advertisement can not be effective if we do not know who it is for. Yet correct brand attribution for individual ads can be abysmal. This is easy to measure.
- Advertising needs to cut through. There are lots of ways to measure this. Some will espouse biometrics, but it’s also broadly accepted that advertising likeability is a valid measure. Sometimes, likeability will also help brand attribution by drawing more attention to the ad (Though it could have the opposite effect in some cases – all the more reason to test!)
Obviously, the actual measure of success is whether it changes a behaviour, but it is more difficult to understand this from a pre-test. You can ask a brand funnel measure pre and post exposure, or just simply ask directly, but this is where the criticisms around pre-testing become very legitimate. A single ad can not be expected to change brand preference (But rather, nudges propensities), and a persuasion model of advertising is flawed.
But testing can help us understand if the ad is liked, whether it is correctly attributed and if not, why. Testing should not be a pass/fail mechanism but a tool to help us build better ads. Does no one know the ad was for your brand after watching? If so, are there distinctive assets or more direct branding that can be used at key points? If humour is used, is this only funny to the people who wrote the joke? Do people simply not understand the message or attribute you are trying to push?
There is also a time and place, and a right way and a wrong way to do ad-testing. Qualitative and quantitative methods serve different purposes at different times and the answer of which to use is always ‘it depends’. There is also the reality that financial restraints mean testing will not always be possible. If a test will cost a lot relative to your working media costs, then maybe leave it. But many brands will pump a lot of money into expensive and important ads. They are often part of a broader platform or campaign that adds up over time and a test may be a small cost to help improve something substantial. While the industry has rightly moved away from continual Link tests until a magic number is hit, we need to be wary about moving too far in the opposite direction. We can make research better, but it will never be flawless, and that is ok.
Latest News

Wednesday TV Wrap: Charlie Pickering’s The Weekly Continues To Deliver But Nine Snags The Night
The ABC’s content continues to deliver bringing the network a strong 20 per cent of the share – We love to see homegrown content soaring! Seven News hit 1,007,000 viewers while Nine News earned 888,000 viewers. Nine’s A Current Affair scored 653,000 viewers – Grimshaw as always brings in big numbers. As per OzTAM’s metro […]

‘The Sims 4’ Now Lets You Use Your Own Pronouns For Your Characters
The latest entry in EA and Maxis’ life simulation series, The Sims 4, has released a new update which allows players to customise their character’s pronouns. This new feature is available only in the English version of the game for the time being, with the developers saying that they will soon release it for other […]

Sky News’ Paul Murray Gets Blasted Online For Launching Resistance Campaign Against Labor Government
Rupert Murdoch’s brainchild Sky News is being blasted online after airing a “Resistance Starts Now!” mantra in reaction to the Labor party’s election win. It kicked off on Paul Murray Live, Murray himself started his usual show by calling it, ‘the first meeting of the resistance.’ Which is very dramatic but okay. The slogan for […]

Samatha Armytage Shares Her Drunk At Work Fiasco While Interviewing Karl Stefanovic
Former Sunrise host Samantha Armytage has admitted that she has pulled a ‘Karl Stefanovic’ and has been drunk while on air. The former morning show golden girl interviewed Karl Stefanovic on her podcast, Something To Talk About With Samatha Armytage. It was a pretty interesting chat, considering Stefanovic and Armytage used to be breakfast show […]

Butterfly Enlists QMS To Highlight Eating Disorders That Kill More Aussies Than The Road Toll
An outdoor media campaign to raise awareness of eating disorders – a mental illness that kills more people than the national road

NSW Government Launches New Sexual Consent Campaign Aimed At Under 25s
A new public awareness campaign has launched which aims to educate sexually active young people around new consent laws. Called “Make no doubt” the ads are for the NSW Government and have been launched ahead of a major change to sexual consent laws set to come into effect in NSW from June the 1st. The […]

Alexa Looks To Expand Her Knowledge Of Australia By Teaming Up With The Betoota Advocate
Alexa continues to learn and lap up Australian culture, with Amazon today announcing the addition of new ‘Strayan slang to her vocabulary. From well-worn phrases like “Dog’s Eye” to emerging social-media slang like “Bachelor’s Handbag”, Alexa’s latest lessons are part of her ongoing Aussie education and make it even easier for her to understand her […]

Twitter Executives Unclear About Future Of Company Under Musk
During a meeting held by all shareholders of social media company Twitter, there were a lot of questions asked regarding the direction they would be taking and when the deal with Tesla owner Elon Musk would be finalised, but no real answers were given. Twitter CEO Parag Agrawal simply said that they were “…working through […]

Ipsos Survey Shows Aussies Aren’t As Excited By The Metaverse And VR/AR Environments As People From Other Countries
Less than half the Australian population (44 percent) are familiar with the metaverse, while only 36 percent express positive feelings about engaging with extended reality in daily life, a new global survey conducted by Ipsos for the World Economic Forum finds. Globally, about half of adults across 29 countries say they are familiar with the […]

Study: Over 90% Of Major US Brands Think Their Creative & Media Needs To Be Better Aligned
A new US study of 100 major brands with annual marketing budgets in excess of $US50 million ($A70 million) has found a worrying mismatch between creative and media spends. Creative management platform Smartly.io released the survey called Releasing the Power of Digital Creative. You can read the study HERE. While 89 per cent of the […]

Bastion Asia Solidifies Its Strategic Partnership With Adventi, Shanghai
Asian marketing communications agency Bastion Asia has announced a strategic partnership with one of Shanghai’s leading PR agencies, Adventi, to drive greater digital and marketing innovation for Australian brands seeking to engage with Chinese audiences. Adventi, a marketing and communications agency specialising in luxury brands, is known for bringing some of Europe’s biggest luxury brands […]

Engine Creative Brings Innovation To The Lost Person Poster Using Modern Day Technology
Engine Creative has reinvented the missing person poster – re-thinking the standard design and introducing a high-impact live portrait – for UK charity Missing People to coincide with International Missing Children’s Day. The striking, high-tech digital posters show a ‘live’ image of the missing person appearing to move their eyes, smile and make facial expressions. […]

VERSA Lands Jen Cerqualia As Client Strategy Partner
With this new position, Cerqualia has now assembled the digital Infinity Gauntlet, and will wield its power malevolently.

Amplify Announces Three Senior Promotions & New Client Wins
Amplify, the global creative agency specialising in experience and culture, has added Canva, PlayStation and Levi’s to its client roster and announced a string of senior promotions in a phase of strong business growth. The agency will provide brand experience services to the three global brands. Coinciding with significant new business momentum, Amplify has boosted […]

Spark Foundry Promotes Sue-Ellen Osborn & Greg Cattelain
Spark Foundry has announced two more key promotions, aimed at further strengthening its investment and digital capabilities. Effective immediately, Sue-Ellen Osborn (left in main photo) moves from her position as group activation director, to the elevated new role of Sydney head of investment. Meanwhile Greg Cattelain (right), formerly national head of programmatic, steps into the […]

Toyota Corolla Gets Brand New Platform Thanks To HERO Agency
Unfortunately, a new campaign doesn't do away with the stereotype of Corolla owners all being terrible drivers.

LinkedIn Puts The Spotlight On Indigenous Australians In The Workplace With New Campaign By Campfire X For Reconciliation Week
LinkedIn, the world’s largest professional network, has today launched the latest iteration of its #ConversationsForChange initiative with a new content series in partnership with Indigenous-led creative consultancy, Campfire X. The three-part series, titled ‘Change Starts With’, aims to highlight the often subtle, everyday moments that can impact Indigenous Australians in the workplace. The campaign encourages […]

Microsoft Build 2022 Conference Unveils A Number Of Key Updates On Windows, Office, And More
Microsoft’s Build conference for developers that took place yesterday showcased several important additions to the US tech giant’s PC software that will soon be released to the public. During the presentation, which was live-streamed for audiences all around the world, the company revealed their plans for various programs such as Teams. Microsoft will soon be […]

Australia’s Ad Standards Received 4675 Complaints In 2021, Up 33% Year-On-Year
Australia’s ad watchdog Ad Standards has released its annual Review Of Operations that provides an insight into advertising complaint statistics and key areas of work in 2021. Some 4675 advertising complaints were lodged last year, 33 per cent more than in 2020. The Ad Standards Community Panel considered a total of 315 cases and upheld […]

Global Study Finds Top Challenges For Journalists & Uncovers Relationships With PRs
Cision, a communications platform helping public relations and marketing professionals around the world understand, influence and amplify their stories, released its annual State of the Media report with responses from more than 3800 journalists from 17 global markets to understand challenges faced in a marketplace that is rapidly evolving, and how public relations can support […]

The Six Habits Of Successful Entrepreneurs
Alan Manly (main photo), OAM, is the CEO of Universal Business School Sydney (UBSS) and author of The Unlikely Entrepreneur. In this guest post, Manly offers tips from turning that side hustle into entrepreneurial greatness… Over the past few years, the success of leading businesses and entrepreneurs has been thrust into the spotlight like we […]

New Australian Circuits Announced For Cash Converters Supercars Eseries
The resident motorsports writer at B&T nearly broke his desk in excitement when this announcement was made.

Metarise Create Innovation With Australia’s First-Ever Web3 Incubator
Collective Campus, Australian innovation consultancy and start-up accelerator, is spinning off Metarise – a web3 venture fund and tokenized start-up accelerator focused on bringing web3 projects to life across Asia-Pacific. Having run numerous web2 accelerators in partnership with brands such as Microsoft, Village Roadshow, Charter Hall, and Mills Oakley, Metarise’s ANZ Launchpad will be run […]

TikTok Unveils A Whole New Field Of Marketing Opportunities Through #TikTokMadeMeBuyIt
The popular video sharing platform TikTok is showing agencies its unique potential as a place to share their products with its latest #TikTokMadeMeBuyIt campaign, which is becoming increasingly appealing with audiences and users. During the pandemic, TikTok became one of the breakout social media outlets, rapidly rising in account numbers as people all across the […]

Kim Kardashian Has Gone Viral After Being Caught Pretending To Eat A Burger In New Ad
Here at B&T we say a day without some Kardashian news is a day without sunshine. Get an instant tan with this stupidity.

Amanda Keller Tears Up While Discussing The Latest Tragic Mass Shooting In America
Amanda Keller gets teary over latest mass school shooting in the US. But what other response is there to the madness?

Creating The “Future-Fit” Marketing Partner
Amelia Thackway is here to explain how companies can search for long-term prospects in new partners after covid.

Nutella And The Oodie Team Up To Provide Some Warmth During Winter
Never mind covering yourself in chocolate, the new sexy food item is a warm jumper with Nutella logos all across it.

Tuesday TV Wrap: The Chase Grabs More Eyeballs Than Big Brother
Not watching this year's series of Big Brother? Well, get set for a guilt trip on these underwhelming numbers.

BBC Apologises For “Any Offence Caused” After “Manchester United Are Rubbish” Appears On Screen
Live gaffes B&T'd like to see are "Socceroos are rubbish", "Kyrgios is rubbish" & "Ryan Papenhuyzen's hair is rubbish!"

Is Smartphone Brand Loyalty Being Challenged In Australia? Find Out In Teads’ Mobile Intenders Study!
Get a first look at the Teads Mobile Intenders study here! It's digital, so arguably not ideal toilet reading material.

Flourish PR Lands Multiple New Clients
Flourish PR's local bottlo flush out of Moët piccolos as agency celebrates raft of new wins.

DDB Sydney Launches The Australiano Coffee For McCafé
DDB Sydney unveils uniquely Australian coffee for American franchise burger chain.

Snap CEO’s Warning Causes Social Media Stock Prices To Plummet
Snap boss Evan Spiegel issues investor warning, as wife Miranda forced to use E10 in the Rangie.

Craig Reucassel To Dissect Political & Social Issues With An All-Star Panel At VIVID
VIVID's not all shiny lights on buildings, it's also biting social discourse. But for most parts, it's shiny lights.