In Defence Of The Humble Ad Pre-Test

In Defence Of The Humble Ad Pre-Test
SHARE
THIS



In this guest post, Visit Victoria’s head of research and insights, Julian Major (main photo), weighs in on the oft-loathed ad pre-test and says the practice doesn’t deserve the negative press it often receives…

Creatives don’t like ad-testing. A big generalisation, I know. But I can empathise. Someone works on a solution to a brief, only to be met with a poor response by consumers in an artificial and contrived environment. Recently, I’ve seen pre-testing described as ‘snake oil’. Is this actually the case?

Many criticisms around ad-testing are valid. The big question of whether performance in ad-testing is predictive of in-market success is incredibly difficult to prove. Many brands will not pre-test at all. Those who do may not launch an ad if it tests poorly, or significantly change it before launch. So, the final sample of ads in-market is biased. And that is before getting to the question of what defines in-market effectiveness. To prove that ad testing works is to prove an ad works in-market and while not impossible, will be difficult for many brands.

Research is an artificial task, and this is often referenced with focus groups. Being in a room with eight people, discussing in depth something you would pay scant attention to real life is unusual. So is answering questions in an online survey on an ad that someone has forced you to watch probably more than once. Some will also strap machines to you to get biometric responses, which is just a tad unusual.

So, I think we are right to be sceptical of any grand claims made by research agencies on the power of ad testing and it is not wrong to reference weaknesses of a methodology.

But does that mean there is no value in pre-testing? I do not think so. After all, we know the alternative is not great. Nicole Hartnett and others from the Ehrenberg Bass Institute found that marketers’ predictions around sales effectiveness of advertisements ‘were correct no more often than random chance’. Marketers are good at what they do but they are not advertising clairvoyants and we should be sceptical around people professing that expertise trumps all else.

Instead, we need to think more humbly about the value pre-testing provides.

While we know there is not a magic formula to create the perfect sales-effective ad, there are some basics that most would agree with. And pre-testing can help with these.

  1. An advertisement can not be effective if we do not know who it is for. Yet correct brand attribution for individual ads can be abysmal. This is easy to measure.
  2. Advertising needs to cut through. There are lots of ways to measure this. Some will espouse biometrics, but it’s also broadly accepted that advertising likeability is a valid measure. Sometimes, likeability will also help brand attribution by drawing more attention to the ad (Though it could have the opposite effect in some cases – all the more reason to test!)

Obviously, the actual measure of success is whether it changes a behaviour, but it is more difficult to understand this from a pre-test. You can ask a brand funnel measure pre and post exposure, or just simply ask directly, but this is where the criticisms around pre-testing become very legitimate. A single ad can not be expected to change brand preference (But rather, nudges propensities), and a persuasion model of advertising is flawed.

But testing can help us understand if the ad is liked, whether it is correctly attributed and if not, why. Testing should not be a pass/fail mechanism but a tool to help us build better ads. Does no one know the ad was for your brand after watching? If so, are there distinctive assets or more direct branding that can be used at key points? If humour is used, is this only funny to the people who wrote the joke? Do people simply not understand the message or attribute you are trying to push?

There is also a time and place, and a right way and a wrong way to do ad-testing. Qualitative and quantitative methods serve different purposes at different times and the answer of  which to use is always ‘it depends’. There is also the reality that financial restraints mean testing will not always be possible. If a test will cost a lot relative to your working media costs, then maybe leave it. But many brands will pump a lot of money into expensive and important ads. They are often part of a broader platform or campaign that adds up over time and a test may be a small cost to help improve something substantial. While the industry has rightly moved away from continual Link tests until a magic number is hit, we need to be wary about moving too far in the opposite direction. We can make research better, but it will never be flawless, and that is ok.

 

 

Please login with linkedin to comment

Julian Major visit victoria

Latest News

‘The Sims 4’ Now Lets You Use Your Own Pronouns For Your Characters
  • Technology

‘The Sims 4’ Now Lets You Use Your Own Pronouns For Your Characters

The latest entry in EA and Maxis’ life simulation series, The Sims 4, has released a new update which allows players to customise their character’s pronouns. This new feature is available only in the English version of the game for the time being, with the developers saying that they will soon release it for other […]

Samatha Armytage Shares Her Drunk At Work Fiasco While Interviewing Karl Stefanovic
  • Media

Samatha Armytage Shares Her Drunk At Work Fiasco While Interviewing Karl Stefanovic

Former Sunrise host Samantha Armytage has admitted that she has pulled a ‘Karl Stefanovic’ and has been drunk while on air. The former morning show golden girl interviewed Karl Stefanovic on her podcast, Something To Talk About With Samatha Armytage. It was a pretty interesting chat, considering Stefanovic and Armytage used to be breakfast show […]

NSW Government Launches New Sexual Consent Campaign Aimed At Under 25s
  • Campaigns

NSW Government Launches New Sexual Consent Campaign Aimed At Under 25s

A new public awareness campaign has launched which aims to educate sexually active young people around new consent laws. Called “Make no doubt” the ads are for the NSW Government and have been launched ahead of a major change to sexual consent laws set to come into effect in NSW from June the 1st. The […]

by B&T Magazine

B&T Magazine
Alexa Looks To Expand Her Knowledge Of Australia By Teaming Up With The Betoota Advocate
  • Campaigns
  • Technology

Alexa Looks To Expand Her Knowledge Of Australia By Teaming Up With The Betoota Advocate

Alexa continues to learn and lap up Australian culture, with Amazon today announcing the addition of new ‘Strayan slang to her vocabulary. From well-worn phrases like “Dog’s Eye” to emerging social-media slang like “Bachelor’s Handbag”, Alexa’s latest lessons are part of her ongoing Aussie education and make it even easier for her to understand her […]

Twitter Executives Unclear About Future Of Company Under Musk
  • Technology

Twitter Executives Unclear About Future Of Company Under Musk

During a meeting held by all shareholders of social media company Twitter, there were a lot of questions asked regarding the direction they would be taking and when the deal with Tesla owner Elon Musk would be finalised, but no real answers were given. Twitter CEO Parag Agrawal simply said that they were “…working through […]

Study: Over 90% Of Major US Brands Think Their Creative & Media Needs To Be Better Aligned
  • Media

Study: Over 90% Of Major US Brands Think Their Creative & Media Needs To Be Better Aligned

A new US study of 100 major brands with annual marketing budgets in excess of $US50 million ($A70 million) has found a worrying mismatch between creative and media spends. Creative management platform Smartly.io released the survey called Releasing the Power of Digital Creative. You can read the study HERE. While 89 per cent of the […]

by B&T Magazine

B&T Magazine
Bastion Asia Solidifies Its Strategic Partnership With Adventi, Shanghai
  • Marketing

Bastion Asia Solidifies Its Strategic Partnership With Adventi, Shanghai

Asian marketing communications agency Bastion Asia has announced a strategic partnership with one of Shanghai’s leading PR agencies, Adventi, to drive greater digital and marketing innovation for Australian brands seeking to engage with Chinese audiences. Adventi, a marketing and communications agency specialising in luxury brands, is known for bringing some of Europe’s biggest luxury brands […]

Engine Creative Brings Innovation To The Lost Person Poster Using Modern Day Technology
  • Campaigns
  • Technology

Engine Creative Brings Innovation To The Lost Person Poster Using Modern Day Technology

Engine Creative has reinvented the missing person poster – re-thinking the standard design and introducing a high-impact live portrait – for UK charity Missing People to coincide with International Missing Children’s Day. The striking, high-tech digital posters show a ‘live’ image of the missing person appearing to move their eyes, smile and make facial expressions. […]

Amplify Announces Three Senior Promotions & New Client Wins
  • Advertising

Amplify Announces Three Senior Promotions & New Client Wins

Amplify, the global creative agency specialising in experience and culture, has added Canva, PlayStation and Levi’s to its client roster and announced a string of senior promotions in a phase of strong business growth. The agency will provide brand experience services to the three global brands. Coinciding with significant new business momentum, Amplify has boosted […]

Spark Foundry Promotes Sue-Ellen Osborn & Greg Cattelain
  • Media

Spark Foundry Promotes Sue-Ellen Osborn & Greg Cattelain

Spark Foundry has announced two more key promotions, aimed at further strengthening its investment and digital capabilities. Effective immediately, Sue-Ellen Osborn (left in main photo) moves from her position as group activation director, to the elevated new role of Sydney head of investment. Meanwhile Greg Cattelain (right), formerly national head of programmatic, steps into the […]

LinkedIn Puts The Spotlight On Indigenous Australians In The Workplace With New Campaign By Campfire X For Reconciliation Week
  • Campaigns

LinkedIn Puts The Spotlight On Indigenous Australians In The Workplace With New Campaign By Campfire X For Reconciliation Week

LinkedIn, the world’s largest professional network, has today launched the latest iteration of its #ConversationsForChange initiative with a new content series in partnership with Indigenous-led creative consultancy, Campfire X. The three-part series, titled ‘Change Starts With’, aims to highlight the often subtle, everyday moments that can impact Indigenous Australians in the workplace. The campaign encourages […]

Microsoft Build 2022 Conference Unveils A Number Of Key Updates On Windows, Office, And More
  • Technology

Microsoft Build 2022 Conference Unveils A Number Of Key Updates On Windows, Office, And More

Microsoft’s Build conference for developers that took place yesterday showcased several important additions to the US tech giant’s PC software that will soon be released to the public. During the presentation, which was live-streamed for audiences all around the world, the company revealed their plans for various programs such as Teams. Microsoft will soon be […]

Australia’s Ad Standards Received 4675 Complaints In 2021, Up 33% Year-On-Year
  • Advertising

Australia’s Ad Standards Received 4675 Complaints In 2021, Up 33% Year-On-Year

Australia’s ad watchdog Ad Standards has released its annual Review Of Operations that provides an insight into advertising complaint statistics and key areas of work in 2021. Some 4675 advertising complaints were lodged last year, 33 per cent more than in 2020. The Ad Standards Community Panel considered a total of 315 cases and upheld […]

by B&T Magazine

B&T Magazine
Global Study Finds Top Challenges For Journalists & Uncovers Relationships With PRs
  • Media

Global Study Finds Top Challenges For Journalists & Uncovers Relationships With PRs

Cision, a communications platform helping public relations and marketing professionals around the world understand, influence and amplify their stories, released its annual State of the Media report with responses from more than 3800 journalists from 17 global markets to understand challenges faced in a marketplace that is rapidly evolving, and how public relations can support […]

The Six Habits Of Successful Entrepreneurs
  • Opinion

The Six Habits Of Successful Entrepreneurs

Alan Manly (main photo), OAM, is the CEO of Universal Business School Sydney (UBSS) and author of The Unlikely Entrepreneur. In this guest post, Manly offers tips from turning that side hustle into entrepreneurial greatness… Over the past few years, the success of leading businesses and entrepreneurs has been thrust into the spotlight like we […]

Opinion

by B&T Magazine

B&T Magazine
Metarise Create Innovation With Australia’s First-Ever Web3 Incubator
  • Technology

Metarise Create Innovation With Australia’s First-Ever Web3 Incubator

Collective Campus, Australian innovation consultancy and start-up accelerator, is spinning off Metarise – a web3 venture fund and tokenized start-up accelerator focused on bringing web3 projects to life across Asia-Pacific. Having run numerous web2 accelerators in partnership with brands such as Microsoft, Village Roadshow, Charter Hall, and Mills Oakley, Metarise’s ANZ Launchpad will be run […]

TikTok Unveils A Whole New Field Of Marketing Opportunities Through #TikTokMadeMeBuyIt
  • Campaigns
  • Technology

TikTok Unveils A Whole New Field Of Marketing Opportunities Through #TikTokMadeMeBuyIt

The popular video sharing platform TikTok is showing agencies its unique potential as a place to share their products with its latest #TikTokMadeMeBuyIt campaign, which is becoming increasingly appealing with audiences and users. During the pandemic, TikTok became one of the breakout social media outlets, rapidly rising in account numbers as people all across the […]