Market Research: Is There Just Too Much of It?
In his latest post, B&T regular Robert Strohfeldt argues that despite all the market research, is it really telling adland what it needs to hear?
Change. The one variable that is constant. And usually, change means better. No one changes to something that was worse than what they had before. For those who hanker for the “good old days”, think about what it would have been like to go to the dentist 100 years ago?
DAYS OF INNOCENCE
Way back in 1981, when I first joined a market research firm fresh out of university, I was engaged initially as a mathematician to ensure proper random sampling.
Research firms had their own field force back then and most quantitative surveys were done door to door, face to face. We worked from Collection Districts from the ABS. A major issue was cost – an interviewer was given a strict set of dwellings and if nobody, or the appropriate person, was not home, they were to try three times before replacement a dwelling was allowed.
Decent sample size, quantitative studies were too expensive for all but the large companies. Anecdotal findings, coupled with experience, were often all small businesses had.
There was also the obscene amount of paper used – imagine a six-page questionnaire for 1000 respondents.
The number of successful interviews achieved in a day was “bugger all” and a sample of 500 or 1000, took ages just to complete the field work. And of course, the cost was often seen as prohibitive for the perceived results. (That the results many times reinforced what the marketing team believed they already knew, was deemed a waste of money).
In the early 80’s there were not any PCs – coded questionnaires (coding is a valuable art that has been lost) were taken to a company who produced a computer print- out. Often, after working through the first run, additional reports were produced – “Can we see how the breakdown how of respondents in question four answered question six?”. On top of this any statistical analysis was done separately, again after all report runs had been completed.
Often, the alternative was “Let’s just run a few groups”. Great, an-depth discussions with 30 odd people was not really an accurate indication of anything, bar the lack of understanding of how research was best used.
Many clients would baulk at spending $30,000-$50,000 to optimise a $5 million campaign.
By 1983 I had graduated to running groups and have fond memories of running focus groups in peoples’ kitchens. The respondents were genuinely thrilled that large consumer companies sought out their opinions and engaged in discussions accordingly.
It was a novelty and as such they gave honest answers and when the topic was exhausted, that was it, the group was over. (Easy to spot) Yes, they were paid, but many times I had respondents state they would have participated even if not paid.
Many golden nuggets of consumer thinking (and statements, or in today’s terminology, content) were gleaned from them.
COST AND TECHOLOGY. AGENTS OF CHANGE
Telephone surveys took over from door to door sampling – no- where near the cost of sending interviewers out into the field.
For market researchers, their most valuable natural resource is respondents. Telephone interviews soon became problematic thanks to telemarketers.
“Hi, I am doing a survey on house cladding”. Several question later they knew your type of dwelling and if you were a candidate for cladding. And in no- time there was a knock at the door, or sometimes a phone call, from a person trying to sell you cladding.
And as telemarketing grew, many people had experienced having their dinner disturbed by someone trying to sell them something, camouflaged as market research.
“Piss off” became a frequent response. Many households grew tired of receiving telephone calls, be they from a genuine research organisation or a telemarketer.
By this time, PCs were in all research companies and the questionnaires were on screen. As the interviewer “filled in” the questionnaire on screen, the first run of data was being done simultaneously. Not anything to marvel at today, but when this first became available, it was revolutionary and research companies tried to push all quantitative surveys into telephone surveys.
But all the while, researchers’ most valuable natural resource, willing and uncorrupted respondents, was gradually being eaten away.
WHO HAS NOT BEEN SURVEYED?
Fast forward to 2019 and I doubt there is a person over the age of 18 who has not been surveyed in some form. As more and more businesses move online, they see it as an opportunity to tag a few research questions at the end.
The quality of the research results is totally dependent upon the quality (accuracy) of the sample – is it representative of the group you want to survey? “Normally distributed, proportionate to size”, was a phrase used in every quantitative study back when door to door was the norm. The sample would be “reweighted” to the exact proportions of the various demographic groups in the population being studied.
Research is the same as computers i.e. rubbish in = rubbish out.
I have no doubt that reputable research firms adhere to strict sampling disciplines, but this does not solve the problem of “too much research”.
I had lunch recently with two friends who have nearly 80 years of market research experience between. They felt that too much research kills our natural resource – the unbiased and honest respondent. Our chat formed the genesis of this article.
Many large research companies today run panels – tens of thousands of people who, for a reward (often cash), are willing to be respondents. By having all the relevant demographic information of each panel member and by filtering via certain behavioural questions, they can supply a ready- made sample.
But just how random is this group of people? They can distort focus group discussions – a couple may attend up to two groups each a month for $80 (or more) per group. And they are also paid to participate in quantitative studies. This can give them up to even $320 a month extra cash in hand, nothing to be sneezed at.
They are paid to give an opinion and so they do, but whether this is what they really believe, or what they think they should say, no one can truly tell.
“Do you buy jam” – “yes”
“What brand did you last buy?”. The brand manager for Cottee’s jams is deeply involved in the brand, but the average shopper only gives it a cursory consideration when shopping. “Last brand? Um, Cottee’s”. It may not be, but as they are paid for an opinion, they better have one. It could have just as easily been another brand that was on special. But Cottee’s is top of mind and gets the vote.
SEVEN SOUNDS LIKE A GOOD NUMBER
And then there is the classic “How likely would you be to recommend…….?” The Nett Promoter Score. I read not long ago the new MD of a large private health insurance company was going to link all senior executives’ bonuses to Nett Promoter Scores. Poor bastards!
Mark Ritson wrote a column recently about CEOs’ messing up marketing and advertising- what a colleague refers to as “Seagull Management” – they fly in, shit over everything and fly off.
This was probably a good example. The MD had probably heard of The Nett Promoter score and thought “Bloody brilliant. I will introduce that”.
Where once a survey was something rare and genuinely regarded as being worthwhile, people are asked to rate something nearly every day. And hence we have “Seven sounds like a good number”. Nett Promoter Scores may have been worthwhile years ago before the natural resource of “virgin” or at least “rarely used respondents” had been used up.
Now, people rattle off numbers between five and seven, unless something extraordinarily good or bad had happened to them.
A/B testing is another hoary chestnut. Not the place to detail an in-depth dissertation on the problems with this technique. It has become so commonly used that I would prefer to take the opinion of an experienced and intuitive creative person to finalise a headline or ad concept, over the answer given by some spreadsheet A/B testing technique.
Survey Monkey has allowed smaller businesses to conduct their own research. Sometimes this does throw up useful information, but often it is conducted by people with no experience in sampling and interviewing. The analogy is of the person who acts as their own legal counsel has a fool for a client.
We know that thanks to Google, Social Media and almost no end of information sources, consumes are far better informed than ever.
I have seen focus groups in which the respondents were acting more as creative directors – “It would have been more impactful if you had used morphing”.
Testing communications against their objectives is becoming more and more testing.
Data seems to have usurped research as a source of insights. In many companies, the market research and data insights are separate departments. Where once we referred to data as Primary Research – all available information (data) was analysed prior to any market research being developed – what do we know, or think we know, what gaps are there is our knowledge etc. Market research and data were all part of marketing intelligence. There was no demarcation between the two, they worked together seamlessly.
IS THERE AN ANSWER?
There have been many fantastic new research techniques and programmes developed over the past 40 years. That is a given. But it does not address the single biggest problem facing any form of market research. As with data, quality should always take precedence over quantity.
No matter how good the technique, if the sample is a dud, then the results will turn out the same. Nielsen introduced, a number of years ago, a qualitative research technique they called “Delta Qual”. Derived from forensic psychology, its aim was to determine what people actually did when grocery shopping, not what they thought they did.
Not a simple or cheap technique, it required two moderators. Though this approach is more likely than many techniques to find the truth, it is still heavily dependent upon the quality of the respondent.
“Killing the goose that laid the golden egg”, was how one researcher described the overuse of research.
This is a topic in which it is far easier to identify the problem than it is to provide an answer. Cliched as it is, the answer is “each case is different”.
Prior to any research study, be aware of the fact that many of the respondents are jagged and give an answer off the top of their head. If you have received feed back from say 5,000 people and 4,000 of them are just going through the motions, then the results will be rubbish.
How to identify and access the respondents who are genuinely interested in providing answers? Of course, this then raises the problem of statistical validity. Make sure you know the demographic and relevant behavioural breakdown of your target group and then reweight the results accordingly.
With qualitative research, speak to the people who are either very happy, or very pissed off.
Have you ever been to a restaurant where the waiter or waitress continuously interrupts to ask, “How is everything”? Maybe we need to learn from this. Stop asking questions after every customer contact and learn to identify those who really do have something to say.
Latest News
Scroll Media Recruits Costa Panagos From Twitch
Digital publisher network Scroll Media has employed ex-Amazon Twitch gaming executive Costa Panagos (lead image) to strengthen its Scroll Gaming capability. Panagos is highly experienced in game advertising and media sales professional with two years in account management at Twitch and three years in media sales at News Corp, so will bring his extensive media agency […]
Year13, Microsoft & KPMG Australia Launch AI Course For Gen Zs
School-leaver service Year13 has launched a new Artificial Intelligence course to upskill young people in AI, made in collaboration with Microsoft and KPMG, at an event to rally industry support behind youth AI fluency. The free online course AI Amplified which is part of Year13’s Academy aims to provide all young people with the opportunity […]
General Motors Snares Heath Walker From Scania
General Motors has announced the appointment of Heath Walker (lead image) as the new marketing director for GM Australia and New Zealand, effective April 23, 2024. Walker brings a wealth of marketing and communications experience gained from working across various industries, including IT, sporting, and auto – notably Tesla, Nissan and most recently, Scania in […]
VML Launches New “Envoyage” Brand For Flight Centre
VML unveils new brand for travel operator Flight Centre. Alas, no sign of those paid actors pretending to be pilots.
Subaru Places Media Account Up For Review
Subaru puts media up for review, as adland journos get set for mandatory "agency drives off with..." headline.
TV Ratings (17/04/2024): Contestants Faced With Harsh Realities As Alone Australia Heats Up (Or Cools Down)
Alone still doing the business for SBS. Overly long train journeys not doing the business, but they persist anyway.
Ben Fordham Loses Number One Spot As Ray Hadley Celebrates 156th Ratings Win
The radio numbers are in! Discover who's off for a boozy lunch today & who's waiting for the dreaded HR death knock.
Gourmet Ice Cream Brand Connoisseur Launches New “Thrill Your Senses” Iteration, Via SICKDOGWOLFMAN
Rattling the old "truth in advertising" adage comes this ice-cream spot full of noticeably thin people.
Paramount’s Global Sales Boss: ‘Australia’s Converged Model Is A Blueprint For How I’d Like All Of Our Markets To Be’
Paramount's global sales boss gives local sales ops the thumbs up. Didn't weigh-in on the Lisa Wilkinson debacle.
TikTok Starts Testing Its Instagram Rival In Australia
In exciting news for piano playing cats & brattish pranks in shopping centres, TikTok unveils its Insta rival plans.
Man Wrongly Named By Seven As Bondi Killer Hires Lawyers
Struggling to save for a house deposit? Why not get wrongly identified by Sunrise!
Smartsheet Appoints Indie Agency Sandbox Media To Its Media Account
Can't stand your colleagues? Like to dob them in when they miss a deadline? These work management platforms are ideal.
Boss Not Letting You Come To Cannes In Cairns? Use This Business Case To Convince Them!
Stingy boss won't spring for a ticket to Cairns? Add this to your persuasive argument repertoire. Or grovel.
Alt/shift/ Brisbane Builds Portfolio With Ausbuild Creative, PR, Content & Social Account Win
The Brisbane comms/PR agency lands constructor Ausbuild. Also hoping for a discount on its new glass conservatory.
Young Guns Versus The Old Guard: Who Adds More Value to Our Industry?
Cannes In Cairns poking this hornet's nest in a lively debate. Just so long as the oldies can get up the stairs.
70% Of Aussies Don’t Have Green Power Plans ENGIE Says In Major Brand Campaign Via HERO
Are you the notorious "light leaver on-er" in your flatshare? Quell any infighting with this green energy news.
PrettyGood Launches Offering Brand & Media Solutions For Australasian SMEs
B&T applauds the charitable nature of this new agency. Although we'd hate to see it impact any Chrissie present sends.
A Blunt End: Dolphins Medicinal Cannabis Sponsorship At Risk
Yes, it's another NRL drug story. Yet, thankfully it doesn't involve coke in Kuta during the off-season.
Slew Of New Creative Hires At Leo Burnett Australia
Ahhh, all black! The outfit of choice for agency creatives, David Jones staff and everyone in Melbourne.
Under Armour Unveils Local “Live in UA” Campaign
American apparel brand set for yet another tilt at the Aussie market, as Nike declares "we'll see about that".
Pepsi Launches New Look, Refreshing Classic Fashion Staples Via Special PR
Are you always the bridesmaid, never the bride, as the old saying goes? How do you think Pepsi feels?
Pure Blonde Returns To A Place Purer Than Yours In New Campaign Via The Monkeys
B&T's always been a huge fan of the 'drink yourself thinner' diet plan. So big thanks to Pure blonde, vodka & tequila.
Spotlight On Sponsors: Carsales Strikes With Another Storm-Based Team
Think our hacks are living their sports journalist dreams vicariously through B&T? Confirm it with this reporting.
Commission Factory & AGL Energy Ignite Strategic Partnership To Empower Australian Energy Consumers
Concerned by global warming yet ran the fans non-stop over summer? Add to the guilt with this read.
Allied Nabs Nicola Gold From Pico International Dubai
Seasoned marketer relocates from Dubai to Sydney. Says the shopping's not as good but at least you can get a drink.
Nine’s Push For Gold, Silver & Bronze Ad Opportunities Could Break New Ground In Broadcasting
Nine insists that AI won't dominate its Olympics coverage. And not merely because Bruce McAvaney doesn't understand it.
TV Ratings (16/04/2024): Toxic City Girls Too Much For Farmer Bert
Fans calling out toxic behaviour on Farmer Wants A Wife, seemingly unaware that's the very premise of any dating show.
Kaimera Wins Afterpay ANZ Media Account
Have couriers even started to mention you're overdoing the online shopping? Probably best to avoid this Afterpay news.
Industry Reacts: Stuff Group Swoops In To Save 6pm News After Newshub Closure
As you'll read here, New Zealand's media's in a right state of flux. And nor have they forgotten 1981's underarm.
a2 Milk Launches “Only a2 Will Do’ Brand Platform In First Work From BMF
A2 milk drinkers tend not to bother everyone with their gut health issues & it's a lesson for vegans & the gluten-free.
Can You Recite It In Under 4 Seconds? Maccas Relaunches Iconic Competition With Modern Twist
Do you know what would make a fantastic ad? Drunk people at 3am trying to recite the ”two all-beef patties"
‘Liar, Liar’: Lehrmann Exposed As High Profile Defamation Case Comes To A Close
Having exhausted all avenues in his defamation case, is there only one option left for Bruce - reality television?
MECCA & Fendi Among Premium Advertisers For oOh!media Woollahra OOH Network Launch
If you've not been to the snooty suburb of Woollahra, the locals would like to thank you again for staying well away.
Yahoo Academy Announces Class Of 2024
This academy is strictly for "emerging leaders". Courses don't include 'sleeping at your desk' or 'bunking off early'.
Top 35 Media Leaders Announced For 2024 TikTok Agency Accelerator Program
Despite the ban threats, TikTok keeps marching on. Cementing its reputation as the Julian Assange of the tech.
AiMCO Announces First-Ever Diversity Webinar To Explore Inclusion In Influencer Marketing
Think influencer marketing's a little on the white side? This free webinar's for you. Sadly, it's not given TV ideas.