Attribution, Net Promoter Scores And Other Fairytales!

In his final column for 2019, B&T’s occasionally difficult columnist, Robert Strohfeldt, delves into the murky marketing world of attribution and net promoter scores. And, if you can hang in until the end, there’s even a fantastic prize up for grabs, too…

Much has been said and written about attribution. So much, that the original meaning of attribution has become foggy:

 An attribution identifies a source or cause of something — Attribution often involves identifying the author or source of written material or a work of art. (Somewhat different to its use in advertising: first or last click?)

ROI has always been an issue, with one of the first quotations on return on advertising by John Wanamaker (1838 to 1922) stating:

 “Half the money I spend on advertising is wasted; the trouble is I don’t know which half”.

This is far from any accurate assessment on the per cent return – he simply states half his advertising “works”, but what does that mean?

I recently read an article the other day on attribution – “From Mobile Based Attribution to People-Based Innovation”.

The major issue/problem – the attribution was a measurement of “devices”, not people. One person may be counted several times due to multi-screen usage. And different teams, in those silos (I can remember when a silo was something in which grain was stored) are different people looking at different KPIs

A nifty little diagram was provided to help “illustrate” the point.

People based Attribution supposedly addresses the problem of one customer having “dozens” of fragmented anonymous profiles i.e.

There is one glaring problem with these diagrams and deductions. All communications are online. In real life, a person may have seen a website, spoken to a friend who recommended the product and service and then saw a TV ad. (Or any “traditional” advertising). This is just one alternative, there are so many more.

The latest figure I have been able to obtain has online retail purchases making up nine per cent of sales in Australia. So, these attribution studies account for less than 10 per cent of total retail purchases. (Or less as online purchases are often influenced by factors that are not online).

The use of online for research/price comparison/location etc. would occur in most purchases. But confining the attribution to online communications and purchases, excludes around 90% of purchases. (Or, more bluntly, totally useless).

Also consider that many purely online businesses advertise on TV. It is a fairly general, but accurate assumption, that once an online business becomes “successful”, they advertise on TV.

TV is mentioned, sort or i.e.

“Additionally, some non-digital channels, such as in traditional TV advertising, simply do not offer a deterministic identifier in the first place. When no deterministic identifier is available, probabilistic matching comes in as a fallback method. As the name suggests, device matching is performed by building statistical models using a combination of selected parameters and machine learning.”

So, although 90 per cent plus of purchases are done in-store, rendering the digital attribution model irrelevant, TV still the single most effective advertising medium, is treated like an “addendum” to the study.

The paper goes into detail on mobile web ad spending versus laptop/desktop web ad spending, the benefits of apps over websites (app vs. the web), the differences between 1^st party and 3^rd party cookies, web attribution flow, last touch and multi touch, without any reference to the all-important “art” of advertising i.e. the messages/creative.

The influence of the creative is not addressed. Yet I am sure A/B testing (which has huge flaws) is used at some stage by most using Attribution. So, the effectiveness of different creative executions is acknowledged, but it plays no role in Attribution. (No Attribution modelling I have seen has factored in the influence of the creative execution.) Then again, to try and do so would invoke huge error. There are not just so many variables at play, and variables for individuals that can change even daily.

Many marketers are obsessed with data. Which relative to the length of time marketing has been around, is relatively recent, particularly in comparison to social sciences such as Psychology and Sociology. (Pure sciences such as physics, chemistry, geology etc. have been built on data).

Nothing wrong with this. Without the quantitative, a marketer is flying blind. The problems are with the application, in the words of the John Maynard Keynes:

“As a means of suggesting possibilities and probabilities rather than anything else, taken with enough grains of salt and applied with superlative common sense, [econometrics] won’t do much harm. That is how the method ought to be used.”

Unfortunately, in marketing there are way too many practitioners who know just enough about mathematics and statistics (which is just one area of applied maths), to be dangerous. Too often the quantitative analysis is applied without “superlative common sense”.

I can recall when market research surveying switched from mostly door to door, face to face, to telephone surveys. Much faster and cheaper. At a Research Society conference in the early 90s, a major topic was the erosion of the most valuable of all resources – respondents. Telemarketers came out of the woodwork and phoned people under the guise of market research. This was utter bullshit. They were fishing for potential customers, using amateurish surveys to qualify likely candidates.

“Hi, my name is Ralph the Mouth and I am doing a survey on house cladding.” A number of questions were asked and if the person seemed to be a possible candidate, they had sales- person often knocking at the front door, or at least phoning.

The impact, over a relatively short time period, was the householder not taking any calls from telemarketers or genuine research field operations.

By 2019, quantitative research was perceived by so many businesses as easy to do, “a piece of piss”. Just about every company uses Survey Monkey (such an apt name), makes up their own questionnaire or has a friend, relative etc design a questionnaire and 99.9 per cent ask “How likely would you be to refer ……”

The valuable source of respondents has almost run dry. Yes, we have panels, but that brings in a bias i.e. people who will sign up to be paid to provide opinions

If the “respondent” cannot honestly answer a question, there is a very real tendency to provide an answer, as they are being paid for an opinion so better have one.

Due to the inundation of market research, which was once the realm of only large businesses, market research’s most valuable resource i.e. unbiased, genuine respondents, has all but been exhausted.

Now, the results of many quantitative research studies are based on highly suspect samples. (And there are all the DYI operators who not only conduct rubbish studies, but also help consume the valuable resource of genuine respondents).

Out of quantitative research came the bastard child, the Nett Promoter Score.

NPSs seem to have taken on a life of their own, reaching the highest levels of businesses and organisations. The CEO of Westpac, who resigned after the huge scandal of money being sent for the purpose of “kiddie fiddlers”, was quoted as stating it was not a major issue and to get back to “improving our NPS.” And this madness is not a one off – it is becoming common. Recently the CEO of one of the largest private insurers said his executive teams bonuses would be linked directly to NTP scores.

No matter what service I use, or product I purchase, I receive a questionnaire, and all have a NPS question.

I Googled NPS and the first item was:

“Make it Easy to Motivate your Entire Organisation to Become More Customer Focused. NPS Anytime, Anywhere via SMS, Email, Website and Many More. Request Now!”

Excuse the language, but my first reaction was “Fuck me!” – Anytime, anywhere via SMS, Email, Website. Yes, your business and everyone else’s is going to pester people and ask how likely they will be to recommend them.

This is not to say that quantitative measures are useless. They can and do provide valuable information. But as Keynes said a long time ago, they need to be taken with “a grain of salt” and used in conjunction with common sense. (Experience).

Though not statistically valid, getting out into the real world observing and speaking to customers and front- line staff can provide the balance to quantitative measures.

Finally, a little “puzzle” for those obsessed with the quantitative Following is a simple equation which shows 2 = 1. This is obviously not the case, but the algebra is 100 per cent correct. So where is the fault?

I put this problem forward a couple of years ago, offering a bottle of Bollinger to the first person to identify what is wrong, No one could pick it. Same offer is on the table. What is the error, what is wrong with the above equation? If you claim to be literate in mathematics and statistics, then the answer to the above should be very simple. A bottle of Bollinger would go down well at Christmas.

A Merry Christmas and a happy and safe New Year to all.

Connect with:

LinkedIn

If from the start X = Y then X - Y = 0 which means you can't divide by (X - Y) and can't arrive at X + Y = Y.

byAdélaïde Durandet

Please login with linkedin to comment

Latest comments

1. If from the start X = Y then X – Y = 0 which means you can’t divide by (X – Y) and can’t arrive at X + Y = Y.

Robert Strohfeldt