Blaggers guide to Big data – How big is the elephant in the room?

David Pountney, DT Melbourne (pictured)

The term ‘Big Data’ hasn’t sat well with me since its recent introduction to boardrooms the world over.

It’s not that I refute it as a concept, there’s no doubt that the explosion in size and variety of data sources can represent considerable data management challenges for large organisations, particularly in sectors such as finance and utility.

My gripe with big data is that much of the rhetoric seems to have been intended to instil a sense of fear into marketing departments, a situation not dissimilar to the CRM gold rush of the 80s/90s.

Yet it’s my view that it’s never been easier to do data driven marketing.  Let’s use the three V’s of Big Data – Volume, Variety and Velocity – to explore why…

1.       Volume

Whilst volumes of data are increasing year on year, Moore’s law has shown that the processing speed and memory capacity of computer hardware doubles every 18 months. Therefore unless you already have a customer database in the millions that is doubling year or year, the chances are your ability to store and analyse that data is increasing faster than the volume itself. Not only that, the associated costs are getting cheaper.

In addition, enhancements in software make it easier to analyse large datasets than ever before. Data analysis programs that took hours to run five years ago take minutes in 2013. Even the current version of Excel can handle over one million records; it wasn’t all that long ago that 65,536 was the glass ceiling of budding Excel based analysts.

2.       Variety

The growth in data breadth (as opposed to depth) in recent years is largely attributable to the digital age. Yet that what digital age has brought with it is an increased uniformity of data structure. Rarely do two customer databases follow the same structure and yet web analytics, social and search data follow a defined structure that can be queried using APIs that follow a universal data standard.

Therefore not only can we leverage the thinking and code of others who are trying to solve the same problem, but the data owners have often done much of the hard work to ensuring the data is clean and fit for purpose.

I can’t remember the last time I encountered an unwanted ASCII character, carriage return or comma that played havoc with my data structure and thank “,|~ for that!

3.       Velocity

For me this is most exciting ‘V’ of Big Data. The real time, live nature of today’s data has brought a whole new level of excitement to the analytical process. It’s the difference between watching the grand final live vs last years highlights on DVD.

Welcome to the real time marketing age that in many respects reduces the reliance on volume. Real-time personalisation, A/B and multivariate testing thrive on a live stream of data, learn in real-time, implement then move on. It’s speed data-ing (you heard it here first), it gets you to the desired result in the most efficient way with minimal baggage.

So if some the technical barriers are coming down, why does big data still appear to be such a confronting issue for many marketers?

Andrew Pink, OgilvyOne  

I similarly find the term ‘Big Data’ a meaningless concept in the current context of all the fears and doubts being thrown around. As Dave points out, the volume, variety and velocity of data is increasing exponentially, but with the systems, tools sets, and physical capabilities to now handle these, what’s most likely to hold us back?

I believe it’s our willingness to embrace Big Data that holds the greatest challenge.

As we realise the technical issues are no longer a barrier, it’s time to recognise that other age-old challenges are still our greatest limitation. The two biggest ones are a lack of understanding and a reluctance to change ingrained habits.

Many still struggle to fully comprehend the value of data in their organisation. Without a greater proportion of executives championing Big Data, those responsible for data management and analysis will continue to lack the investment and resources to show its true potential.

And what a potential it has.

Digital businesses are often best able to demonstrate the true value of data-led insights as they have grown up with it hardwired in. For instance, Amazon’s introduction of ‘Customers who bought also bought’ when checking out results in regular 100% increases in basket size. Let’s read that again – regular 100% increases in basket size. A simple data insight led improvement has let them regularly double the amount they sell.

Of course, once we do recognise the power of data, we face our second challenge – overhauling the way we do things. It’s certainly easier and faster to say ‘I think’ than crunch the numbers and say ‘I know’. But data makes for a very powerful ally. At a P&G meeting early in my career I witnessed a vice president announced his decision for the direction for a new product launch.  In awe I then watched the most junior employee at the boardroom table nervously clear his throat and challenge the call. The VP quickly overturned his decision. Apparently an ‘I know’ trumps even the highest ranking ‘I think’. No surprise then that P&G is one of the most successful companies in the world.

Still, taking the guessing out of the game suffers the slings and arrows of boredom and unfortunately, some feel it’s in opposition to the inspiration of creativity that drew them to the industry in the first place. Data for me however is the springboard for creativity, taking you in directions you’d most probably never imagined. Therefore, we must continually underline the benefits of data-led decision making and educate others to the value of investing in the systems and tools that let us derive the most from what data has to offer.

It is this I think that has brought the issue of Big Data to the fore, our realisation that we are still lacklustre in our ability to effectively recognise the true value of data, big or small. As some firms embrace the full potential of Big Data, others become increasingly fearful that if they don’t invest they’ll be relegated to the also-ran scrap heap.

With new systems and tools offering no excuses, consigning the complexity and cost of such operational implementations to the bin, even the minnows of the marketing world now have the opportunity to take data by the scruff of the neck and wring it for all that it’s worth.

So while I quietly loathe the term ‘Big Data’, if it helps us focus on the gold mine of data-driven insights that too often get tossed into the too-hard basket, then I’ll be its biggest champion.

Hail BIG DATA!

So if you’re a marketer staring down the fire hose of Big Data, we’d like to leave you with this thought…

There has never been a better time in the last decade to embrace data driven marketing, the data is more accessible, structured, recent and diverse than ever before but best of all we are now equipped with more tools and processing power to make sense of it all. Data analytics has never been sexier.

Ultimately though, the data and technology is just an enabler. It’s the creative use and application of data that makes the real difference.

Put another way, it’s not how big your data is, but what you do with it.

Over the next few weeks we’ll explore some of the creative applications of data across retail, finance and utility. 

David Pountney is general manager DT Melbourne and Andrew Pink is strategy director OgilvyOne Melbourne.