The Real Challenge With Big Data

By Doug Stephens

It’s hard (read impossible) to go a day without hearing that big data is poised to revolutionize business.  The speed, volume and variety of data now available at our fingertips, holds incredible promise for companies in all categories. For the average marketer, big data must seem like a giant catapult being rolled into a battle that’s currently being waged with sticks and stones.   It is indeed a new war machine and by most accounts, including my own, it will be a formidable force. Of that there seems little question.

Within this new data landscape, companies are scrambling to comprehend the technical aspects of big data  – what it is, how it works, what it costs and the true capabilities it might bring.  These are early days on the big data hype curve, and it’s particularly challenging to separate fact from fantasy in terms of what the average company can actually expect to do with big data once they harness it.  And the truth is that grasping this new and awe-inspiring technology is not without some challenges.

But as is often the case, understanding the technology is only a small portion of the full test that companies embarking on big data will face.  Far less obvious but infinitely more critical are the human issues prompted by the use of big data analytics. These, to my mind, are the true challenges that companies should be addressing early, honestly and sensitively.

I see these challenges affecting three key constituents:

Leaders

Many of today’s corporate leaders were developed to believe that “strong leadership” means gathering all relevant facts and data, performing thorough and astute analysis and then making the most sage and correct decision possible. Measurement of the decision may come weeks or months later.  Then and only then might there be any course correction.  Decision makers have been expected to be firm and unwavering, because in today’s corporate world, equivocation and plasticity of thinking have often been seen as signs of weakness.

Big data leadership however, requires a completely different decision-making style.  It involves making nimble and quick decisions based on limited information and best guesses, and then implementing through a series of tests, course corrections and pivots based on an immediate, robust and constant flow of data. It’s like navigating a ship using a GPS unit versus an antique sextant and the stars. In essence, the big data leader won’t set out to be “right first time” but to be “less wrong over time.”

The best retail leaders therefore, will no longer be those that aspire to be the most intuitively correct but instead those that are the most innately curious, and capable of asking the best questions of their data.  In other words, instead of having the right the answers, it will be the leaders smart enough to develop the right queries that will drive results.

This requires a fundamental re-learning of what it means to be a leader and my guess is that many organizations (and their leaders) will fail to make the transition.

Employees

Cameras can record employee interactions with customers.  RFID tags can track worker’s bodies through the store and measure their attentiveness to consumers and pace of work.  Social filters can sift through millions of conversations to find mentions of staff names or their store locations for commendation or condemnation. So, companies venturing into the use of big data will have to ask themselves what sort of relationship they want to have with their employees.

Will big data be used as a device purely for routing out the weak or as a valuable means of developing the strong? Does the collection of data work to help the employee excel or does it work for the company only? And are employees entitled to their own performance (big) data or is it merely stored and deployed at them?

This new and microscopically invasive tool can be either remarkably positive or devastatingly destructive depending on how it’s positioned, shared and executed with the workforce.

Customers

Take it from someone who has scoured the earth for good retail big data use cases, there aren’t many.  In fact, the best big data case studies tend to be found in other areas, such as health, government, medicine and education. Why? Because in these situations, consumers have little choice but to share accurate and complete data.   They have no such obligation in most retail situations. Retail customers lie, share loyalty cards and coupons, move and yes, even die, all without informing retailers. Consequently, at any given time, much of the data in a retailer’s system is garbage…big garbage.

The only way for retailers to gather good data is to compel customers to willingly and even happily share it with them.  This requires two things; implicit trust on the part of the consumer and a one-to-one sharing of value between the two.  If data is, as some call it, a new currency, what does the customer get when they spend their data currency with you?  More advertising or a tangibly better shopping experience?  More junk mail or more relevant and personalized communications? It’s a critical philosophical question that your company has to align on.  Trust will represent as great a competitive advantage as a fabulous product or standout service – perhaps even greater. So that privacy policy that currently sits as an obscurely written, 5 page, legal footnote on most websites?  Well it may be time to put it in plain English and move it up to the header.

Table stakes

Three to five years from now, I suspect we’ll take big data for granted.  It will be table stakes to play in chain retail, and most companies will have honed a relatively acute understanding of big data’s technical traits.  They’ll get that piece pretty quickly.  But it’s the companies that recognized early on that big data is not a technical challenge but rather a human challenge that will be more likely to master it’s true potential.  Get the human piece wrong, however, and all the distributed computing, in-memory processing and Hadoop databases in the world won’t help you.