Who Will Win In Emerging Big-Data World?

It seems like everyone in our business today is talking about Big Data, at conferences and in articles in the trades, including this one. Big data even pops up in every sort of job description for new roles in media, marketing and advertising. And it is even invading earnings calls for holding companies.

Why has Big Data become so important in our industry? The explosion and invasion of connected computing devices in all aspects of consumer and business fields over the past 10 years -- from home computers to smartphones, from store checkout scanners to those laser-like theater or Amtrak ticket readers -- has created a volume of consumer and business data exhaust that exponentially overwhelms any notions of data we've dealt with before.

Our potential to capture, synthesize and act on data -- even from disparate, disconnected sources -- is now virtually limitless. We can no longer compare the size of databases to multiples of the amount of information in the Library of Congress. That reference point is too small to even matter anymore.

Meanwhile, the cost and complexity of capturing, storing and synthesizing data has been significantly reduced. Six years ago it cost $10-$15 million and took nine months to build and configure a robust 30-terabyte data-mart with systems from Oracle and IBM. Today, you can build a robust 300-terabyte data-mart with $500,000 of off-the-rack Dell server storage arrays running free, open-source software (Hadoop, MySQL, PostreSQL) -- or you can use a credit card for cloud-based data warehouses from companies like MongoDB.

Big Data elsewhere means rapid progress in the accuracy of weather forecasting or tracking the spread of communicable diseases. But for marketers, Big Data means that you can analyze a marketing activity like the delivery of banner ad impressions or TV spots and determine relatable purchases made by tens of millions of exposed and non-exposed consumers with a level of precision that dramatically outstrips any marketing metric we've ever seen before. You can begin to see how every piece of marketing material or advertising affects upper-funnel activity like consideration, along with actual purchase and customer loyalty.

Big Data is not just revolutionizing data management and data mining -- it's also democratizing it. If your data infrastructure investments are already about six years old, you are most likely operating at a 300-1 cost disadvantage against competitors with modern Big Data "systems." But you no longer need to be Walmart to have the world's biggest and best retail transaction database.

How will this trend affect those of us working in media and marketing? At the very least, it means that more and more of what we do every day will need to be empirically justified, either before, during, or after we recommend or take an action -- or, most probably, at all three points. If you are not armed with data, you can be sure that someone else in your organization will be, particularly those who want either your job, your budget or your industry standing.

Marketing data analysts will become both more valuable and less in control. First, those who are good with making data relevant for business decisions will do well. But those who have relied on being the only data-enabled executive in the room will now find lots of competition. Big Data democratization means more competition for those who used to rule the data analysis roost in organizations.

Who within marketing and media companies will win in the emerging Big Data world? I believe that the spoils will go to those who can move past basic data analysis and convert it into predictable business outcomes. Big Data means that we're no longer limited to looking in the rear-view mirror. The future will reward those who can convert data into foresight and take bold, market-changing actions.