In big data we hope and distrust.

• Author: Hall, Robert
• Position: MARKETING SOLUTIONS

"In God we trust. All others must bring data."

--W. Edwards Deming, statistician, quality guru

BIG DATA HELPED RE-ELECT A PRESIDENT, find Osama bin Laden, and contribute to the meltdown of our financial system. We are in the midst of a data revolution in which social media introduces new terms like Arab Spring, Facebook Depression and Twitter anxiety that reflect a new reality: Big data is changing the social and relationship fabric of our culture.

We spend hours installing and learning how to use the latest versions of our ever expanding technology while enduring a never ending battle to protect our information. Then we labor while developing practices to rid ourselves of technology--rules for turning devices off during meetings or movies, legislation to outlaw texting while driving, restrictions in class rooms to prevent cheating, and scheduling meals or family time where devices are turned off. Information and technology: We love it, hate it, can't live with it, can't live without it, use it voraciously and distrust it immensely. I am schizophrenic and so am I.

Big data is not only big but growing rapidly. According to IBM we create 2.5 quintillion bytes a day. Ninety percent of the data in the world has been created in the last two years. Vast new computing capacity can analyze Web-browsing trails that track our every click, sensor signals from every conceivable device, GPS tracking and social network traffic. It is now possible to measure and monitor people and machines to an astonishing degree. How exciting, how promising. And how scary.

What started this?

This is not our first data rodeo. The early stages of the customer relationship management movement were filled with hope and with hype. Large data warehouses were going to provide the kind of information that would make companies masters of customer relationships. There were just two problems. First, getting the data out of the warehouse wasn't nearly as hard as getting it into the person or device interacting with the customers in a way that added value, trust and expanded relationships. We seem to always underestimate the speed of technology and overestimate the speed at which we can absorb it and socialize around it.

Second, unfortunately the customers didn't get the memo and mostly decided in their own rich wisdom they did not need or want "masters." In fact as providers became masters of knowing all the details about our lives, consumers became more concerned. So while many...