Although the term "big data" sounds vaguely sinister--like a relative of Big Brother or Big Government--it's an unfair rap. At least in the case of state governments, it is being used to increase public safety, uncover fraud, save money, create efficiencies, and improve health and human services, among other things.
Big data is exactly what it implies: collections of data so vast that the time and effort required to manage and sort through them make traditional approaches of analysis infeasible. Yet, within these big data collections lie valuable patterns and useful information now being mined with the advent of predictive analytics software.
By cross-referencing information from a variety of sources, predictive analytics can uncover previously obscure connections between events and people and reveal anomalies and inconsistencies worth investigating. Overlaying data about health care, transportation or crime, for example, with geographical information can identify exactly where the greatest needs or problems exist, helping policymakers allocate resources most effectively.
For several years, businesses have been collecting enormous amounts of data about consumers and their lifestyles from mobile and web technologies, which they use to track customers' habits and tailor marketing materials. "Your smartphone is not just a phone. It's a tracking device. ... And if I can tap into that information, I can predict and persuade your behavior by targeting it with the right ads," says Hewlett Packard's Chris Surdak, author of "Data Crush: How the Information Tidal Wave is Driving New Business Opportunities."
Now state governments--generators of large amounts data themselves--have begun tapping into the wealth of possibilities big data offers to transform operations, such as improving public safety and health care, unearthing fraud and much more.
Of all the data now captured, experts believe about a third of it could be useful, yet only 0.5 percent of it is currently being mined. Richard Leadbeater, global solutions manager at Esri, a firm that specializes in geographic information system technology, says that big data has such significant economic potential that it should be viewed as a natural resource.
A recent report by the National Association of State Chief Information Officers describes state governments as "enormous data generation engines." State officials who treat this data as an asset, analyzing it to discover new patterns, correlations and insights, the report says, can "gain a competitive advantage." Here are some ways states are doing just that in various areas.
Big data is helping uncover tax fraud which cost states billions of dollars annually. Connecticut, Georgia, Indiana, Louisiana, New York, South Carolina and Washington, among other states, have contracted with big data companies to use data analytics technology to detect tax schemes and reduce the number of fraudulent refunds states pay out each tax season. Using identity-based filters, the technology screens the information contained in each tax refund request...