We could predict things like potential disease outbreaks to the outcome of elections and the direction of the financial markets. We could be moving closer to the sci fi movie Minority Report where we could predict crime before it even occurs.
If all that data could be dropped into a black box and analysed using past data and real time data what could we learn and predict? In a discussion with BBC, Eyal Gever a digital visionary describes a crystal ball app that could change the world.
Potential ProblemsOne of the problems we have is the processing power needed for all that data, but that’s exponentially excelerating. The next problem is being able to get to all that data and the privacy concerns surrounding it. And what of all the power that someone would have that holds that data and can accurately predict the futue or if that data could be manipulated for the wrong reasons.
A few companies are already leveraging our data to better the world and predict future events. Lets look at some examples of the power that can be found within all that data.
Data Behind Texting Could Save More Lives Then PenicillinNancy of DoSomething.org describes the data behind texting and the lives that could be saved, policies that could be made, the effects of a hatefull speech or bad legislation and using trends and anlysis in that data to help better the lives of our children. She points out one example about noticing bad things occurring around 3pm in which a school could then change policies to fix this trend. She even goes on to say that Texting could save more lives than penicillin.
Disease and Health PredictionA website called Sickweather uses social media to correctly predict and map disease outbreaks and in fact predicted six weeks earlier then the cdc in October 2012 about the early outbreak of this years flu season. Using Social media to predict disease outbreaks
Google Flu Trends uses flu related search terms as indicators of flu activity. There work is based on research described in an article in the Nature scientific journal titled ‘Detecting influenza epidemics using search engine query data’.
Another attempt at uncovering health trends and public health concerns using twitter was a study run by two researchers from John Hopkins. They uncovered interesting patterns that showed medications and home treatments being used by people to treat their symptoms. One helpful trend they noticed was that people were trying to treat the flu with antibiotics, raising a public health concern that should be addressed.
Financial Market PredictionTopsy Labs, a company that has access to twitters historical data and has a real time search engine for twitter mining finds that there is a statistically significant correlation between Twitter sentiment and Market prices.
Predicting Election OutcomesIn an article about Big Data in the 2012 election Forbes points out that were not quite there yet with using big data to accuratly predict ellection outcomes. One of the reasons they point out is that there are paralels to 1948 when Dewey was innacturatly predicted to win over Truman based on phone polling. The majority of people did not own phones and those that did tended to skew toward a particular demographic. Similar to today we dont have 80 year old Grandmas tweeting about there party favorites.
But Forbes concludes that it does add some useful information even though we still need to figure out how to process all that data effectively and filter out the signal from the noise.