Back in the Swing of Things

This quarter I have a class called "Futuring and Innovation" that encourages us to create and write in a blog and I realize now that it's been quite a while since I last updated this blog.  My original intent of course was to write about topics that interested me as I made my way through this Doctoral degree but as seems to happen I got busy with classes and research and have not updated as often as I would have liked.

So, in this new spirit of blogging, I'm going to discuss a video that I found entitled  “Big Data is Better Data” by Kenneth Cukeir that touches on one of the topics in my dissertation. Feel free to watch the video below.

Mr. Cukier is of course discussing the innovation of big data. The buzzword "big data" is now a little bit passé however for those been living under a rock for the last couple of years the quick summary of the topic:  big data refers to our ability to process information in ways previously impossible. recent advances in storage allow us to collect and keep more data than ever before and some recent advancements in computer algorithms, Namely the MapReduce algorithm made famous by Google in 2008 (Dean & Ghemawat, 2008), give us the ability to process larger amounts of data in smaller amounts time. Another unique factor is that this new processing ability can't handle unstructured data, Such as pictures or videos or sound files, whereas in the past data processing required some form of structure such as you would find in a database.

Mr. Cukier eloquently discusses some of the pros and cons of big data analytics (meaning the ability to turn the data into some sort of useful knowledge). He makes a compelling case for the collection and usage of the data and in that same vein I had a few thoughts on the matter myself. 

The Good: We have become used to big data being used in our everyday lives in fairly benign ways - Amazon and Neflix recommendation engines are just two examples we take for granted. However, some possibly more life changing uses than suggesting you watch “Orange is the Next Black” or recommending shoes to go with that skirt, big data analysis is being used by the medical community to help find cures for cancer and even tailor drugs and therapies to individual patients, research to develop drought-resistant strains of rice, and the public sector to evaluate air quality and where to allocate police resources.

The Bad: There is a now-legendary story of big data analysis attempting to produce influenza outbreaks based on social media interaction that went awry (just because people are talking about the flu doesn’t mean they have it). Shaw (2014) tells of a big data researcher attempting to use cell phone data in Africa to predict peoples’ movements who thought at first to have found a predictor for cholera outbreaks when what he really found was confirmation of local flooding. Stories such as this serve to remind us that correlation does not equal causation.

The Ugly: Perhaps the most famous recent example of the misuse of big data comes from stories of the NSA collecting phone records and other data for the purpose of spying on American residents. These types of stories help us to understand the fragility of privacy. When so much data from so many sources is available, easy is it to pierce the think veil of privacy?

References:

Dean, J., & Ghemawat, S. (2008). MapReduce : Simplified Data Processing on Large Clusters. Communications of the ACM, 51(1), 1–13. doi:10.1145/1327452.1327492

Shaw, J. (2014, March). Why Big Data is a Big Deal. Harvard Magazine, 116(4), 30-35. Retrieved from http://harvardmagazine.com/2014/03/why-big-data-is-a-big-deal