Put Big Data Information to Work with Insight Enabling Infrastructures

The vastness of Big Data has become so infinite that government agencies are asking for outside help by initiating a Big Data challenge. NASA, the Department of Energy and the National Science Foundation have invited analytic experts, coders and geeks alike to join together and participate in a contest series.

The goal of the competition is to crowd source solutions to the increasing fragmented and incompatible federal data issues. The announcement was made in GigaOM’s article “NASA Tries to Free Creativity with Big Data Challenge”, along with the first challenge question for participants:

How can we make heterogeneous (dissimilar and incompatible) data sets homogeneous (uniformly accessible, compatible, able to be grouped and/or matched) so usable information can be extracted? How can information then be converted into real knowledge that can inform critical decisions and solve societal challenges?

Variances in government software platforms have caused data solutions to be elusive or inefficient:

The dream of creating spectacular algorithms, beautiful visualizations and uncovering hidden insights often only comes after untold man-hours spent cleaning and munging data (often with Hadoop) into formats that a software system can work with.

The Big Data challenge should prove quite efficient since no one is tracking man-hours, however challenges could be avoided were NASA utilizing software with an insight enabling infrastructure that puts Big Data information to work. Government security is of the utmost importance and this technology provides secure, enterprise wide access to relevant information while continuing to break down fragmented or incompatible data. Challenge completed.

Jennifer Shockley, October 11, 2012

Sponsored by ArnoldIT.com, developer of Augmentext