Big data does not come neatly wrapped up and organized. When companies seek big data, they find bare and raw data that has little to no value on its own. The key is the strength in numbers. While there are many technologies that can help straighten out big data, there are not quite as many data scientists rearing to go according to the article, "Getting Big Data Organized is an Iterative Process."
Or perhaps the problem lies in communication as the article suggests. There is a disparity between what quanitatifiable measurements executives ask for and what analysts seek in data mining. Data scientists, or people who understand both the business and the technological side of data need to appear in the job market.
The article states:
[W]e need someone who understands both the business drivers and the details of the structure and semantics of big data sets: a data scientist. The work of a data scientist is iterative. There will be days spent generating descriptive statistics about data sets, determining how to join multiple data sets, and performing other data exploration tasks. Sometimes these exercises don’t lead anywhere and its back to the drawing board.
Data management technologies have made a huge dent in the problems associated with gleaning insights from big data, but the article is correct to point out the human workforce of data scientists that we need.
Megan Feil, August 7, 2012