Blog Moved

Future posts related to technology are directly published to LinkedIn
https://www.linkedin.com/today/author/prasadchitta

Friday, May 24, 2013

Data Philosophers and data quality


After data scientists and data artists, another need is for "data philosophers”.
http://www.ocdqblog.com/home/the-need-for-data-philosophers.html made me think about the data philosophers.

So, the data scientists are focusing on the underlying technology to gather validate and process the 'big' data and the artists are using the processed 'big' data to paint and visualize the insights.

In this whole process due to its wide variety and velocity (two 'V's of big data!) are we missing on the rigor of quality of data?

Considering the 36 attributes of data quality in the 1972 paper of Kristo Ivanov - http://www8.informatik.umu.se/~kivanov/diss-avh.html and evaluating today's big data insights, I somehow feel there is a 'big' gap in the quality of 'big data'.

I see some parallels in big data processing and orbit determination. As long as the key laws governing the planetary motion are unknown, whatever is the amount of the data from observation we have, we will not be able to explain the ‘retrograde motion’ of the planets. In the same way, if we do not have a clear understanding of underlying principles of the data streams, we will not be able to explain them. That is where we need the philosophers!

Now, I think I am becoming a “Data Philosopher” already!

Friday, May 17, 2013

Data Artist - A new professional skillset?

In past few days, I have seen at least two blogs talking about "Data Artist"

1. http://www.thetibcoblog.com/2013/05/04/forget-being-a-data-scientist-and-become-a-data-artist/
2. http://www.datasciencecentral.com/profiles/blogs/the-rise-of-the-data-artist-in-business

The trend seems to go towards business centric data visualization of so called "big data".

Definition:
One who can use data as the paint and create art that can represent massive flows of data and visualize the patterns in a way business users are delivered with a lot of “information” in a single glance.


It is slightly different from the “Data Scientist” profession. Data Scientists are focused on technical process of collecting, preparing and analyzing the data for patterns where as the Data Artists specialize in visualizing the discoveries in an artistic manner!

"Scientific Artists" and "Artistic Scientists" with Data! Are we complicating the matter too much??