2013: Why you are not likely to come across many Big Data success stories


Pick any technology forecast for 2013 and chances are very high that Big Data will find a place in that. And there will be a discussion on the big opportunity provided by Big Data.

But, let me give you 2 reasons why you will not come across too many Big Data success stories …

Reason 1: Competitive Advantage

Key proposition of Big Data technologies is the ability to analyze large data and provide you with insight which can then be translated to (1) more sales, (2) lesser cost or (3) better service.

The assumption is that such insight would not be available to you using traditional techniques.

Without getting into the technique or the feasibility of how such insight might be obtained, let us just assume that you have got such an insight.

What is the chance that you would share it with the world?

I would say that it would be close to zero.

The insight which you would have obtained will be a clear competitive advantage … would you not like to keep it a top secret?

Reason 2: Fooled by Randomness

Vincent Granville, in a post, has explained this beautifully. Here is what he says:

…when you search for patterns in very, very large data sets … you are bound to identify coincidences that have no predictive power – even worse, the strongest patterns might be entirely caused by chance…

I do not think I could have said it better! It is almost like looking for a needle in a haystack!

Big data

Udayan Banerjee on Google+

Related Articles

About these ads
Comments
9 Responses to “2013: Why you are not likely to come across many Big Data success stories”
  1. Tina Jin says:

    Hi Udayan,

    I just read your post, and I want to thank for providing insight into big data. I don’t know much about the subject, but I know that big data is becoming increasingly important for businesses. Good job!

    At the moment, I’m looking for bloggers and contributors for a storage and big data website. Would you perhaps be interested in contributing your past and future blog articles? We want this website to be a thriving community of experts generating conversations on big data, cloud computing and storage virtualization.

    It’s free to join, and only the title and the first few sentences of your blog entries will be published on the website. We want readers to engage with your content and be directed to your blog for the full article. This way, you’ll get traffic! :)

    If you’re interested or have any questions, please send me an email at tinajin [at] atomicreach.com with “Tech” in the subject line. I’ll be glad to answer any questions and get you started on being an expert contributor!

    Sincerely,
    Tina Jin

  2. Sundeep says:

    Bumped into this online post. Wasn’t looking. Just serendipity? Pattern recoginition? Like when you buy a red car suddenly you seem to notice that everyone else has a red car too? Ditto for a black car or a car of any other color :-)

    Anyway, back on-topic, here is the online post.

    http://slashdot.org/topic/bi/library-of-congress-offers-update-on-huge-twitter-archive-project/

    • Udayan Banerjee says:

      I call this a pink cow syndrome! That is if you see a pink cow in the morning and one in the afternoon then you will definitely see one in the evening!

  3. SUNDEEP says:

    Nice thought!
    Allow me to cross-pollinate ideas from a different domain :-)

    Reason#2 may not just be a Big Data challenge per-se. After all, hasn’t the field of Artificial Intelligence struggled with “Pattern Recognition” problem statement long enough for us to find parallels in their failures, and maybe learn from their successes?

    In fact this reminded me of a story about military intelligence trying to teach a computer how to identify enemy mobile missile launcher deployments from satellite surveillance photos. For a short while it felt like the project was a success until they realized that what the computer had actually learned to identify in each satellite photograph was the onset of evening or the break of dawn. You see during the “teaching” phase all photographs of mobile missile sites were taken at dawn while the pictures of the same site without the missiles deployed were taken in the evening. This story was in a book I read as a teenager and was titled “The Book Of Heroic Failures” that was borrowed from a friend. Unfortunately this was before the Kindles and ePubs and I haven’t been able to find a link to it although I have found some similar ones on Amazon.

Trackbacks
Check out what others are saying...
  1. [...] Why you are not likely to come across many Big Data success stories [...]

  2. [...] 2013: Why You Are Not Likely To Come Across Many Big Data Success Stories [...]



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

Join 963 other followers

%d bloggers like this: