Big Data

by Phil Buckley on November 18, 2010

in Social Media

Shoot, running late out of the keynote address means I get to one of the sessions I was excited about late!

Rod Smith taking about harnessing the power of big data.

When I walk in Rod is talking about CoreMetrics and tracking iPad data out of Seton Hall University. Within 7 weeks of it’s launch, the iPad was outpacing retail type purchases on the iPhone.

If you’re a retailer using Flash on your page to drive conversions, you may want to rethink that.

Next up is Joe Gregorio from Google who says he’s going to be coming at this subject from a slightly more technical angle.

Here’s his 5 rules of big data:

  1. Save everything
  2. Use the simplest model first
  3. Get better by adding more data
  4. Do the math
  5. Be lazy

What is a large data set? Data that spans more than 1 computer, processing the data requires more than 1 computer, traditional methods fail.

Examples of big data = The Web, or n-grams, meterology, genomics, Flickr.

For example, Flick has about 4 billion photos x 1mb per photo = 4 PB of data <– that’s a lot

So what can you do with all that data? “Did you mean?” on Google. Since they know every search everyone has ever done on Google, they know a lot about what you really mean.

More data will always give you a more accurate conclusion.

Before you start dealing with big data, do the math. Just because something is on a hard drive doesn’t mean it’s easy to access or easy to manipulate.

Google BigQuery

Google Prediction API

Google Storage

Find out if someone else has already done the analysis that you are about to do.

Wow, that’s it? These guys both tore through their presentations.

{ 1 comment… read it below or add one }

Rab December 8, 2010 at 12:34 pm

Hey Phil, I live-blogged the Big Data session and have the cool stuff from Rod Smith. Check it out: http://mobthink.blogspot.com/2010/11/bigdata-talk-at-internet-summit.html

Reply

Leave a Comment

Want to add a picture to your comments here on Media Two Point {OH}? Upload your picture at Gravatar to make it happen.

Your comment may be nofollow free.

{ 1 trackback }

Previous post:

Next post: