Getting started with massive data

Posted by Max on Stack Overflow See other posts from Stack Overflow or by Max
Published on 2010-05-30T04:04:28Z Indexed on 2010/05/30 4:12 UTC
Read the original article Hit count: 314

I'm a math guy and occasionally do some statistics/machine learning analysis consulting projects on the side. The data I have access to are usually on the smaller side, at most a couple hundred of megabytes (and almost always far less), but I want to learn more about handling and analyzing data on the gigabyte/terabyte scale. What do I need to know and what are some good resources to learn from?

  1. Hadoop/MapReduce is one obvious start.
  2. Is there a particular programming language I should pick up? (I primarily work now in Python, Ruby, R, and occasionally Java, but it seems like C and Clojure are often used for large-scale data analysis?)
  3. I'm not really familiar with the whole NoSQL movement, except that it's associated with big data. What's a good place to learn about it, and is there a particular implementation (Cassandra, CouchDB, etc.) I should get familiar with?
  4. Where can I learn about applying machine learning algorithms to huge amounts of data? My math background is mostly on the theory side, definitely not on the numerical or approximation side, and I'm guessing most of the standard ML algorithms don't really scale.
  5. Any other suggestions on things to learn would be great!

© Stack Overflow or respective owner

Related posts about data

Related posts about nosql