Map Reduce Frameworks/Infrastructure

Posted by Johannes Rudolph on Stack Overflow See other posts from Stack Overflow or by Johannes Rudolph
Published on 2010-03-15T20:35:29Z Indexed on 2010/03/15 20:39 UTC
Read the original article Hit count: 325

Map Reduce is a pattern that seems to get a lot of traction lately and I start to see it manifest in one of my projects that is focused on an event processing pipeline (iPhone Accelerometer and GPS data). I needed to built a lot of infrastructure for this project, in fact it overweighs the logic code interacting with it by 2x. Some of the components I built where EventProcessors (with in- and output plus buffering, timing etc.), multiplexers and aggregators.

This leads me to my question what the "common" required infrastrucutre for map reduce is. Since I am working with .Net a lot I can see map reduce infrastructure built into the Framework and language constructs. Functional languages support this paradigm per se. It seems every language can be used with map reduce, some have better support than others, others again are built around that concept (e.g. Go). And there are Frameworks like Apache Hadoop to support map reduce.

© Stack Overflow or respective owner

Related posts about mapreduce

Related posts about language-agnostic