Global variables in hadoop.
        Posted  
        
            by Deepak Konidena
        on Stack Overflow
        
        See other posts from Stack Overflow
        
            or by Deepak Konidena
        
        
        
        Published on 2010-05-22T16:06:40Z
        Indexed on 
            2010/05/22
            16:10 UTC
        
        
        Read the original article
        Hit count: 248
        
hadoop
Hi,
My program follows a iterative map/reduce approach. And it needs to stop if certain conditions are met. Is there anyway i can set a global variable that can be distributed across all map/reduce tasks and check if the global variable reaches the condition for completion.
Something like this.
While(Condition != true){
            Configuration conf = getConf();
            Job job = new Job(conf, "Dijkstra Graph Search");
            job.setJarByClass(GraphSearch.class);
            job.setMapperClass(DijkstraMap.class);
            job.setReducerClass(DijkstraReduce.class);
            job.setOutputKeyClass(IntWritable.class);
            job.setOutputValueClass(Text.class);
}
Where condition is a global variable that is modified during/after each map/reduce execution.
© Stack Overflow or respective owner