Are there any existing batch log file aggregation solutions?
- by Mohan Gulati
I wish to export from multiple nodes log files (in my case apache access and error logs) and aggregate that data in batch, as a scheduled  job.  I have seen multiple solutions that work with streaming data (i.e think scribe).  I would like a tool that gives me the flexibility to define the destination.  This requirement comes from the fact that I want to use HDFS as the destination.
I have not been able to find a tool that supports this in batch.  Before re-creating the wheel I wanted to ask the StackOverflow community for their input.
If a solution exists already in python that would be even better.