Real-time aggregation of files from multiple machines to one

Posted by dmitry-kay on Stack Overflow See other posts from Stack Overflow or by dmitry-kay
Published on 2010-04-24T13:01:24Z Indexed on 2010/04/24 13:03 UTC
Read the original article Hit count: 277

Filed under:
|
|
|
|

I need a tool which gets a list of machine names and file wildcards. Then it connects to all these machines (SSH) and begins to monitor changes (appendings to the end) in each file matched by wildcards. New lines in each such file are saved to the local machine to the file with the same name.

(This is a task of real-time log files collecting.)

I could use ssh + tail -f, of course, but it is not very robust: if a monitoring process dies and then restarts, some data from remote files may be lost (because tail -f does not save the position at which it is finished before). I may write this tool manually, but before - I'd like to know if such tool already exists or not.

© Stack Overflow or respective owner

Related posts about linux

Related posts about ssh