Fast extraction of a time range from syslog logfile?
        Posted  
        
            by mike
        on Server Fault
        
        See other posts from Server Fault
        
            or by mike
        
        
        
        Published on 2010-01-11T22:50:31Z
        Indexed on 
            2010/06/15
            20:53 UTC
        
        
        Read the original article
        Hit count: 322
        
I've got a logfile in the standard syslog format. It looks like this, except with hundreds of lines per second:
Jan 11 07:48:46 blahblahblah...
Jan 11 07:49:00 blahblahblah...
Jan 11 07:50:13 blahblahblah...
Jan 11 07:51:22 blahblahblah...
Jan 11 07:58:04 blahblahblah...
It doesn't roll at exactly midnight, but it'll never have more than two days in it.
I often have to extract a timeslice from this file. I'd like to write a general-purpose script for this, that I can call like:
$ timegrep 22:30-02:00 /logs/something.log
...and have it pull out the lines from 22:30, onward across the midnight boundary, until 2am the next day.
There are a few caveats:
- I don't want to have to bother typing the date(s) on the command line, just the times. The program should be smart enough to figure them out.
 - The log date format doesn't include the year, so it should guess based on the current year, but nonetheless do the right thing around New Year's Day.
 - I want it to be fast -- it should use the fact that the lines are in order to seek around in the file and use a binary search.
 
Before I spend a bunch of time writing this, does it already exist?
© Server Fault or respective owner