Strategy to store/average logs of pings
        Posted  
        
            by 
                José Tomás Tocino
            
        on Programmers
        
        See other posts from Programmers
        
            or by José Tomás Tocino
        
        
        
        Published on 2014-03-12T16:21:02Z
        Indexed on 
            2014/06/10
            21:46 UTC
        
        
        Read the original article
        Hit count: 258
        
I'm developing a site to monitor web services. The most basic type of check is sending a ping, storing the response time in a CheckLog object. By default, PingCheck objects are triggered every minute, so in one hour you get 60 CheckLogs and in one day you get 1440 CheckLogs. 
That's a lot of them, I don't need to store such level of detail, so I've set a up collapsing mechanism that periodically takes the uncollapsed CheckLogs older than 24h and collapses (averages) them in intervals of 30 minutes. So, if you have 360 CheckLogs that have been saved from 0:00 to 6:00, after collapsing you retain just 12 of them. The problem.. well, is this:

After averaging the response times, the graph changes drastically. What can I do to improve this? Guess one option could be narrowing the interval duration to 15 min.
I've seen the graphs at the GitHub status page and they do not seem to suffer from this problem.
I'd appreciate any kind of information you could give me about this area.
© Programmers or respective owner