Why is my server performance degrading to the point of stopping, periodically?
        Posted  
        
            by 
                Pascal Aschwanden
            
        on Server Fault
        
        See other posts from Server Fault
        
            or by Pascal Aschwanden
        
        
        
        Published on 2011-03-03T02:22:09Z
        Indexed on 
            2012/06/03
            16:42 UTC
        
        
        Read the original article
        Hit count: 295
        
So, once in a while, I see in firebug that a request takes over 15 or even 60 seconds to respond and sometimes never. Here is what I've ruled out:
- It's not the CPU, cuz every time I check the Server load its less then 6 for all 3 numbers
 - It's not the memory, because thats fairly low too, less the 50%
 - It's not the I/O anymore, because I've seen the graphs that Joyent sent back to me when I requested them, and they show less then 3MB of I/O (mostly all read).
 - It's not the SQL performance - I've profiled every last SQL command that runs, and they're all (99.9% of them anyway) running in less then 30ms, most run in less then 5ms.
 - Oh and I've been profiling all the script execution times, and even the when the problem occurs, the script always manages to finish in 50ms or less (that's 1 / 20th of a second ). Now, I do run alot of ajax calls. 1 every 2 seconds per user and I have 300 DAU+. But, even if all 300 are playing simultaneously, thats still only 150 calls per second max. The only other thing I can think of is that one of my neighbors is funky. The problem is highly intermittent. 99% of the time it works perfectly and there's excellent performance. but 99%+ is not good enough.
 - Eventually the performance gets so bad I have to restart the server, at which point everything is fine again. I've done this about 4 times now. Any ideas? Note: this is on joyent, vps, intro package 256mb of ram with bursting.
 
here are the mysql dump info:
Traffic     ø per hour
Received    18 MiB  29 MiB
Sent    134 MiB 221 MiB
Total   151 MiB 251 MiB
Connections ø per hour  %
max. concurrent connections 5   --- ---
Failed attempts 0   0.00    0.00%
Aborted 0   0.00    0.00%
Total   9,418   15.59 k 100.00%
        © Server Fault or respective owner