Why is my server performance degrading to the point of stopping, periodically?

Posted by Pascal Aschwanden on Server Fault See other posts from Server Fault or by Pascal Aschwanden
Published on 2011-03-03T02:22:09Z Indexed on 2012/06/03 16:42 UTC
Read the original article Hit count: 199

Filed under:
|

So, once in a while, I see in firebug that a request takes over 15 or even 60 seconds to respond and sometimes never. Here is what I've ruled out:

  • It's not the CPU, cuz every time I check the Server load its less then 6 for all 3 numbers
  • It's not the memory, because thats fairly low too, less the 50%
  • It's not the I/O anymore, because I've seen the graphs that Joyent sent back to me when I requested them, and they show less then 3MB of I/O (mostly all read).
  • It's not the SQL performance - I've profiled every last SQL command that runs, and they're all (99.9% of them anyway) running in less then 30ms, most run in less then 5ms.
  • Oh and I've been profiling all the script execution times, and even the when the problem occurs, the script always manages to finish in 50ms or less (that's 1 / 20th of a second ). Now, I do run alot of ajax calls. 1 every 2 seconds per user and I have 300 DAU+. But, even if all 300 are playing simultaneously, thats still only 150 calls per second max. The only other thing I can think of is that one of my neighbors is funky. The problem is highly intermittent. 99% of the time it works perfectly and there's excellent performance. but 99%+ is not good enough.
  • Eventually the performance gets so bad I have to restart the server, at which point everything is fine again. I've done this about 4 times now. Any ideas? Note: this is on joyent, vps, intro package 256mb of ram with bursting.

here are the mysql dump info:

Traffic     ø per hour
Received    18 MiB  29 MiB
Sent    134 MiB 221 MiB
Total   151 MiB 251 MiB
Connections ø per hour  %
max. concurrent connections 5   --- ---
Failed attempts 0   0.00    0.00%
Aborted 0   0.00    0.00%
Total   9,418   15.59 k 100.00%

© Server Fault or respective owner

Related posts about apache2

Related posts about mysql