Server Memory with Magento
- by Mohamed Elgharabawy
I have a cloud server with the following specifications:
2vCPUs
4G RAM
160GB Disk Space
Network 400Mb/s
System Image: Ubuntu 12.04 LTS
I am only running Magento CE 1.7.0.2 on this server. Nothing else.
Usually, the server has a loading time of 4-5 seconds. Recently, this has dropped to over 30 seconds and sometimes the server just goes away and I get HTTP error reports to my email stating that HTTP requests took more than 20000ms.
Running top command and sorting them returns the following:
top - 15:29:07 up 3:40, 1 user, load average: 28.59, 25.95, 22.91
Tasks: 112 total, 30 running, 82 sleeping, 0 stopped, 0 zombie
Cpu(s): 90.2%us, 9.3%sy, 0.0%ni, 0.0%id, 0.0%wa, 0.0%hi, 0.3%si, 0.2%st
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
31901 www-data 20 0 360m 71m 5840 R 7 1.8 1:39.51 apache2
32084 www-data 20 0 362m 72m 5548 R 7 1.8 1:31.56 apache2
32089 www-data 20 0 348m 59m 5660 R 7 1.5 1:41.74 apache2
32295 www-data 20 0 343m 54m 5532 R 7 1.4 2:00.78 apache2
32303 www-data 20 0 354m 65m 5260 R 7 1.6 1:38.76 apache2
32304 www-data 20 0 346m 56m 5544 R 7 1.4 1:41.26 apache2
32305 www-data 20 0 348m 59m 5640 R 7 1.5 1:50.11 apache2
32291 www-data 20 0 358m 69m 5256 R 6 1.7 1:44.26 apache2
32517 www-data 20 0 345m 56m 5532 R 6 1.4 1:45.56 apache2
30473 www-data 20 0 355m 66m 5680 R 6 1.7 2:00.05 apache2
32093 www-data 20 0 352m 63m 5848 R 6 1.6 1:53.23 apache2
32302 www-data 20 0 345m 56m 5512 R 6 1.4 1:55.87 apache2
32433 www-data 20 0 346m 57m 5500 S 6 1.4 1:31.58 apache2
32638 www-data 20 0 354m 65m 5508 R 6 1.6 1:36.59 apache2
32230 www-data 20 0 347m 57m 5524 R 6 1.4 1:33.96 apache2
32231 www-data 20 0 355m 66m 5512 R 6 1.7 1:37.47 apache2
32233 www-data 20 0 354m 64m 6032 R 6 1.6 1:59.74 apache2
32300 www-data 20 0 355m 66m 5672 R 6 1.7 1:43.76 apache2
32510 www-data 20 0 347m 58m 5512 R 6 1.5 1:42.54 apache2
32521 www-data 20 0 348m 59m 5508 R 6 1.5 1:47.99 apache2
32639 www-data 20 0 344m 55m 5512 R 6 1.4 1:34.25 apache2
32083 www-data 20 0 345m 56m 5696 R 5 1.4 1:59.42 apache2
32085 www-data 20 0 347m 58m 5692 R 5 1.5 1:42.29 apache2
32293 www-data 20 0 353m 64m 5676 R 5 1.6 1:52.73 apache2
32301 www-data 20 0 348m 59m 5564 R 5 1.5 1:49.63 apache2
32528 www-data 20 0 351m 62m 5520 R 5 1.6 1:36.11 apache2
31523 mysql 20 0 3460m 576m 8288 S 5 14.4 2:06.91 mysqld
32002 www-data 20 0 345m 55m 5512 R 5 1.4 2:01.88 apache2
32080 www-data 20 0 357m 68m 5512 S 5 1.7 1:31.30 apache2
32163 www-data 20 0 347m 58m 5512 S 5 1.5 1:58.68 apache2
32509 www-data 20 0 345m 56m 5504 R 5 1.4 1:49.54 apache2
32306 www-data 20 0 358m 68m 5504 S 4 1.7 1:53.29 apache2
32165 www-data 20 0 344m 55m 5524 S 4 1.4 1:40.71 apache2
32640 www-data 20 0 345m 56m 5528 R 4 1.4 1:36.49 apache2
31888 www-data 20 0 359m 70m 5664 R 4 1.8 1:57.07 apache2
32511 www-data 20 0 357m 67m 5512 S 3 1.7 1:47.00 apache2
32054 www-data 20 0 357m 68m 5660 S 2 1.7 1:53.10 apache2
1 root 20 0 24452 2276 1232 S 0 0.1 0:01.58 init
Moreover, running free -m returns the following:
total used free shared buffers cached
Mem: 4003 3919 83 0 118 901
-/+ buffers/cache: 2899 1103
Swap: 0 0 0
To investigate this further, I have installed apache buddy, it recommeneded that I need to reduce the maxclient connections. Which I did.
I also installed MysqlTuner and it suggests that I need to set my innodb_buffer_pool_size to = 3.0G.
However, I cannot do that, since the whole memory is 4G.
Here is the output from apache buddy:
### GENERAL REPORT ###
Settings considered for this report:
Your server's physical RAM: 4003MB
Apache's MaxClients directive: 40
Apache MPM Model: prefork
Largest Apache process (by memory): 73.77MB
[ OK ] Your MaxClients setting is within an acceptable range.
Max potential memory usage: 2950.8 MB
Percentage of RAM allocated to Apache 73.72 %
And this is the output of MySQLTuner:
-------- Performance Metrics -------------------------------------------------
[--] Up for: 47m 22s (675K q [237.552 qps], 12K conn, TX: 1B, RX: 300M)
[--] Reads / Writes: 45% / 55%
[--] Total buffers: 2.1G global + 2.7M per thread (151 max threads)
[OK] Maximum possible memory usage: 2.5G (64% of installed RAM)
[OK] Slow queries: 0% (0/675K)
[OK] Highest usage of available connections: 26% (40/151)
[OK] Key buffer size / total MyISAM indexes: 36.0M/18.7M
[OK] Key buffer hit rate: 100.0% (245K cached / 105 reads)
[OK] Query cache efficiency: 92.5% (500K cached / 541K selects)
[!!] Query cache prunes per day: 302886
[OK] Sorts requiring temporary tables: 0% (1 temp sorts / 15K sorts)
[!!] Joins performed without indexes: 12135
[OK] Temporary tables created on disk: 25% (8K on disk / 32K total)
[OK] Thread cache hit rate: 90% (1K created / 12K connections)
[!!] Table cache hit rate: 17% (400 open / 2K opened)
[OK] Open file limit used: 12% (123/1K)
[OK] Table locks acquired immediately: 100% (196K immediate / 196K locks)
[!!] InnoDB buffer pool / data size: 2.0G/3.5G
[OK] InnoDB log waits: 0
-------- Recommendations -----------------------------------------------------
General recommendations:
Run OPTIMIZE TABLE to defragment tables for better performance
MySQL started within last 24 hours - recommendations may be inaccurate
Enable the slow query log to troubleshoot bad queries
Adjust your join queries to always utilize indexes
Increase table_cache gradually to avoid file descriptor limits
Read this before increasing table_cache over 64: http://bit.ly/1mi7c4C
Variables to adjust:
query_cache_size ( 64M)
join_buffer_size ( 128.0K, or always use indexes with joins)
table_cache ( 400)
innodb_buffer_pool_size (= 3G)
Last but not least, the server still has more than 60% of free disk space.
Now, based on the above, I have few questions:
Are these numbers normal? Do they make sense?
Do I need to upgrade the server?
If I don't need to upgrade and my configuration is not correct, how do I optimize it?
