memory tuning with rails/unicorn running on ubuntu
- by user970193
I am running unicorn on Ubuntu 11, Rails 3.0, and Ruby 1.8.7.
It is an 8 core ec2 box, and I am running 15 workers. CPU never seems to get pinned, and I seem to be handling requests pretty nicely.
My question concerns memory usage, and what concerns I should have with what I am seeing. (if any)
Here is the scenario:
Under constant load (about 15 reqs/sec coming in from nginx), over the course of an hour, each server in the 3 server cluster loses about 100MB / hour. This is a linear slope for about 6 hours, then it appears to level out, but still maybe appear to lose about 10MB/hour.
If I drop my page caches using the linux command echo 1 /proc/sys/vm/drop_caches, the available free memory shoots back up to what it was when I started the unicorns, and the memory loss pattern begins again over the hours.
Before:
total used free shared buffers cached
Mem: 7130244 5005376 2124868 0 113628 422856
-/+ buffers/cache: 4468892 2661352
Swap: 33554428 0 33554428
After:
total used free shared buffers cached
Mem: 7130244 4467144 2663100 0 228 11172
-/+ buffers/cache: 4455744 2674500
Swap: 33554428 0 33554428
My Ruby code does use memoizations and I'm assuming Ruby/Rails/Unicorn is keeping its own caches... what I'm wondering is should I be worried about this behaviour?
FWIW, my Unicorn config:
worker_processes 15
listen "#{CAPISTRANO_ROOT}/shared/pids/unicorn_socket", :backlog = 1024
listen 8080, :tcp_nopush = true
timeout 180
pid "#{CAPISTRANO_ROOT}/shared/pids/unicorn.pid"
GC.respond_to?(:copy_on_write_friendly=) and GC.copy_on_write_friendly = true
before_fork do |server, worker|
STDERR.puts "XXXXXXXXXXXXXXXXXXX BEFORE FORK"
print_gemfile_location
defined?(ActiveRecord::Base) and ActiveRecord::Base.connection.disconnect!
defined?(Resque) and Resque.redis.client.disconnect
old_pid = "#{CAPISTRANO_ROOT}/shared/pids/unicorn.pid.oldbin"
if File.exists?(old_pid) && server.pid != old_pid
begin
Process.kill("QUIT", File.read(old_pid).to_i)
rescue Errno::ENOENT, Errno::ESRCH
# already killed
end
end
File.open("#{CAPISTRANO_ROOT}/shared/pids/unicorn.pid.ok", "w"){|f| f.print($$.to_s)}
end
after_fork do |server, worker|
defined?(ActiveRecord::Base) and ActiveRecord::Base.establish_connection
defined?(Resque) and Resque.redis.client.connect
end
Is there a need to experiment enforcing more stringent garbage collection using OobGC (http://unicorn.bogomips.org/Unicorn/OobGC.html)? Or is this just normal behaviour, and when/as the system needs more memory, it will empty the caches by itself, without me manually running that cache command? Basically, is this normal, expected behaviour?
tia
