Resque: Slow worker startup and Forking

Posted by David John on Stack Overflow See other posts from Stack Overflow or by David John
Published on 2010-06-01T12:57:56Z Indexed on 2010/06/01 13:03 UTC
Read the original article Hit count: 234

Filed under:

ruby-on-rails

|

resque

I'm currently moving my application from a Linode setup to EC2. Redis is currently installed on a remote instance with various worker instances interacting with the queue. Thats all going fantastic.

My problem is with the amount of time it takes for a worker to be 'instantiated' and slow forking. Starting a worker will usually take between 30 seconds and a minute(from god.rb starting the worker rake task and the worker actively starting work on the queue). I could live with that, but I've not experienced such a wait time on my current Linode production box so I believe its one of my symptoms to a bigger problem. Next issue is that jobs that took a second or less in my previous environment now seem to take about 5 to 10 times longer..

I'm assuming this must be some sort of issue with my Ubuntu install on EC2? One notable difference is that I'm running REE 1.8.7-2010.01 in my new setup, and REE 1.8.6 on the old Linode boxes.

Anyone else experienced these issues?

© Stack Overflow or respective owner

Related posts about ruby-on-rails

Ruby on Rails - How can I start? [closed]

as seen on Programmers - Search for 'Programmers'
I have misconception in understanding the relationship between Ruby language and Ruby on Rails Framework. Because of 'I am an absolute beginner' in web development I have no idea if I have to grasp the fundamentals of Ruby before I go with Ruby on Rails! I also want to ask who is behind both Ruby… >>> More
DES3 decryption in Ruby on Rails

as seen on Stack Overflow - Search for 'Stack Overflow'
My RoR server receives a string, that was encrypted in C++ application using des3 with base64 encoding The cipher object is created so: cipher = OpenSSL::Cipher::Cipher::new("des3") cipher.key = key_str cipher.iv = iv_str key_str and iv_str: are string representations of key and initialization… >>> More
Ruby on rails: Image downloads with Authentication/Authorization/Time outs

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi Guys, I'm having few doubts on implementing file downloads. I'm creating an app where I use attachment_fu with Amazon s3 to upload files. Things are working pretty well so far on uploading side. Now its the time to start the file downloads. Here is what I need, a logged in user search and browse… >>> More
Ruby on Rails deployment, on "thin" server with lot of attachments

as seen on Stack Overflow - Search for 'Stack Overflow'
A lot of PDFs are stored inside MySQL as a BLOB field for each PDF file. The average file size is 500K each. The Rails app will stream the :binary data as file downloads, where there is a user click on the download link. Assume there is a maximum of 5 users downloading 5 PDFs concurrently, what… >>> More
Apply Behavior Driven Development to Ruby on Rails with Rspec

as seen on Internet.com - Search for 'Internet.com'
Applying the Behavior Driven Development (BDD) methodology to your Rails development takes you beyond traditional Test Driven Development. Learn how to do it with the Ruby-based BDD framework RSpec. >>> More

Related posts about resque

Resque Runtime Error at /workers: wrong number of arguments for 'exists' command

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm having a runtime errror when i'm looking at the "workers" tab on resque-web (localhost). Everything else works. Edit: when this error occurs, i also have some (3 or 4) unknown workers 'not working'. I think they are responsible for the error but i don't understand how they got here Can you help… >>> More
Troubles installing/starting Redis via Resque

as seen on Server Fault - Search for 'Server Fault'
Trying to complete instructions for Resque/Redis installation here: https://github.com/defunkt/resque/blob/master/README.markdown Am stuck at where I'm trying to start up Redis via Resque at the following command: Craig:/usr/local/src/resque$ rake redis:start (in /usr/local/src/resque) Detach… >>> More
Rails Resque workers fail with PGError: server closed the connection unexpectedly

as seen on Stack Overflow - Search for 'Stack Overflow'
I have site running rails application and resque workers running in production mode, on Ubuntu 9.10, Rails 2.3.4, ruby-ee 2010.01, PostgreSQL 8.4.2 Workers constantly raised errors: PGError: server closed the connection unexpectedly. My best guess is that master resque process establishes connection… >>> More
Resque: Slow worker startup and Forking

as seen on Stack Overflow - Search for 'Stack Overflow'
I'm currently moving my application from a Linode setup to EC2. Redis is currently installed on a remote instance with various worker instances interacting with the queue. Thats all going fantastic. My problem is with the amount of time it takes for a worker to be 'instantiated' and slow forking… >>> More
Guide to migrate from delayed_job to resque?

as seen on Stack Overflow - Search for 'Stack Overflow'
Does anyone have or know of a guide for how to migrate from Delayed Job to Resque? I can't find any on Google and figure I must not be the first one doing this. Just a general list of changes that need to made and things to watch out for would be great. >>> More