What do I use when a cron job isn't enough? (php)

Posted by mike on Stack Overflow See other posts from Stack Overflow or by mike
Published on 2010-04-21T06:10:33Z Indexed on 2010/04/21 6:13 UTC
Read the original article Hit count: 327

Filed under:

php

|

cronjob

|

beanstalkd

I'm trying to figure out the most efficient way to running a pretty hefty PHP task thousands of times a day. It needs to make an IMAP connection to Gmail, loop over the emails, save this info to the database and save images locally.

Running this task every so often using a cron isn't that big of a deal, but I need to run it every minute and I know eventually the crons will start running on top of each other and cause memory issues.

What is the next step up when you need to efficiently run a task multiple times a minute? I've been reading about beanstalk & pheanstalk and I'm not entirely sure if that will do what I need. Thoughts???

© Stack Overflow or respective owner

Related posts about php

Magento, NGINX, PHP-FPM, APC, MEMCACHED, 16gb Ram CentOS, Spiking PHP-FPM to 100% CPU

as seen on Server Fault - Search for 'Server Fault'
I have been trying to resolve my issue of spiking cpu caused by php-fpm processes. I've reduced the php-fpm config settings to: pm = ondemand pm.max_children = 12 pm.start_servers = 2 pm.min_spare_servers = 2 pm.max_spare_servers = 10 pm.max_requests = 500 php_admin_value[memory_limit] = 128M Problem… >>> More
PHP Pear Installation on CentOS

as seen on Server Fault - Search for 'Server Fault'
[root@ip ~]# yum install php-pear* Reducing CentOS-5 Testing to included packages only Finished Setting up Install Process Package 1:php-pear-1.8.1-2.el5.centos.noarch already installed and latest versio … >>> More
Apache configurations for php "AddType text/html php" or "AddType application/x-httpd-php php .php"

as seen on Server Fault - Search for 'Server Fault'
I am taking over an application server and discover that it contain the following settings: AddType text/html php Although it works, but my understanding is that it should set as following: AddType application/x-httpd-php php .php What are the key differences between the two settings?… >>> More
mod_rewrite settings causes server to throw HTTP 500 errors instead of 404

as seen on Server Fault - Search for 'Server Fault'
Hello. I have a server with VBulletin forum (working under Apache 2.2, CentOS). The default settings for it in .htaccess are as follows: RewriteEngine on RewriteCond %{HTTP_HOST} ^gsmforum\.ru RewriteRule (.*) http://www.gsmforum.ru/$1 [R=301,L] # If you are having problems or are using VirtualDocumentRoot… >>> More
Problems installing Memcache (PECL extension)

as seen on Server Fault - Search for 'Server Fault'
I have installed memcached fine, and now I will need to install PECL extension memcache. Im running RedHat x86_64 es5. The installation gives me this: downloading memcache-2.2.6.tgz ... Starting to download memcache-2.2.6.tgz (35,957 bytes) ..........done: 35,957 bytes 11 source files, building running:… >>> More

Related posts about cronjob

PHP vs Batch file for mysql cronjob?

as seen on Server Fault - Search for 'Server Fault'
Hi, My server details: OS: Windows Server 2003 IIS6 Plesk 8.xx installed (currently using Plesk to set the cronjob) I need your advice. I have 2 methods: Method 1: Using php + mysqldump, create databases backup files into gzip, and then send email with attachment (each databases has around about… >>> More
Script executes successfully in commandline but not as a cronjob

as seen on Stack Overflow - Search for 'Stack Overflow'
I've a bash script that runs a ruby script that fetches my twitter feeds. ## /home/username/twittercron #!/bin/bash cd /home/username/twitter ruby twitter.rb friends It runs successfully in command line. /home/username/twittercron But when I try to run it as a cronjob, it ran but wasn't able… >>> More
PHP/Cronjob: What is my Cronjobs problem?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello there, I have no idea whats going on. But I have a script that looks like this. Cron job refuses to run it: include_once 'class_lib/mime_mail/mimeDecode.php'; include_once 'class_lib/Mail/IMAPv2.php'; include_once 'inc-functions.php'; include_once "$_SERVER[DOCUMENT_ROOT]/class_lib/DbFuctioneer… >>> More
Mac : Run Foundation Tool app as cronjob?

as seen on Stack Overflow - Search for 'Stack Overflow'
This may not be completely programming related ... In Xcode I wrote a little Foundation Tool application for maintenance. Copy files from A to B , delete logs and so on. Now I want to run the application in background once a day or once an hour. How do I set this up? Can it be done with a Foundation… >>> More
How do I increase timeout for a cronjob/crontab?

as seen on Stack Overflow - Search for 'Stack Overflow'
I have written a script that gets data from solr for which date is within the specified period, and I run the script using as a daily cron. The problem is the cronjob does not complete the task. If I manually run the script (for the same time period), it works well. If I reduce the specified time… >>> More