Need advice on cron job'ing a very large process

Posted by Arms on Stack Overflow See other posts from Stack Overflow or by Arms
Published on 2010-03-25T00:10:16Z Indexed on 2010/03/25 0:13 UTC
Read the original article Hit count: 160

Filed under:
|

I have a PHP script that grabs data from an external service and saves data to my database. I need this script to run once every minute for every user in the system (of which I expect to be thousands). My question is, what's the most efficient way to run this per user, per minute? At first I thought I would have a function that grabs all the user Ids from my database, iterate over the ids and perform the task for each one, but I think that as the number of users grow, this will take longer, and no longer fall within 1 minute intervals. Perhaps I should queue the user Ids, and perform the task individually for each one? In which case, I'm actually unsure of how to proceed.

Thanks in advance for any advice.

© Stack Overflow or respective owner

Related posts about php

Related posts about cron