Windows Azure worker roles: One big job or many small jobs?
- by Ryan Elkins
Is there any inherent advantage when using multiple workers to process pieces of procedural code versus processing the entire load?
In other words, if my workflow looks like this:
Get work from queue0 and do A
Store result from A in queue1
Get result from queue 1 and do B
Store result from B in queue2
Get result from queue2 and do C
Is there an inherent advantage to using 3 workers who each do the entire process themselves versus 3 workers that each do a part of the work (Worker 1 does 1 & 2, worker 2 does 3 & 4, worker 3 does 5).
If we only care about working being done (finished with step 5) it would seem that it scales the same way (once you're using at least 3 workers). Maybe the big job is better because workers with that setup have less bottleneck issues?
