Shared value in parallel python

Posted by Jonathan on Stack Overflow See other posts from Stack Overflow or by Jonathan
Published on 2010-05-05T01:58:44Z Indexed on 2010/05/05 2:08 UTC
Read the original article Hit count: 400

Filed under:
|

Hey all-

I'm using ParallelPython to develop a performance-critical script. I'd like to share one value between the 8 processes running on the system. Please excuse the trivial example but this illustrates my question.

def findMin(listOfElements):
    for el in listOfElements:
        if el < min:
             min = el

import pp
min = 0
myList = range(100000)
job_server = pp.Server()
f1 = job_server.submit(findMin, myList[0:25000])
f2 = job_server.submit(findMin, myList[25000:50000])
f3 = job_server.submit(findMin, myList[50000:75000]) 
f4 = job_server.submit(findMin, myList[75000:100000]) 

The pp docs don't seem to describe a way to share data across processes. Is it possible?

If so, is there a standard locking mechanism (like in the threading module) to confirm that only one update is done at a time?

l = Lock()
if(el < min):
     l.acquire
     if(el < min):
         min = el
     l.release

I understand I could keep a local min and compare the 4 in the main thread once returned, but by sharing the value I can do some better pruning of my BFS binary tree and potentially save a lot of loop iterations.

Thanks-

Jonathan

© Stack Overflow or respective owner

Related posts about parallel-processing

Related posts about python