Shared value in parallel python
- by Jonathan
Hey all-
I'm using ParallelPython to develop a performance-critical script. I'd like to share one value between the 8 processes running on the system. Please excuse the trivial example but this illustrates my question.
def findMin(listOfElements):
for el in listOfElements:
if el < min:
min = el
import pp
min = 0
myList = range(100000)
job_server = pp.Server()
f1 = job_server.submit(findMin, myList[0:25000])
f2 = job_server.submit(findMin, myList[25000:50000])
f3 = job_server.submit(findMin, myList[50000:75000])
f4 = job_server.submit(findMin, myList[75000:100000])
The pp docs don't seem to describe a way to share data across processes. Is it possible?
If so, is there a standard locking mechanism (like in the threading module) to confirm that only one update is done at a time?
l = Lock()
if(el < min):
l.acquire
if(el < min):
min = el
l.release
I understand I could keep a local min and compare the 4 in the main thread once returned, but by sharing the value I can do some better pruning of my BFS binary tree and potentially save a lot of loop iterations.
Thanks-
Jonathan
