Problems with Threading in Python 2.5, KeyError: 51, Help debugging?
- by vignesh-k
I have a python script which runs a particular script large number of times (for monte carlo purpose) and the way I have scripted it is that, I queue up the script the desired number of times it should be run then I spawn threads and each thread runs the script once and again when its done.
Once the script in a particular thread is finished, the output is written to a file by accessing a lock (so my guess was that only one thread accesses the lock at a given time). Once the lock is released by one thread, the next thread accesses it and adds its output to the previously written file and rewrites it.
I am not facing a problem when the number of iterations is small like 10 or 20 but when its large like 50 or 150, python returns a KeyError: 51 telling me element doesn't exist and the error it points out to is within the lock which puzzles me since only one thread should access the lock at once and I do not expect an error.
This is the class I use:
class errorclass(threading.Thread):
def __init__(self, queue):
self.__queue=queue
threading.Thread.__init__(self)
def run(self):
while 1:
item = self.__queue.get()
if item is None: break
result = myfunction()
lock = threading.RLock()
lock.acquire()
ADD entries from current thread to entries in file and
REWRITE FILE
lock.release()
queue = Queue.Queue()
for i in range(threads):
errorclass(queue).start()
for i in range(desired iterations):
queue.put(i)
for i in range(threads):
queue.put(None)
Python returns with KeyError: 51 for large number of desired iterations during the adding/write file operation after lock access, I am wondering if this is the correct way to use the lock since every thread has a lock operation rather than every thread accessing a shared lock? What would be the way to rectify this?
