python threading and performace?

Posted by kumar on Stack Overflow See other posts from Stack Overflow or by kumar
Published on 2010-06-10T07:39:04Z Indexed on 2010/06/10 7:42 UTC
Read the original article Hit count: 251

Filed under:
|
|

I had to do heavy I/o bound operation, i.e Parsing large files and converting from one format to other format. Initially I used to do it serially, i.e parsing one after another..! Performance was very poor ( it used take 90+ seconds). So I decided to use threading to improve the performance. I created one thread for each file. ( 4 threads)

 for file in file_list:
            t=threading.Thread(target = self.convertfile,args = file)
            t.start()
            ts.append(t)
        for t in ts:
            t.join()

But for my astonishment, there is no performance improvement whatsoever. Now also it takes around 90+ seconds to complete the task. As this is I/o bound operation , I had expected to improve the performance. What am I doing wrong?

© Stack Overflow or respective owner

Related posts about python

Related posts about Performance