Python sqlite3 and concurrency

Posted by RexE on Stack Overflow See other posts from Stack Overflow or by RexE
Published on 2008-12-26T06:51:54Z Indexed on 2010/04/05 12:43 UTC
Read the original article Hit count: 151

Filed under:
|

I have a Python program that uses the "threading" module. Once every second, my program starts a new thread that fetches some data from the web, and stores this data to my hard drive. I would like to use sqlite3 to store these results, but I can't get it to work. The issue seems to be about the following line:

conn = sqlite3.connect("mydatabase.db")
  • If I put this line of code inside each thread, I get an OperationalError telling me that the database file is locked. I guess this means that another thread has mydatabase.db open through a sqlite3 connection and has locked it.
  • If I put this line of code in the main program and pass the connection object (conn) to each thread, I get a ProgrammingError, saying that SQLite objects created in a thread can only be used in that same thread.

Previously I was storing all my results in CSV files, and did not have any of these file-locking issues. Hopefully this will be possible with sqlite. Any ideas?

© Stack Overflow or respective owner

Related posts about python

Related posts about sqlite