Multithreading recommendation based on program description
- by user260197
I would like to describe some specifics of my program and get feedback on what the best multithreading model to use would be most applicable. I've spent a lot of time now reading on ThreadPool, Threads, Producer/Consumer, etc. and have yet to come to solid conclusions.
I have a list of files (all the same format) but with different contents. I have to perform work on each file. The work consists of reading the file, some processing that takes about 1-2 minutes of straight number crunching, and then writing large output files at the end.
I would like the UI interface to still be responsive after I initiate the work on the specified files.
Some questions:
What model/mechanisms should I use? Producer/Consumer, WorkPool, etc.
Should I use a BackgroundWorker in the UI for responsiveness or can I launch the threading from within the Form as long as I leave the UI thread alone to continue responding to user input?
How could I take results or status of each individual work on each file and report it to the UI in a thread safe way to give user feedback as the work progresses (there can be close to 1000 files to process)
Update:
Great feedback so far, very helpful. I'm adding some more details that are asked below:
Output is to multiple independent files. One set of output files per "work item" that then themselves gets read and processed by another process before the "work item" is complete
The work items/threads do not share any resources.
The work items are processed in part using a unmanaged static library that makes use of boost libraries.
