Multithreading recommendation based on program description

Posted by user260197 on Stack Overflow See other posts from Stack Overflow or by user260197
Published on 2010-03-10T15:41:05Z Indexed on 2010/03/12 7:57 UTC
Read the original article Hit count: 510

Filed under:
|
|

I would like to describe some specifics of my program and get feedback on what the best multithreading model to use would be most applicable. I've spent a lot of time now reading on ThreadPool, Threads, Producer/Consumer, etc. and have yet to come to solid conclusions.

I have a list of files (all the same format) but with different contents. I have to perform work on each file. The work consists of reading the file, some processing that takes about 1-2 minutes of straight number crunching, and then writing large output files at the end.

I would like the UI interface to still be responsive after I initiate the work on the specified files.

Some questions:

  1. What model/mechanisms should I use? Producer/Consumer, WorkPool, etc.
  2. Should I use a BackgroundWorker in the UI for responsiveness or can I launch the threading from within the Form as long as I leave the UI thread alone to continue responding to user input?
  3. How could I take results or status of each individual work on each file and report it to the UI in a thread safe way to give user feedback as the work progresses (there can be close to 1000 files to process)

Update:

Great feedback so far, very helpful. I'm adding some more details that are asked below:

  • Output is to multiple independent files. One set of output files per "work item" that then themselves gets read and processed by another process before the "work item" is complete

  • The work items/threads do not share any resources.

  • The work items are processed in part using a unmanaged static library that makes use of boost libraries.

© Stack Overflow or respective owner

Related posts about .NET

Related posts about threading