How to work threading with ConcurrentQueue<T>.
- by dboarman
I am trying to figure out what the best way of working with a queue will be. I have a process that returns a DataTable. Each DataTable, in turn, is merged with the previous DataTable. There is one problem, too many records to hold until the final BulkCopy (OutOfMemory).
So, I have determined that I should process each incoming DataTable immediately. Thinking about the ConcurrentQueue<T>...but I don't see how the WriteQueuedData() method would know to dequeue a table and write it to the database.
For instance:
public class TableTransporter
{
private ConcurrentQueue<DataTable> tableQueue = new ConcurrentQueue<DataTable>();
public TableTransporter()
{
tableQueue.OnItemQueued += new EventHandler(WriteQueuedData); // no events available
}
public void ExtractData()
{
DataTable table;
// perform data extraction
tableQueue.Enqueue(table);
}
private void WriteQueuedData(object sender, EventArgs e)
{
BulkCopy(e.Table);
}
}
My first question is, aside from the fact that I don't actually have any events to subscribe to, if I call ExtractData() asynchronously will this be all that I need? Second, is there something I'm missing about the way ConcurrentQueue<T> functions and needing some form of trigger to work asynchronously with the queued objects?
