multithreading issue

Posted by vbNewbie on Stack Overflow See other posts from Stack Overflow or by vbNewbie
Published on 2010-03-18T15:19:41Z Indexed on 2010/03/18 20:41 UTC
Read the original article Hit count: 311

Filed under:
|

I have written a multithreaded crawler and the process is simply creating threads and having them access a list of urls to crawl. They then access the urls and parse the html content. All this seems to work fine. Now when I need to write to tables in a database is when I experience issues. I have 2 declared arraylists that will contain the content each thread parse. The first arraylist is simply the rss feed links and the other arraylist contains the different posts. I then use a for each loop to iterate one while sequentially incrementing the other and writing to the database. My problem is that each time a new thread accesses one of the lists the content is changed and this affects the iteration. I tried using nested loops but it did not work before and this works fine using a single thread.I hope this makes sense. Here is my code:

          SyncLock dlock
                For Each l As String In links
                    finallinks.Add(l)
                Next
            End SyncLock

                SyncLock dlock
                For Each p As String In posts
                    finalposts.Add(p)
                Next
                End SyncLock
             ...
                Dim i As Integer = 0
          SyncLock dlock
         For Each rsslink As String In finallinks
             postlink = finalposts.Item(i)
             i = i + 1

finallinks and finalposts are the two arraylists. I did not include the rest of the code which shows the threads working but this is the essential part where my error occurs which is basically here

postlink = finalposts.Item(i)
i = i + 1
ERROR: index was out of range. Must be non-negative and less than the size of the collection

Is there an alternative?

© Stack Overflow or respective owner

Related posts about vb.net

Related posts about multithreading