Standard term for a thread I/O reorder buffer?

Posted by Crashworks on Stack Overflow See other posts from Stack Overflow or by Crashworks
Published on 2010-05-28T21:46:04Z Indexed on 2010/05/28 21:52 UTC
Read the original article Hit count: 200

I have a case where many threads all concurrently generate data that is ultimately written to one long, serial file. I need to somehow serialize these writes so that the file gets written in the right order.

ie, I have an input queue of 2048 jobs j0..jn, each of which produces a chunk of data oi. The jobs run in parallel on, say, eight threads, but the output blocks have to appear in the file in the same order as the corresponding input blocks — the output file has to be in the order o0o1o2...

The solution to this is pretty self evident: I need some kind of buffer that accumulates and writes the output blocks in the correct order, similar to a CPU reorder buffer in Tomasulo's algorithm, or to the way that TCP reassembles out-of-order packets before passing them to the application layer.

Before I go code it, I'd like to do a quick literature search to see if there are any papers that have solved this problem in a particularly clever or efficient way, since I have severe realtime and memory constraints. I can't seem to find any papers describing this though; a Scholar search on every permutation of [threads, concurrent, reorder buffer, reassembly, io, serialize] hasn't yielded anything useful. I feel like I must just not be searching the right terms.

Is there a common academic name or keyword for this kind of pattern that I can search on?

© Stack Overflow or respective owner

Related posts about concurrency

Related posts about computer-science