I'm building a messaging solution with the followingsetup: all the messages are saved on a DB, two or more reader processes will read from this DB and send data to other process(es) which will send it over the network.
My approach is depicted below,
The following have 4 sender process with 4 fifos, and 2 readers with 2 fifos
reader0 ? read data from DB
reader1 ? read data from DB
sending part
network_handler0 ? network_handler_fifo0 ? reader0
network_handler1 ? network_handler_fifo1 ? reader1
network_handler2 ? network_handler_fifo2 ? reader0
network_handler3 ? network_handler_fifo3 ? reader1
receiving part
network_handler0 ? reader_fifo0 ? reader0 ? write to DB
network_handler1 ? reader_fifo1 ? reader1 ? write to DB
network_handler2 ? reader_fifo0 ? reader0 ? write to DB
network_handler3 ? reader_fifo1 ? reader1 ? write to DB
I have few problem with this setup, and please note that the number of processes could be more than that based on the environment, so I could make it 20 readers and 10 network_handlers or it it could as shown above.
The size of the buffer is 64K and the message size is 200k, is this small enough to make the write/read to/from fifo atomic?
How can make the processes aware of each other, so for example, reader 0 writes to network_handler_fifo0 and network_handler_fifo2, how can I make it start writing on other fifo if the current ones are full or their network_handlers are dea
d
I thought about making the reader process writing more general in writing, so for example it writes to all network fifos using lock mechanism and stop writing on the one that its process dead, I didn't use it as lock mechanism could slow thing down.
BTW, each network_handler is an SCTP association, so network_handler0 is association 0, network_handler1 is association 1 and so on.
Any idea is appreciated. I mean even if I have to change the setup above.
