Writing to a log4net FileAppender with multiple threads performance problems
- by Wayne
TickZoom is a very high performance app which uses it's own parallelization library and multiple O/S threads for smooth utilization of multi-core computers.
The app hits a bottleneck where users need to write information to a LogAppender from separate O/S threads.
The FileAppender uses the MinimalLock feature so that each thread can lock and write to the file and then release it for the next thread to write.
If MinimalLock gets disabled, log4net reports errors about the file being already locked by another process (thread).
A better way for log4net to do this would be to have a single thread that takes care of writing to the FileAppender and any other threads simply add their messages to a queue.
In that way, MinimalLock could be disabled to greatly improve performance of logging.
Additionally, the application does a lot of CPU intensive work so it will also improve performance to use a separate thread for writing to the file so the CPU never waits on the I/O to complete.
So the question is, does log4net already offer this feature? If so, how do you do enable threaded writing to a file? Is there another, more advanced appender, perhaps?
If not, then since log4net is already wrapped in the platform, that makes it possible to implement a separate thread and queue for this purpose in the TickZoom code.
Sincerely,
Wayne
