Memory mapping of files and system cache behavior in WinXP
- by Canopus
Our application is memory intensive and deals with reading a large number of disk files. The total load can be more than 3 GB.
There is a custom memory manager that uses memory mapped files to achieve reading of such a huge data. The files are mapped into the process memory space only when needed and with this the process memory is well under control. But what is observed is, with memory mapping, the system cache keeps on increasing until it occupies the available physical memory. This leads to the slowing down of the entire system.
My question is how to prevent system cache from hogging the physical memory? I attempted to remove the file buffering (by using FILE_FLAG_NO_BUFFERING ), but with this, the read operations take considerable amount of time and slows down the application performance. How to achieve the scalability without sacrificing much on performance. What are the common techniques used in such cases?
I dont have a good understanding of the WinXP OS caching behavior. Any good links explaining the same would also be helpful.
