I want to be able to read and write a large file in parallel, or if not in parallel, at least in blocks so that I don't use up so much memory.
This is my current code:
// Define memory stream which will be used to hold encrypted data.
MemoryStream memoryStream = new MemoryStream();
// Define cryptographic stream (always use Write mode for encryption).
CryptoStream cryptoStream = new CryptoStream(memoryStream,
encryptor,
CryptoStreamMode.Write);
//start encrypting
using (BinaryReader reader = new BinaryReader(File.Open(fileIn, FileMode.Open))) {
byte[] buffer = new byte[1024 * 1024];
int read = 0;
do {
read = reader.Read(buffer, 0, buffer.Length);
cryptoStream.Write(buffer, 0, read);
} while (read == buffer.Length);
}
// Finish encrypting.
cryptoStream.FlushFinalBlock();
// Convert our encrypted data from a memory stream into a byte array.
//byte[] cipherTextBytes = memoryStream.ToArray();
//write our memory stream to a file
memoryStream.Position = 0;
using (BinaryWriter writer = new BinaryWriter(File.Open(fileOut, FileMode.Create))) {
byte[] buffer = new byte[1024 * 1024];
int read = 0;
do {
read = memoryStream.Read(buffer, 0, buffer.Length);
writer.Write(buffer, 0, read);
} while (read == buffer.Length);
}
// Close both streams.
memoryStream.Close();
cryptoStream.Close();
As you can see, it reads the entire file into memory, encrypts it, then writes it out. If I happen to be encrypting files that are very large (2GB+) it tends not to work, or at the very least, consumes ~97% of my memory.
How could I do it in a more effective manner?
