Computing MD5SUM of large files in C#

Posted by spkhaira on Stack Overflow See other posts from Stack Overflow or by spkhaira
Published on 2009-04-30T07:11:25Z Indexed on 2010/03/19 19:11 UTC
Read the original article Hit count: 581

Filed under:

c#

|

md5

|

hashing

|

large-files

I am using following code to compute MD5SUM of a file -

byte[] b = System.IO.File.ReadAllBytes(file);
string sum = BitConverter.ToString(new MD5CryptoServiceProvider().ComputeHash(b));

This works fine normally, but if I encounter a large file (~1GB) - e.g. an iso image or a DVD VOB file - I get an Out of Memory exception.

Though, I am able to compute the MD5SUM in cygwin for the same file in about 10secs.

Please suggest how can I get this to work for big files in my program.

Thanks

© Stack Overflow or respective owner

Related posts about c#

.NET WebRequest.PreAuthenticate not quite what it sounds like

as seen on West-Wind - Search for 'West-Wind'
I’ve run into the problem a few times now: How to pre-authenticate .NET WebRequest calls doing an HTTP call to the server – essentially send authentication credentials on the very first request instead of waiting for a server challenge first? At first glance this sound like it should be easy:… >>> More
HttpWebRequest and Ignoring SSL Certificate Errors

as seen on West-Wind - Search for 'West-Wind'
Man I can't believe this. I'm still mucking around with OFX servers and it drives me absolutely crazy how some these servers are just so unbelievably misconfigured. I've recently hit three different 3 major brokerages which fail HTTP validation with bad or corrupt certificates at least according to… >>> More
Dynamic Type to do away with Reflection

as seen on West-Wind - Search for 'West-Wind'
The dynamic type in C# 4.0 is a welcome addition to the language. One thing I’ve been doing a lot with it is to remove explicit Reflection code that’s often necessary when you ‘dynamically’ need to walk and object hierarchy. In the past I’ve had a number of ReflectionUtils that used string based expressions… >>> More
The dynamic Type in C# Simplifies COM Member Access from Visual FoxPro

as seen on West-Wind - Search for 'West-Wind'
I’ve written quite a bit about Visual FoxPro interoperating with .NET in the past both for ASP.NET interacting with Visual FoxPro COM objects as well as Visual FoxPro calling into .NET code via COM Interop. COM Interop with Visual FoxPro has a number of problems but one of them at least got a lot… >>> More
Finding a Relative Path in .NET

as seen on West-Wind - Search for 'West-Wind'
Here’s a nice and simple path utility that I’ve needed in a number of applications: I need to find a relative path based on a base path. So if I’m working in a folder called c:\temp\templates\ and I want to find a relative path for c:\temp\templates\subdir\test.txt I want to receive back subdir\test… >>> More

Related posts about md5

Getting Oracle's MD5 to match PHP's MD5

as seen on Stack Overflow - Search for 'Stack Overflow'
Hi all, I'm trying to compare an MD5 checksum generated by PHP to one generated by Oracle 10g. However it seems I'm comparing apples to oranges. Here's what I did to test the comparison: //md5 tests //php md5 print md5('testingthemd5function'); print '<br/><br/>'; //oracle… >>> More
Python: How to display the calculated MD5 value in my browser?

as seen on Stack Overflow - Search for 'Stack Overflow'
Hello, I was given this Python code that would calculate an MD5 value for any phrase: import md5 md5.new("Nobody inspects the spammish repetition").digest() (The phrase here is: "Nobody inspects the spammish repetition") What I want to do is display this value in my browser. How do I do it in… >>> More
How are hash functions like MD5 unique?

as seen on Stack Overflow - Search for 'Stack Overflow'
Im aware that MD5 has had some collisions but this is more of a high level question about hashing functions. If MD5 hashes any arbitrary string into a 32-digit hex value, then according to the Pigeonhole Principle surely this can not be unique as there are more unique arbitrary strings than there… >>> More
Converting a md5 hash byte array to a string

as seen on Stack Overflow - Search for 'Stack Overflow'
How can I convert the hashed result, which a byte array, to a string? byte[] bytePassword = Encoding.UTF8.GetBytes(password); using (MD5 md5 = MD5.Create()) { byte[] byteHashedPassword = md5.ComputeHash(bytePassword); } So I need to convert… >>> More
How do I compare the md5sum of a file with the md5 file (that was available to download with the file)?

as seen on Ask Ubuntu - Search for 'Ask Ubuntu'
Images are available for a distro on http://livedistro.org/gnulinux/israel-remix-team-mint-12. I want to use the 32-bit version. I have downloaded the ISO file for the 32-bit version (customdist.iso). I have downloaded the md5 file for the ISO file (customdist.iso.md5). I want to calculate the md5sum… >>> More