473,385 Members | 1,397 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

MD5 for Large Files

I am new to VB.NET and I am trying to learn. So, your indulgence for the
triviality of my questions is kindly requested.

I would like to calculate an MD5 hash for very lage files. The examples I
came across read the file into a byte array and apply the hash to that
array. The following code illustrates what I am doing. I would like to
perform the hash calculation on a stream. Is this possible? If so, an
example would be greatly appreciated.

Imports System.Text
Imports System.Security.Cryptography
Imports System.IO

Public Class Form1
Inherits System.Windows.Forms.Form
Dim fs As FileStream = New FileStream("c:\ConfDenise.PDF",
FileMode.Open)
Dim r As BinaryReader = New BinaryReader(fs)

Public Function GenerateHash(ByRef Buff() As Byte) As String
Dim Md5 As New MD5CryptoServiceProvider()
Dim ByteHash() As Byte = Md5.ComputeHash(Buff)
Return Convert.ToBase64String(ByteHash)
End Function

#Region " Windows Form Designer generated code "
' snip
#End Region

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As
System.EventArgs) Handles Button1.Click
fs.Seek(0, SeekOrigin.Begin)
Label1.Text = GenerateHash(r.ReadBytes(fs.Length))
End Sub
End Class

--
Richard
Please reply to group or
Remove 'REMOVE' from my email address.
Nov 20 '05 #1
2 2886
If you are familiar with Block cipher encryption in .NET, hashes are
supposed to work the same way.
In other words, hash algorithms (like MD5) implement the ICryptoTransform
interface, which allows them to be used by the CryptoStream object.
You could create a memory stream, and wrap it with a CryptoStream, and pass
the MD5 instance to the CryptoStream constructor. Anytime you "write" to the
CryptoStream, it should transform the data through the protecte HashCore
function of MD5. Then, when you are done, you call FlushFinalBlock on the
CryptoStream, which in turn calls the MD5 HashFinal method. Then you can
check the Hash property of the md5 object for the final hash value.

Function GetHash() As String

Dim cs As CryptoStream
Dim ms As MemoryStream = New MemoryStream
Dim md5Hash As MD5CryptoServiceProvider = New MD5CryptoServiceProvider

Try

Dim buffer As Byte() = New Byte() {1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
11, 12, 13, 14, 15}

cs = New CryptoStream(ms, md5Hash, CryptoStreamMode.Write)
cs.Write(buffer, 0, buffer.Length)

cs.FlushFinalBlock()
Return Convert.ToBase64String(md5Hash.Hash())

Catch ex As Exception

MsgBox("Error during hash operation: " + ex.ToString())

Finally
If Not (cs Is Nothing) Then cs.Close()
If Not (md5Hash Is Nothing) Then md5Hash.Clear()
End Try
End Function

The only thing to note is that in my sample, i just filled the buffer once
with a bunch of numbers.
In your case, you need to wrap a Loop around the cs.Write(...) line, and
read from your file to the buffer one chunk at a time, and write the buffer
to the CryptoStream until you run out of file.

-Rob Teixeira [MVP]

"Richard Lemay" <rl*********************@mailblocks.com> wrote in message
news:ul**************@TK2MSFTNGP11.phx.gbl...
I am new to VB.NET and I am trying to learn. So, your indulgence for the
triviality of my questions is kindly requested.

I would like to calculate an MD5 hash for very lage files. The examples I
came across read the file into a byte array and apply the hash to that
array. The following code illustrates what I am doing. I would like to
perform the hash calculation on a stream. Is this possible? If so, an
example would be greatly appreciated.

Imports System.Text
Imports System.Security.Cryptography
Imports System.IO

Public Class Form1
Inherits System.Windows.Forms.Form
Dim fs As FileStream = New FileStream("c:\ConfDenise.PDF",
FileMode.Open)
Dim r As BinaryReader = New BinaryReader(fs)

Public Function GenerateHash(ByRef Buff() As Byte) As String
Dim Md5 As New MD5CryptoServiceProvider()
Dim ByteHash() As Byte = Md5.ComputeHash(Buff)
Return Convert.ToBase64String(ByteHash)
End Function

#Region " Windows Form Designer generated code "
' snip
#End Region

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As
System.EventArgs) Handles Button1.Click
fs.Seek(0, SeekOrigin.Begin)
Label1.Text = GenerateHash(r.ReadBytes(fs.Length))
End Sub
End Class

--
Richard
Please reply to group or
Remove 'REMOVE' from my email address.

Nov 20 '05 #2
Thanks Rob,
I'll give it a try.
Richard

"Rob Teixeira [MVP]" <RobTeixeira@@msn.com> wrote in message
news:%2****************@tk2msftngp13.phx.gbl...
If you are familiar with Block cipher encryption in .NET, hashes are
supposed to work the same way.
In other words, hash algorithms (like MD5) implement the ICryptoTransform
interface, which allows them to be used by the CryptoStream object.
You could create a memory stream, and wrap it with a CryptoStream, and pass the MD5 instance to the CryptoStream constructor. Anytime you "write" to the CryptoStream, it should transform the data through the protecte HashCore
function of MD5. Then, when you are done, you call FlushFinalBlock on the
CryptoStream, which in turn calls the MD5 HashFinal method. Then you can
check the Hash property of the md5 object for the final hash value.

Function GetHash() As String

Dim cs As CryptoStream
Dim ms As MemoryStream = New MemoryStream
Dim md5Hash As MD5CryptoServiceProvider = New MD5CryptoServiceProvider

Try

Dim buffer As Byte() = New Byte() {1, 2, 3, 4, 5, 6, 7, 8, 9, 10,
11, 12, 13, 14, 15}

cs = New CryptoStream(ms, md5Hash, CryptoStreamMode.Write)
cs.Write(buffer, 0, buffer.Length)

cs.FlushFinalBlock()
Return Convert.ToBase64String(md5Hash.Hash())

Catch ex As Exception

MsgBox("Error during hash operation: " + ex.ToString())

Finally
If Not (cs Is Nothing) Then cs.Close()
If Not (md5Hash Is Nothing) Then md5Hash.Clear()
End Try
End Function

The only thing to note is that in my sample, i just filled the buffer once
with a bunch of numbers.
In your case, you need to wrap a Loop around the cs.Write(...) line, and
read from your file to the buffer one chunk at a time, and write the buffer to the CryptoStream until you run out of file.

-Rob Teixeira [MVP]

"Richard Lemay" <rl*********************@mailblocks.com> wrote in message
news:ul**************@TK2MSFTNGP11.phx.gbl...
I am new to VB.NET and I am trying to learn. So, your indulgence for the triviality of my questions is kindly requested.

I would like to calculate an MD5 hash for very lage files. The examples I came across read the file into a byte array and apply the hash to that
array. The following code illustrates what I am doing. I would like to
perform the hash calculation on a stream. Is this possible? If so, an
example would be greatly appreciated.

Imports System.Text
Imports System.Security.Cryptography
Imports System.IO

Public Class Form1
Inherits System.Windows.Forms.Form
Dim fs As FileStream = New FileStream("c:\ConfDenise.PDF",
FileMode.Open)
Dim r As BinaryReader = New BinaryReader(fs)

Public Function GenerateHash(ByRef Buff() As Byte) As String
Dim Md5 As New MD5CryptoServiceProvider()
Dim ByteHash() As Byte = Md5.ComputeHash(Buff)
Return Convert.ToBase64String(ByteHash)
End Function

#Region " Windows Form Designer generated code "
' snip
#End Region

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As
System.EventArgs) Handles Button1.Click
fs.Seek(0, SeekOrigin.Begin)
Label1.Text = GenerateHash(r.ReadBytes(fs.Length))
End Sub
End Class

--
Richard
Please reply to group or
Remove 'REMOVE' from my email address.


Nov 20 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Edvard Majakari | last post by:
Hi all ya unit-testing experts there :) Code I'm working on has to parse large and complex files and detect equally complex and large amount of errors before the contents of the file is fed to...
6
by: Greg | last post by:
I am working on a project that will have about 500,000 records in an XML document. This document will need to be queried with XPath, and records will need to be updated. I was thinking about...
3
by: Buddy Ackerman | last post by:
I'm trying to write files directly to the client so that it forces the client to open the Save As dialog box rather than display the file. On some occasions the files are very large (100MB+). On...
3
by: A.M-SG | last post by:
Hi, I have a ASP.NET aspx file that needs to pass large images from a network storage to client browser. The requirement is that users cannot have access to the network share. The aspx file...
2
by: jdev8080 | last post by:
We are looking at creating large XML files containing binary data (encoded as base64) and passing them to transformers that will parse and transform the data into different formats. Basically,...
20
by: mike | last post by:
I help manage a large web site, one that has over 600 html pages... It's a reference site for ham radio folks and as an example, one page indexes over 1.8 gb of on-line PDF documents. The site...
1
by: Lars B | last post by:
Hey guys, I have written a C++ program that passes data from a file to an FPGA board and back again using software and DMA buffers. In my program I need to compare the size of a given file against...
8
by: theCancerus | last post by:
Hi All, I am not sure if this is the right place to ask this question but i am very sure you may have faced this problem, i have already found some post related to this but not the answer i am...
1
by: =?Utf-8?B?UVNJRGV2ZWxvcGVy?= | last post by:
Using .NET 2.0 is it more efficient to copy files to a single folder versus spreading them across multiple folders. For instance if we have 100,000 files to be copied, Do we copy all of them to...
17
by: byte8bits | last post by:
How does C++ safely open and read very large files? For example, say I have 1GB of physical memory and I open a 4GB file and attempt to read it like so: #include <iostream> #include <fstream>...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.