471,594 Members | 1,982 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,594 software developers and data experts.

SHA1 Hash question with large Files



Hi all,

I guess I don't fully understand how a SHA1 hash value is calculated
in C# / .NET for a large file... I'm trying to calculate SHA1 values
for large files that are much larger than my physical main memory. It
seems the way to derive a SHA1 value involves opening a file stream to
the large file, passing it to a byte array, and passing the byte array
to the .NET hash method.

Does this load the entire file into main memory (within the byte[] )
??? I see hash values for DVD isos all the time and feel that there
must be a way to derive a hash value via passing a file stream to the
hash method whereas very low main memory consumption occurs.

Any help or pointers to documentation that addresses this would be
super great.
Thanks,
Mike.
Nov 17 '05 #1
5 6922
Hi Michael,
SHA1 is a block cypher. Blocks of 512 bits are read in at a time and the
160bit hash is computed using the blocks, therefore it is not necesary to
read in the entire contents of the file into memory at once. The value is
continually computer from the individual blocks that are read in. you will
have to read inthe entire contents of hte file to produce the hash but there
is no need to have the entire file in memory at once to do this.

If you are interested in the internal workings the RFC is a good place to
look:
http://www.faqs.org/rfcs/rfc3174.html

Hope that helps
Mark R Dawson.

"Michael H" wrote:


Hi all,

I guess I don't fully understand how a SHA1 hash value is calculated
in C# / .NET for a large file... I'm trying to calculate SHA1 values
for large files that are much larger than my physical main memory. It
seems the way to derive a SHA1 value involves opening a file stream to
the large file, passing it to a byte array, and passing the byte array
to the .NET hash method.

Does this load the entire file into main memory (within the byte[] )
??? I see hash values for DVD isos all the time and feel that there
must be a way to derive a hash value via passing a file stream to the
hash method whereas very low main memory consumption occurs.

Any help or pointers to documentation that addresses this would be
super great.
Thanks,
Mike.

Nov 17 '05 #2
One of the overloads for ComputeHash() takes a Stream object. Just pass the
file stream to this and it will run over the file and compute the hash
without loading it all into memory. You could do the same thing yourself
for a for loop, but it is already there. HTH

--
William Stacey [MVP]

"Michael H" <gf*****@gmail.com> wrote in message
news:e1********************************@4ax.com...


Hi all,

I guess I don't fully understand how a SHA1 hash value is calculated
in C# / .NET for a large file... I'm trying to calculate SHA1 values
for large files that are much larger than my physical main memory. It
seems the way to derive a SHA1 value involves opening a file stream to
the large file, passing it to a byte array, and passing the byte array
to the .NET hash method.

Does this load the entire file into main memory (within the byte[] )
??? I see hash values for DVD isos all the time and feel that there
must be a way to derive a hash value via passing a file stream to the
hash method whereas very low main memory consumption occurs.

Any help or pointers to documentation that addresses this would be
super great.
Thanks,
Mike.

Nov 17 '05 #3
On Thu, 28 Jul 2005 01:22:18 -0400, "William Stacey [MVP]"
<st*****@mvps.org> wrote:

--> One of the overloads for ComputeHash() takes a Stream object.
Just pass the
--> file stream to this and it will run over the file and compute the
hash
--> without loading it all into memory. You could do the same thing
yourself
--> for a for loop, but it is already there. HTH
-->
--> --
--> William Stacey [MVP]

How about using a Stream object to calculate a hash value for a file
on a mobile device running a Windows OS / .NET CF1.0 ?
Nov 17 '05 #4
The docs say it is supported by Supported by the .NET Compact Framework. Is
this in error?

--
William Stacey [MVP]

"Michael H" <gf*****@gmail.com> wrote in message
news:0u********************************@4ax.com...
On Thu, 28 Jul 2005 01:22:18 -0400, "William Stacey [MVP]"
<st*****@mvps.org> wrote:

--> One of the overloads for ComputeHash() takes a Stream object.
Just pass the
--> file stream to this and it will run over the file and compute the
hash
--> without loading it all into memory. You could do the same thing
yourself
--> for a for loop, but it is already there. HTH
-->
--> --
--> William Stacey [MVP]

How about using a Stream object to calculate a hash value for a file
on a mobile device running a Windows OS / .NET CF1.0 ?

Nov 17 '05 #5
On Fri, 29 Jul 2005 00:30:21 -0400, "William Stacey [MVP]"
<st*****@mvps.org> wrote:

--> The docs say it is supported by Supported by the .NET Compact
Framework. Is
--> this in error?
-->
--> --
--> William Stacey [MVP]
We may be looking at different things? Here's what I'm looking at:

**************
namespace: System.Security.Cryptography

[C#]
public byte[] ComputeHash( Stream inputStream )
Parameters
inputStream - The input for which to compute the hash code.

Return Value
The computed hash code.
Requirements
Platforms: Windows 98, Windows NT 4.0, Windows Millennium Edition,
Windows 2000, Windows XP Home Edition, Windows XP Professional,
Windows Server 2003 family
****************

This isn't supported in CF1.0 on OpenNetCF1.3 right ?
Mike
Nov 17 '05 #6

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

3 posts views Thread by Randell D. | last post: by
reply views Thread by Paul Rubin | last post: by
6 posts views Thread by Chang | last post: by
reply views Thread by Dil via .NET 247 | last post: by
8 posts views Thread by Adam Carpenter | last post: by
3 posts views Thread by Paul Spielvogel | last post: by
8 posts views Thread by sathyashrayan | last post: by
6 posts views Thread by LaundroMat | last post: by
reply views Thread by XIAOLAOHU | last post: by
reply views Thread by leo001 | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.