How can I zip a 8 Gb file using c# ?

[MrNobody]

I need to zip up an 8Gb file using C# - do you know of any freeware libraries
I can use to accomplish this? I guess it needs to be Zip64 to support such
file sizes.

I tried something called sharpziplib but it doesn't support >2Gb files -
they are working on it however but I cannot wait...

[Willy Denoyette [MVP]]

"Thomas T. Veldhouse" <ve*****@yahoo. comwrote in message
news:i4******** ********@textfe .usenetserver.c om...
| Willy Denoyette [MVP] <wi************ *@telenet.bewro te:
| >
| "MrNobody" <Mr******@discu ssions.microsof t.comwrote in message
| news:E0******** *************** ***********@mic rosoft.com...
| |
| |
| | "Vadym Stetsyak" wrote:
| |
| | Hello, MrNobody!
| | >
| | MI need to zip up an 8Gb file using C# - do you know of any
freeware
| | Mlibraries
| | MI can use to accomplish this? I guess it needs to be Zip64 to
support
| | Msuch
| | Mfile sizes.
| | >
| | MI tried something called sharpziplib but it doesn't support >2Gb
| | Mfiles -
| | Mthey are working on it however but I cannot wait...
| | >
| | Did you try GZipStream?
| | (
| >
http://msdn2.microsoft.com/en-us/lib...zipstream.aspx )
| | You must be using .NET 2.0 to work with it.
| | >
| | --
| | Regards, Vadym Stetsyak
| | www: http://vadmyst.blogspot
| |
| | Looks like GZip wouldn't work either, from the 'Remarks' :
| |
| | "This class cannot be used to compress files larger than 4 GB."
| >
| Which is wrong too, actually the limit is 2GB, that's the max.size of an
| object in .NET and the GZIP library needs the complete file to be read
in a
| single byte array (which obviously is limitted to 2GB).
| >
|
| Why does GZIP require the entire file to be in memory first? It is a
sliding
| window algorithm similar to LZW. In any case, implementation is via a
| GZIPStream, which doesn't seem to require the entire file be loaded into
| memory first, so I believe 4GB is indeed the limit (which is the limit of
| addressable memory with a 32-bit pointer).
|
The limit is indeed 4GB, but it has nothing to do with the addressable
memory, this comes from the gzip RFC which requires the input file size to
be less than 4GB.
The input size is written to the compressed file as an unsigned int (module
2^32), the GZipStream implementation keeps an internal count of the number
of bytes read from the input stream and throws:
"System.IO.Inva lidDataExceptio n: The gzip stream can't contain more than 4GB
data." when the number of bytes read exeeds 2^32.

Willy.

[Thomas T. Veldhouse]

Willy Denoyette [MVP] <wi************ *@telenet.bewro te:

|
| Why does GZIP require the entire file to be in memory first? It is a
sliding
| window algorithm similar to LZW. In any case, implementation is via a
| GZIPStream, which doesn't seem to require the entire file be loaded into
| memory first, so I believe 4GB is indeed the limit (which is the limit of
| addressable memory with a 32-bit pointer).
|
The limit is indeed 4GB, but it has nothing to do with the addressable
memory, this comes from the gzip RFC which requires the input file size to
be less than 4GB.
The input size is written to the compressed file as an unsigned int (module
2^32), the GZipStream implementation keeps an internal count of the number
of bytes read from the input stream and throws:
"System.IO.Inva lidDataExceptio n: The gzip stream can't contain more than 4GB
data." when the number of bytes read exeeds 2^32.

Agreed. I shouldn't have limitted it to addressable memory and instead
limitted it to the resolution of a 32-bit WORD. GZIP was designed originally
for 32-bit processing. The same reason that a 32-bit system is limitted to
4GB addressable memory is the reason that GZIP is limitted to 4GB file sizes.

As I indicated before, there is no reason that one can not compress a file
using GZIPStream as long as it is less than 4GB in length (it doesn't have to
be loaded into memory to compress using the stream).

--
Thomas T. Veldhouse
Key Fingerprint: 2DB9 813F F510 82C2 E1AE 34D0 D69D 1EDC D5EC AED1