By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
434,894 Members | 2,002 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 434,894 IT Pros & Developers. It's quick & easy.

How can I zip a 8 Gb file using c# ?

P: n/a
I need to zip up an 8Gb file using C# - do you know of any freeware libraries
I can use to accomplish this? I guess it needs to be Zip64 to support such
file sizes.

I tried something called sharpziplib but it doesn't support >2Gb files -
they are working on it however but I cannot wait...
Oct 2 '06 #1
Share this Question
Share on Google+
11 Replies


P: n/a
Hello, MrNobody!

MI need to zip up an 8Gb file using C# - do you know of any freeware
Mlibraries
MI can use to accomplish this? I guess it needs to be Zip64 to support
Msuch
Mfile sizes.

MI tried something called sharpziplib but it doesn't support >2Gb
Mfiles -
Mthey are working on it however but I cannot wait...

Did you try GZipStream?
( http://msdn2.microsoft.com/en-us/lib...zipstream.aspx )
You must be using .NET 2.0 to work with it.

--
Regards, Vadym Stetsyak
www: http://vadmyst.blogspot.com
Oct 2 '06 #2

P: n/a


"Vadym Stetsyak" wrote:
Hello, MrNobody!

MI need to zip up an 8Gb file using C# - do you know of any freeware
Mlibraries
MI can use to accomplish this? I guess it needs to be Zip64 to support
Msuch
Mfile sizes.

MI tried something called sharpziplib but it doesn't support >2Gb
Mfiles -
Mthey are working on it however but I cannot wait...

Did you try GZipStream?
( http://msdn2.microsoft.com/en-us/lib...zipstream.aspx )
You must be using .NET 2.0 to work with it.

--
Regards, Vadym Stetsyak
www: http://vadmyst.blogspot
Looks like GZip wouldn't work either, from the 'Remarks' :

"This class cannot be used to compress files larger than 4 GB."
Oct 2 '06 #3

P: n/a
MrNobody <Mr******@discussions.microsoft.comwrote:
>

"Vadym Stetsyak" wrote:
>Hello, MrNobody!

MI need to zip up an 8Gb file using C# - do you know of any freeware
Mlibraries
MI can use to accomplish this? I guess it needs to be Zip64 to support
Msuch
Mfile sizes.

MI tried something called sharpziplib but it doesn't support >2Gb
Mfiles -
Mthey are working on it however but I cannot wait...

Did you try GZipStream?
( http://msdn2.microsoft.com/en-us/lib...zipstream.aspx )
You must be using .NET 2.0 to work with it.

--
Regards, Vadym Stetsyak
www: http://vadmyst.blogspot

Looks like GZip wouldn't work either, from the 'Remarks' :

"This class cannot be used to compress files larger than 4 GB."
4GB is the maximum addressible size for a 32-bit pointer. Also, GZIP is not
the same as ZIP, so if you need PKZIP compatibility, then you need to get
another library.

--
Thomas T. Veldhouse
Key Fingerprint: 2DB9 813F F510 82C2 E1AE 34D0 D69D 1EDC D5EC AED1
Oct 2 '06 #4

P: n/a

"MrNobody" <Mr******@discussions.microsoft.comwrote in message
news:E0**********************************@microsof t.com...
|
|
| "Vadym Stetsyak" wrote:
|
| Hello, MrNobody!
| >
| MI need to zip up an 8Gb file using C# - do you know of any freeware
| Mlibraries
| MI can use to accomplish this? I guess it needs to be Zip64 to support
| Msuch
| Mfile sizes.
| >
| MI tried something called sharpziplib but it doesn't support >2Gb
| Mfiles -
| Mthey are working on it however but I cannot wait...
| >
| Did you try GZipStream?
| (
http://msdn2.microsoft.com/en-us/lib...zipstream.aspx )
| You must be using .NET 2.0 to work with it.
| >
| --
| Regards, Vadym Stetsyak
| www: http://vadmyst.blogspot
|
| Looks like GZip wouldn't work either, from the 'Remarks' :
|
| "This class cannot be used to compress files larger than 4 GB."

Which is wrong too, actually the limit is 2GB, that's the max.size of an
object in .NET and the GZIP library needs the complete file to be read in a
single byte array (which obviously is limitted to 2GB).

Willy.
Oct 2 '06 #5

P: n/a
OK guys, found someone who made a modification to the sharpziplib out there
to support Zip64 and it works. Compressed my 7.6 Gb file and then used Winzip
to uncompress it. Works good !

thanks for your time trying to help me :)
Oct 2 '06 #6

P: n/a
Would you like to share the name/web address of "someone". Might be useful
to know.

Steve
"MrNobody" <Mr******@discussions.microsoft.comwrote in message
news:D2**********************************@microsof t.com...
OK guys, found someone who made a modification to the sharpziplib out
there
to support Zip64 and it works. Compressed my 7.6 Gb file and then used
Winzip
to uncompress it. Works good !

thanks for your time trying to help me :)

Oct 3 '06 #7

P: n/a
http://www.icsharpcode.net/OpenSourc.../Download.aspx

Nirosh

"Steve Barnett" <no****@nodomain.comwrote in message
news:u4**************@TK2MSFTNGP02.phx.gbl...
Would you like to share the name/web address of "someone". Might be useful
to know.

Steve
"MrNobody" <Mr******@discussions.microsoft.comwrote in message
news:D2**********************************@microsof t.com...
>OK guys, found someone who made a modification to the sharpziplib out
there
to support Zip64 and it works. Compressed my 7.6 Gb file and then used
Winzip
to uncompress it. Works good !

thanks for your time trying to help me :)


Oct 3 '06 #8

P: n/a
Isn't that the original source. I was interested in the Zip64 variant.

Steve

"Champika Nirosh" <te**@tc.comwrote in message
news:uW**************@TK2MSFTNGP06.phx.gbl...
http://www.icsharpcode.net/OpenSourc.../Download.aspx

Nirosh

"Steve Barnett" <no****@nodomain.comwrote in message
news:u4**************@TK2MSFTNGP02.phx.gbl...
>Would you like to share the name/web address of "someone". Might be
useful to know.

Steve
"MrNobody" <Mr******@discussions.microsoft.comwrote in message
news:D2**********************************@microso ft.com...
>>OK guys, found someone who made a modification to the sharpziplib out
there
to support Zip64 and it works. Compressed my 7.6 Gb file and then used
Winzip
to uncompress it. Works good !

thanks for your time trying to help me :)



Oct 3 '06 #9

P: n/a
Willy Denoyette [MVP] <wi*************@telenet.bewrote:
>
"MrNobody" <Mr******@discussions.microsoft.comwrote in message
news:E0**********************************@microsof t.com...
|
|
| "Vadym Stetsyak" wrote:
|
| Hello, MrNobody!
| >
| MI need to zip up an 8Gb file using C# - do you know of any freeware
| Mlibraries
| MI can use to accomplish this? I guess it needs to be Zip64 to support
| Msuch
| Mfile sizes.
| >
| MI tried something called sharpziplib but it doesn't support >2Gb
| Mfiles -
| Mthey are working on it however but I cannot wait...
| >
| Did you try GZipStream?
| (
http://msdn2.microsoft.com/en-us/lib...zipstream.aspx )
| You must be using .NET 2.0 to work with it.
| >
| --
| Regards, Vadym Stetsyak
| www: http://vadmyst.blogspot
|
| Looks like GZip wouldn't work either, from the 'Remarks' :
|
| "This class cannot be used to compress files larger than 4 GB."

Which is wrong too, actually the limit is 2GB, that's the max.size of an
object in .NET and the GZIP library needs the complete file to be read in a
single byte array (which obviously is limitted to 2GB).
Why does GZIP require the entire file to be in memory first? It is a sliding
window algorithm similar to LZW. In any case, implementation is via a
GZIPStream, which doesn't seem to require the entire file be loaded into
memory first, so I believe 4GB is indeed the limit (which is the limit of
addressable memory with a 32-bit pointer).

--
Thomas T. Veldhouse
Key Fingerprint: 2DB9 813F F510 82C2 E1AE 34D0 D69D 1EDC D5EC AED1
Oct 3 '06 #10

P: n/a

"Thomas T. Veldhouse" <ve*****@yahoo.comwrote in message
news:i4****************@textfe.usenetserver.com...
| Willy Denoyette [MVP] <wi*************@telenet.bewrote:
| >
| "MrNobody" <Mr******@discussions.microsoft.comwrote in message
| news:E0**********************************@microsof t.com...
| |
| |
| | "Vadym Stetsyak" wrote:
| |
| | Hello, MrNobody!
| | >
| | MI need to zip up an 8Gb file using C# - do you know of any
freeware
| | Mlibraries
| | MI can use to accomplish this? I guess it needs to be Zip64 to
support
| | Msuch
| | Mfile sizes.
| | >
| | MI tried something called sharpziplib but it doesn't support >2Gb
| | Mfiles -
| | Mthey are working on it however but I cannot wait...
| | >
| | Did you try GZipStream?
| | (
| >
http://msdn2.microsoft.com/en-us/lib...zipstream.aspx )
| | You must be using .NET 2.0 to work with it.
| | >
| | --
| | Regards, Vadym Stetsyak
| | www: http://vadmyst.blogspot
| |
| | Looks like GZip wouldn't work either, from the 'Remarks' :
| |
| | "This class cannot be used to compress files larger than 4 GB."
| >
| Which is wrong too, actually the limit is 2GB, that's the max.size of an
| object in .NET and the GZIP library needs the complete file to be read
in a
| single byte array (which obviously is limitted to 2GB).
| >
|
| Why does GZIP require the entire file to be in memory first? It is a
sliding
| window algorithm similar to LZW. In any case, implementation is via a
| GZIPStream, which doesn't seem to require the entire file be loaded into
| memory first, so I believe 4GB is indeed the limit (which is the limit of
| addressable memory with a 32-bit pointer).
|
The limit is indeed 4GB, but it has nothing to do with the addressable
memory, this comes from the gzip RFC which requires the input file size to
be less than 4GB.
The input size is written to the compressed file as an unsigned int (module
2^32), the GZipStream implementation keeps an internal count of the number
of bytes read from the input stream and throws:
"System.IO.InvalidDataException: The gzip stream can't contain more than 4GB
data." when the number of bytes read exeeds 2^32.

Willy.


Oct 4 '06 #11

P: n/a
Willy Denoyette [MVP] <wi*************@telenet.bewrote:
|
| Why does GZIP require the entire file to be in memory first? It is a
sliding
| window algorithm similar to LZW. In any case, implementation is via a
| GZIPStream, which doesn't seem to require the entire file be loaded into
| memory first, so I believe 4GB is indeed the limit (which is the limit of
| addressable memory with a 32-bit pointer).
|
The limit is indeed 4GB, but it has nothing to do with the addressable
memory, this comes from the gzip RFC which requires the input file size to
be less than 4GB.
The input size is written to the compressed file as an unsigned int (module
2^32), the GZipStream implementation keeps an internal count of the number
of bytes read from the input stream and throws:
"System.IO.InvalidDataException: The gzip stream can't contain more than 4GB
data." when the number of bytes read exeeds 2^32.
Agreed. I shouldn't have limitted it to addressable memory and instead
limitted it to the resolution of a 32-bit WORD. GZIP was designed originally
for 32-bit processing. The same reason that a 32-bit system is limitted to
4GB addressable memory is the reason that GZIP is limitted to 4GB file sizes.

As I indicated before, there is no reason that one can not compress a file
using GZIPStream as long as it is less than 4GB in length (it doesn't have to
be loaded into memory to compress using the stream).

--
Thomas T. Veldhouse
Key Fingerprint: 2DB9 813F F510 82C2 E1AE 34D0 D69D 1EDC D5EC AED1
Oct 4 '06 #12

This discussion thread is closed

Replies have been disabled for this discussion.