469,904 Members | 2,048 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,904 developers. It's quick & easy.

GZipStream

DR
Why is its substantialy slower to load 50GB of gzipped file (20GB gzipped
file) then loading 50GB unzipped data? im using
System.IO.Compression.GZipStream and its not maxing out the cpu while
loading the gzip data! Im using the default buffer of the stream that i open
on the 20GB gzipped file and pass it into the GZipStream ctor. then
System.IO.Compression.GZipStream takes an hour! when just loading 50GB file
of data takes a few minutes!
Sep 4 '08 #1
2 2663
"DR" <so*******************@yahoo.comwrote in message
news:ek**************@TK2MSFTNGP02.phx.gbl...
Why is its substantialy slower to load 50GB of gzipped file (20GB gzipped
file) then loading 50GB unzipped data? im using
System.IO.Compression.GZipStream and its not maxing out the cpu while
loading the gzip data! Im using the default buffer of the stream that i
open on the 20GB gzipped file and pass it into the GZipStream ctor. then
System.IO.Compression.GZipStream takes an hour! when just loading 50GB
file of data takes a few minutes!
GZipStream is broken in many ways. To list a few:

https://connect.microsoft.com/Visual...edbackID=93636
https://connect.microsoft.com/Visual...edbackID=94784

My suggestion is to vote on those on Microsoft Connect in hope they will get
fixed eventually, and use SharpZipLib meanwhile.
Sep 5 '08 #2

"DR" wrote:
Why is its substantialy slower to load 50GB of gzipped file (20GB gzipped
file) then loading 50GB unzipped data? im using
System.IO.Compression.GZipStream and its not maxing out the cpu while
loading the gzip data! Im using the default buffer of the stream that i open
on the 20GB gzipped file and pass it into the GZipStream ctor. then
System.IO.Compression.GZipStream takes an hour! when just loading 50GB file
of data takes a few minutes!
Because loading compressed data is more complex than loading uncompressed
data. I don't know the internals of the zip algorithm, but I'm guessing the
amount of IO reading prevents choking the CPU.
--
Happy Coding!
Morten Wennevik [C# MVP]
Sep 5 '08 #3

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

reply views Thread by Craig Buchanan | last post: by
1 post views Thread by Robinson | last post: by
1 post views Thread by hakan.thornqvist | last post: by
2 posts views Thread by =?Utf-8?B?TWFydGluIE1hZHJlemE=?= | last post: by
1 post views Thread by Waqarahmed | last post: by
reply views Thread by Salome Sato | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.