473,473 Members | 1,546 Online
Bytes | Software Development & Data Engineering Community
Create Post

Home Posts Topics Members FAQ

Why is its substantialy slower to load 50GB of gzipped file (20GB gzipped file) then loading 50GB unzipped data? im using System.IO.Compression.GZipStream and its not maxing out the cpu while loading the gzip data! Im using the default buffer of the

DR
Why is its substantialy slower to load 50GB of gzipped file (20GB gzipped
file) then loading 50GB unzipped data? im using
System.IO.Compression.GZipStream and its not maxing out the cpu while
loading the gzip data! Im using the default buffer of the stream that i open
on the 20GB gzipped file and pass it into the GZipStream ctor. then
System.IO.Compression.GZipStream takes an hour! when just loading 50GB file
of data takes a few minutes!
Sep 4 '08 #1
3 1081
DR,

What do you mean by load, I am not aware of a Load method in standard .Net?

I know IO.File and IO.Streamreader to get files?

Cor

"DR" <so*******************@yahoo.comschreef in bericht
news:%2****************@TK2MSFTNGP02.phx.gbl...
Why is its substantialy slower to load 50GB of gzipped file (20GB gzipped
file) then loading 50GB unzipped data? im using
System.IO.Compression.GZipStream and its not maxing out the cpu while
loading the gzip data! Im using the default buffer of the stream that i
open on the 20GB gzipped file and pass it into the GZipStream ctor. then
System.IO.Compression.GZipStream takes an hour! when just loading 50GB
file of data takes a few minutes!
Sep 5 '08 #2
On Sep 4, 5:54 pm, "DR" <softwareengineer98...@yahoo.comwrote:
Why is its substantialy slower to load 50GB of gzipped file (20GB gzipped
file) then loading 50GB unzipped data? im using
System.IO.Compression.GZipStream and its not maxing out the cpu while
loading the gzip data! Im using the default buffer of the stream that i open
on the 20GB gzipped file and pass it into the GZipStream ctor. then
System.IO.Compression.GZipStream takes an hour! when just loading 50GB file
of data takes a few minutes!
DR PLEASE!!!! PLEASE!!! STOP putting your entire message into the
subject line!!! Put only a short description of your problem there.

Chris
Sep 5 '08 #3

"Chris Dunaway" <du******@gmail.comwrote in message
news:e2**********************************@m36g2000 hse.googlegroups.com...
On Sep 4, 5:54 pm, "DR" <softwareengineer98...@yahoo.comwrote:
>Why is its substantialy slower to load 50GB of gzipped file (20GB gzipped
file) then loading 50GB unzipped data? im using
System.IO.Compression.GZipStream and its not maxing out the cpu while
loading the gzip data! Im using the default buffer of the stream that i
open
on the 20GB gzipped file and pass it into the GZipStream ctor. then
System.IO.Compression.GZipStream takes an hour! when just loading 50GB
file
of data takes a few minutes!

DR PLEASE!!!! PLEASE!!! STOP putting your entire message into the
subject line!!! Put only a short description of your problem there.

Chris
I don't gzipp files I zip files.
Sep 5 '08 #4
Create Post

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics
4
trouble measuring filesize of gzipped data
by: Marek Möhling | last post by:
My server (Apache/1.3.28 - PHP/4.3.3) is configured to receive gzipped data via: Header append Accept-Encoding "gzip, deflate" PHP is configured to send gzipped data via: php_value...
Latest Bytes
8
Add a file to a compressed tarfile
by: Dennis Hotson | last post by:
Hi, I'm trying to write a function that adds a file-like-object to a compressed tarfile... eg ".tar.gz" or ".tar.bz2" I've had a look at the tarfile module but the append mode doesn't support...
Latest Bytes
3
Read a gzip file from inside a tar file
by: rohisingh | last post by:
I have a tar file. The content of the file are as following. rohits@sandman 12-08-04 $ tar tvf 20041208.tar drwxr-xr-x root/root 0 2004-12-08 21:39:19 20041208/ -rw-r--r-- root/root ...
Latest Bytes
12
Large image load error that do work in C++
by: Sharon | last post by:
I’m wrote a small DLL that used the FreeImage.DLL (that can be found at http://www.codeproject.com/bitmap/graphicsuite.asp). I also wrote a small console application in C++ (unmanaged) that uses...
Latest Bytes
9
Zipping a file programmatically in vs2005
by: ReidarT | last post by:
How can I zip (compress) a file in vs2005 with windows own zip-program? reidarT
Latest Bytes
6
Javascript file compactor wanted
by: Aaron Gray | last post by:
Hi, Is there any tool that will concatinate JavaScript files compacting them spacewise and removing comment fields ? I have some sed script that sort of does the job but was wondering if there...
Latest Bytes
1
Can sqlite read gzipped databases?
by: Paul Smith | last post by:
Hi, I'd like to read a series of sqlite database files that have already been gzipped and was wondering if this can be done on the fly. In other words, can I avoid explicitly unzipping the file...
Latest Bytes
13
How to unzip a file ?
by: fniles | last post by:
In VB.NET 2003 how can you programmatically unzip a ZIP file ? Thank you.
Latest Bytes
1
How to read gzipped utf8 file in Python?
by: John Nagle | last post by:
I have a large (gigabytes) file which is encoded in UTF-8 and then compressed with gzip. I'd like to read it with the "gzip" module and "utf8" decoding. The obvious approach is fd =...
Latest Bytes
2
reg zip file
by: =?Utf-8?B?QVZM?= | last post by:
Hi I've a requirement wherein I need to unzip a zipped folder programatically in .net. I've tried out System.IO.Compression namespace..but it writes the content into bytes which I dont need..I...
Latest Bytes
0
How to build RAID in BIOS?
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
Latest Bytes
0
marktang
What is ONU?
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
Latest Bytes
0
Oralloy
Problem With Comparison Operator <=> in G++
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
Latest Bytes
0
jinu1996
Maximizing Business Potential: The Nexus of Website Design and Digital Marketing
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
Latest Bytes
0
tracyyun
Discussion: How does Zigbee compare with other wireless protocols in smart home applications?
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
Latest Bytes
1
isladogs
Access Europe - Using VBA to create a class based on a table - Wed 1 May
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...
Latest Bytes
0
Windows Forms - .Net 8.0
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
Latest Bytes
0
transfer the data from one system to another through ip address
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated ...
Latest Bytes
0
muto222
php
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
Latest Bytes

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.

BYTES.COM © 2024
About Bytes
Terms Of Use
Privacy Policy
Sitemap
Advertise on Bytes
How to Post and Respond on Bytes
How to Promote and Link on Bytes
How to increase your Ranking on Bytes
Become a Recognized Expert on Bytes
Feedback Welcomed! Contact Us