473,386 Members | 1,832 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,386 software developers and data experts.

File compression in .NET - for very large files

Hi

I want to compress and decomress files. There is GZipStream class
in .Net. However this class requires to allocate buffer equal to the
file length to zip it (at least all sample code I saw were like that).
The file size in my case can be really very large and hence I want
something which does not require to load entire file in memory. Is
there any way?

Thanks and best regards
Amit Dedhia

Jun 21 '07 #1
3 4536
"Amit Dedhia" <am********@yahoo.comwrote in message
news:11*********************@i38g2000prf.googlegro ups.com...
Is there any way?
http://www.chilkatsoft.com/zip-dotnet.asp
--
http://www.markrae.net

Jun 21 '07 #2
On Jun 21, 8:12 am, Amit Dedhia <amitded...@yahoo.comwrote:
I want to compress and decomress files. There is GZipStream class
in .Net. However this class requires to allocate buffer equal to the
file length to zip it (at least all sample code I saw were like that).
The file size in my case can be really very large and hence I want
something which does not require to load entire file in memory. Is
there any way?
You don't need to read the whole file in. Just copy the data from the
input stream to the GZipStream a chunk (eg 32K) at a time.

I've got a utility class which makes this easier:
http://pobox.com/~skeet/csharp/miscutil
and
http://pobox.com/~skeet/csharp/miscu...treamutil.html
in particular.

Jon

Jun 21 '07 #3
On Jun 21, 8:20 am, "Mark Rae" <m...@markNOSPAMrae.netwrote:
Is there any way?

http://www.chilkatsoft.com/zip-dotnet.asp
There's no need to go into commercial software here - the only problem
for the OP is that all the samples read the whole of the data into
memory before pumping any of it through the output stream. That's easy
to fix and worth knowing for other situations too :)

Jon

Jun 21 '07 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Alexander Gilman Carver | last post by:
I have written a pair of scripts that are supposed to work together to display an index of files and then, upon the user choosing the files (with checkboxes on an HTML form submitted to itself),...
10
by: Dennis Farr | last post by:
It has been suggested that rather than convert an already large flat file, with many similar rows, to XML, some type of header be attached to the file, containing some sort of meta-XML description...
1
by: Phil Price | last post by:
Hi there, I'm developing a shape recognition application for the tablet PC for a) fun b) university project. Currently I'm working on the learning stage using neural networks, and have to store...
7
by: Kilian A. Foth | last post by:
I just found this amazing video puzzle game written with the pygame library, which promises to be infinite fun - but I can't get it to decode any video file I own, except the game's own example...
22
by: petermichaux | last post by:
Hi, I'm curious about server load and download time if I use one big javascript file or break it into several smaller ones. Which is better? (Please think of this as the first time the scripts...
52
by: paytam | last post by:
Hi all Can anyone tell me how can I check that a file exist or no.I mean when you use this commands FILE *fp; if(!fp) //Could not open the file doen't show why it can not open it,may be the...
16
by: Claudio Grondi | last post by:
I have a 250 Gbyte file (occupies the whole hard drive space) and want to change only eight bytes in this file at a given offset of appr. 200 Gbyte (all other data in that file should remain...
1
by: laredotornado | last post by:
Hi, I'm using PHP 4.4.4 on Apache 2 on Fedora Core 5. PHP was installed using Apache's apxs and the php library was installed to /usr/local/php. However, when I set my "error_reporting"...
3
by: Amit Dedhia | last post by:
Hi I want to compress and decomress files. There is GZipStream class in .Net. However this class requires to allocate buffer equal to the file length to zip it (at least all sample code I saw...
3
by: Sean Davis | last post by:
I have a set of numpy arrays which I would like to save to a gzip file. Here is an example without gzip: b=numpy.ones(1000000,dtype=numpy.uint8) a=numpy.zeros(1000000,dtype=numpy.uint8) fd =...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.