473,241 Members | 1,664 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,241 software developers and data experts.

compression - dropping the last byte

It might just be my tired eyes, but I can't see what is wrong in the
following:

I have a byte array filled with random data (with a fixed seed to make
reproducable); I then compress and decompress this using the 2.0 GZip
compression classes - however, as you see, the last byte is getting dropped.

Any ideas? Almost certainly me having a blond moment ;-(

Marc

Overview:
GetByteData : returns an array of random data
Describe : details the first / last 5 values to the console (first: last,
second : penultimate, etc...)
CopyStream : reads / writes between two streams until the source is
exhausted (is there an wasier way of doing this?)
Compress / Decompress : use the GZip classes to manipulate the data

Results:

* 50000
0=170 49999=187
1=200 49998=183
2=81 49997=158
3=81 49996=216
4=66 49995=154

* 49999
0=170 49998=183
1=200 49997=158
2=81 49996=216
3=81 49995=154
4=66 49994=133
Code : watch for wrap
================

using System;
using System.IO;
using System.IO.Compression;
namespace ConsoleApplication1 {
static class Program {
public static void Main() {
byte[] data = GetByteData();
Describe(data);
Console.WriteLine();
byte[] compressed = Compress(data);
byte[] decompressed = Decompress(compressed);
Describe(decompressed);
Console.ReadLine();
}

private static void Describe(byte[] data) {
Console.WriteLine("* " + data.Length.ToString());
int length = data.Length;
for (int i = 0; i < 5; i++) {
int otherEnd = length - 1 - i;
Console.WriteLine(string.Format("{0}={1}\t{2}={3}" ,i,data[i],otherEnd,data[otherEnd]));
}
}
private static byte[] Compress(byte[] data) {
using (Stream input = new MemoryStream(data))
using (MemoryStream output = new MemoryStream())
using (Stream zipper = new GZipStream(output,
CompressionMode.Compress, true)) {
CopyStream(input, zipper);
return output.ToArray();
}
}
private static byte[] Decompress(byte[] data) {
using (Stream input = new MemoryStream(data))
using (Stream unzipper = new GZipStream(input,
CompressionMode.Decompress, false))
using (MemoryStream output = new MemoryStream()) {
CopyStream(unzipper, output);
return output.ToArray();
}
}

private const int SEED = 141566, SIZE = 50000;

public static byte[] GetByteData() {
Random rand = new Random(SEED); // some seed to make
reproducable
byte[] data = new byte[SIZE];
rand.NextBytes(data);
return data;

}

public static long CopyStream(System.IO.Stream source,
System.IO.Stream destination) {
const int BUFFER_SIZE = 512;
long copied = 0;
byte[] buffer = new byte[BUFFER_SIZE];
int bytes;
while ((bytes = source.Read(buffer, 0, BUFFER_SIZE)) > 0) {
destination.Write(buffer, 0, bytes);
copied += bytes;
}
return copied;
}
}
}

Feb 1 '06 #1
3 2584
Marc Gravell wrote:
It might just be my tired eyes, but I can't see what is wrong in the
following:

I have a byte array filled with random data (with a fixed seed to make
reproducable); I then compress and decompress this using the 2.0 GZip
compression classes - however, as you see, the last byte is getting dropped.

Any ideas? Almost certainly me having a blond moment ;-(


Move the return output.ToArray(); call *outside* the "using (Stream
zipper...)" block. That way the GZipStream is disposed of (and
therefore flushed and terminated appropriately) before you return the
data.

Making that change (which requires extra bracing) makes your sample
work.

Jon

Feb 1 '06 #2
A huge thanks to you Jon - that does indeed fix it. Quite a subtle one! Even
calling zipper.Flush() (just before ToArray) doesn't fix it...

I guess there's a cautionary lesson in there for all of use regarding
bracing etc : at a casual glance the following look functionally identical -
but one works and one doesn't... a bug that I think got introduced when
"tidying" the braces because they obviously [sic] do the same thing...

// works
using (Stream input = new MemoryStream(data))
using (MemoryStream output = new MemoryStream()) {
using (Stream zipper = new GZipStream(output, CompressionMode.Compress,
true)) {
CopyStream(input, zipper);
}
return output.ToArray();
}

and

// fails
using (Stream input = new MemoryStream(data))
using (MemoryStream output = new MemoryStream())
using (Stream zipper = new GZipStream(output, CompressionMode.Compress,
true)) {
CopyStream(input, zipper);
return output.ToArray();
}

Marc
Feb 1 '06 #3
one would think flush should have solved this. Is this by design?

--
William Stacey [MVP]

"Marc Gravell" <mg******@rm.com> wrote in message
news:%2****************@TK2MSFTNGP10.phx.gbl...
|A huge thanks to you Jon - that does indeed fix it. Quite a subtle one!
Even
| calling zipper.Flush() (just before ToArray) doesn't fix it...
|
| I guess there's a cautionary lesson in there for all of use regarding
| bracing etc : at a casual glance the following look functionally
identical -
| but one works and one doesn't... a bug that I think got introduced when
| "tidying" the braces because they obviously [sic] do the same thing...
|
| // works
| using (Stream input = new MemoryStream(data))
| using (MemoryStream output = new MemoryStream()) {
| using (Stream zipper = new GZipStream(output, CompressionMode.Compress,
| true)) {
| CopyStream(input, zipper);
| }
| return output.ToArray();
| }
|
| and
|
| // fails
| using (Stream input = new MemoryStream(data))
| using (MemoryStream output = new MemoryStream())
| using (Stream zipper = new GZipStream(output, CompressionMode.Compress,
| true)) {
| CopyStream(input, zipper);
| return output.ToArray();
| }
|
| Marc
|
|
Feb 2 '06 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: orekinbck | last post by:
Hi There Is there any easy way to use the System.Compression tools in .NET 2.0 to compress an entire directory ? All my source code is kept in a single directory so I have written a utility...
17
by: dunric | last post by:
After writing the computing urban legend "The Helsinki Code", I spent several nights thinking up how in the world Gustav Larsson, the Finnish PDP-8 computer programmer, could have managed to...
2
by: TerryStone | last post by:
Thanks to anyone who reads this. Below is some C# that compresses an array of bytes, and then decompresses, and compares the original data with the new. Firstly, the length of the decompressed...
1
by: Smokey Grindel | last post by:
I have a bitmap object I want to return as a JPEG image with a compression set at 90% and progressive passes enabled, how can I do this in .NET 2.0? Progressive passes are not necessary but the...
6
by: Fla | last post by:
Hi! I would like to use Compression namespace for Array, i.e. use .NET native Compression for compress a String, or an Array of Integer whose elements are returned values of AscW for each char of...
5
by: jeremyje | last post by:
I'm writing some code that will convert a regular string to a byte for compression and then beable to convert that compressed string back into original form. Conceptually I have.... For...
20
by: chance | last post by:
Hello, I want to add compression to a memory stream and save it in an Oracle database. This is the code I have so far: //save the Word document to a binary field, MemoryStream dataStream = new...
6
by: mike_dba | last post by:
I have been testing compression for update operations. Can anyone tell me why I require more log for an update of a compressed table than I do for the same table that is not compressed ? I...
6
by: pooppoop | last post by:
Hi, and thanks for viewing my post. i have an odd result when trying to compress and decompress a string. it seems that when i replace the Zero's in the input stream it works, if not the string...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.