471,306 Members | 868 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,306 software developers and data experts.

compression - dropping the last byte

It might just be my tired eyes, but I can't see what is wrong in the
following:

I have a byte array filled with random data (with a fixed seed to make
reproducable); I then compress and decompress this using the 2.0 GZip
compression classes - however, as you see, the last byte is getting dropped.

Any ideas? Almost certainly me having a blond moment ;-(

Marc

Overview:
GetByteData : returns an array of random data
Describe : details the first / last 5 values to the console (first: last,
second : penultimate, etc...)
CopyStream : reads / writes between two streams until the source is
exhausted (is there an wasier way of doing this?)
Compress / Decompress : use the GZip classes to manipulate the data

Results:

* 50000
0=170 49999=187
1=200 49998=183
2=81 49997=158
3=81 49996=216
4=66 49995=154

* 49999
0=170 49998=183
1=200 49997=158
2=81 49996=216
3=81 49995=154
4=66 49994=133
Code : watch for wrap
================

using System;
using System.IO;
using System.IO.Compression;
namespace ConsoleApplication1 {
static class Program {
public static void Main() {
byte[] data = GetByteData();
Describe(data);
Console.WriteLine();
byte[] compressed = Compress(data);
byte[] decompressed = Decompress(compressed);
Describe(decompressed);
Console.ReadLine();
}

private static void Describe(byte[] data) {
Console.WriteLine("* " + data.Length.ToString());
int length = data.Length;
for (int i = 0; i < 5; i++) {
int otherEnd = length - 1 - i;
Console.WriteLine(string.Format("{0}={1}\t{2}={3}" ,i,data[i],otherEnd,data[otherEnd]));
}
}
private static byte[] Compress(byte[] data) {
using (Stream input = new MemoryStream(data))
using (MemoryStream output = new MemoryStream())
using (Stream zipper = new GZipStream(output,
CompressionMode.Compress, true)) {
CopyStream(input, zipper);
return output.ToArray();
}
}
private static byte[] Decompress(byte[] data) {
using (Stream input = new MemoryStream(data))
using (Stream unzipper = new GZipStream(input,
CompressionMode.Decompress, false))
using (MemoryStream output = new MemoryStream()) {
CopyStream(unzipper, output);
return output.ToArray();
}
}

private const int SEED = 141566, SIZE = 50000;

public static byte[] GetByteData() {
Random rand = new Random(SEED); // some seed to make
reproducable
byte[] data = new byte[SIZE];
rand.NextBytes(data);
return data;

}

public static long CopyStream(System.IO.Stream source,
System.IO.Stream destination) {
const int BUFFER_SIZE = 512;
long copied = 0;
byte[] buffer = new byte[BUFFER_SIZE];
int bytes;
while ((bytes = source.Read(buffer, 0, BUFFER_SIZE)) > 0) {
destination.Write(buffer, 0, bytes);
copied += bytes;
}
return copied;
}
}
}

Feb 1 '06 #1
3 2473
Marc Gravell wrote:
It might just be my tired eyes, but I can't see what is wrong in the
following:

I have a byte array filled with random data (with a fixed seed to make
reproducable); I then compress and decompress this using the 2.0 GZip
compression classes - however, as you see, the last byte is getting dropped.

Any ideas? Almost certainly me having a blond moment ;-(


Move the return output.ToArray(); call *outside* the "using (Stream
zipper...)" block. That way the GZipStream is disposed of (and
therefore flushed and terminated appropriately) before you return the
data.

Making that change (which requires extra bracing) makes your sample
work.

Jon

Feb 1 '06 #2
A huge thanks to you Jon - that does indeed fix it. Quite a subtle one! Even
calling zipper.Flush() (just before ToArray) doesn't fix it...

I guess there's a cautionary lesson in there for all of use regarding
bracing etc : at a casual glance the following look functionally identical -
but one works and one doesn't... a bug that I think got introduced when
"tidying" the braces because they obviously [sic] do the same thing...

// works
using (Stream input = new MemoryStream(data))
using (MemoryStream output = new MemoryStream()) {
using (Stream zipper = new GZipStream(output, CompressionMode.Compress,
true)) {
CopyStream(input, zipper);
}
return output.ToArray();
}

and

// fails
using (Stream input = new MemoryStream(data))
using (MemoryStream output = new MemoryStream())
using (Stream zipper = new GZipStream(output, CompressionMode.Compress,
true)) {
CopyStream(input, zipper);
return output.ToArray();
}

Marc
Feb 1 '06 #3
one would think flush should have solved this. Is this by design?

--
William Stacey [MVP]

"Marc Gravell" <mg******@rm.com> wrote in message
news:%2****************@TK2MSFTNGP10.phx.gbl...
|A huge thanks to you Jon - that does indeed fix it. Quite a subtle one!
Even
| calling zipper.Flush() (just before ToArray) doesn't fix it...
|
| I guess there's a cautionary lesson in there for all of use regarding
| bracing etc : at a casual glance the following look functionally
identical -
| but one works and one doesn't... a bug that I think got introduced when
| "tidying" the braces because they obviously [sic] do the same thing...
|
| // works
| using (Stream input = new MemoryStream(data))
| using (MemoryStream output = new MemoryStream()) {
| using (Stream zipper = new GZipStream(output, CompressionMode.Compress,
| true)) {
| CopyStream(input, zipper);
| }
| return output.ToArray();
| }
|
| and
|
| // fails
| using (Stream input = new MemoryStream(data))
| using (MemoryStream output = new MemoryStream())
| using (Stream zipper = new GZipStream(output, CompressionMode.Compress,
| true)) {
| CopyStream(input, zipper);
| return output.ToArray();
| }
|
| Marc
|
|
Feb 2 '06 #4

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

3 posts views Thread by orekinbck | last post: by
17 posts views Thread by dunric | last post: by
1 post views Thread by Smokey Grindel | last post: by
6 posts views Thread by Fla | last post: by
20 posts views Thread by chance | last post: by
6 posts views Thread by mike_dba | last post: by
reply views Thread by rosydwin | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.