compressing data stored in a database

[Tarscher]

Hi all,

My application uses a database to store big blobs (0.5 - 1 Gb). I have
a List of a class Sample that I store in a blob. Since the blob can
become very big I want to use compression.

The class Sample contains a lot of properties. When I store the List
of samples only a few of the properties are used. Does it makes sense
to make a class sampleSimple that only contains the properties I use
for database storage and thus save storage space?

Is it possible to first compress the blob and then store it in the
database? Does .NET provides such functionality?

Many thanks in advance.

Stijn

[Peter Duniho]

Tarscher wrote:

[...]
The class Sample contains a lot of properties. When I store the List
of samples only a few of the properties are used. Does it makes sense
to make a class sampleSimple that only contains the properties I use
for database storage and thus save storage space?

I would say so. Though, my first thought would be to try to understand
better why you are using a class that has many more properties than you
consider essential for saving.

Also, I admit I don't really get the "blob" stuff -- outside my database
use experience -- but if it's anything like the regular serialization
stuff, you should be able to suppress serialization of specific
properties without creating a whole new class (there's an attribute that
does this per-property).

Is it possible to first compress the blob and then store it in the
database? Does .NET provides such functionality?

Depends on how you are creating the blob. If the mechanism involves
getting a MemoryStream, byte array or similar binary representation of
the data, then you can probably use the GzipStream class to compress the
data before you store it in the database.

Pete

[Peter K]

Peter Duniho <Np*********@NnOwSlPiAnMk.comwrote in news:13cihm9ip5mpje4
@corp.supernews.com:

<snip>

Depends on how you are creating the blob. If the mechanism involves
getting a MemoryStream, byte array or similar binary representation of
the data, then you can probably use the GzipStream class to compress the
data before you store it in the database.

Note also that some databases can be configured to automatically compress &
decompress data.

[Cor Ligthert [MVP]]

Tarcher,

If your blob is used for images, than AFAIK you can spare the time. Most
image types are so optimalized that compressing can even result in some
extra bytes.

Cor

"Tarscher" <ta******@gmail.comschreef in bericht
news:11**********************@19g2000hsx.googlegro ups.com...

Hi all,

My application uses a database to store big blobs (0.5 - 1 Gb). I have
a List of a class Sample that I store in a blob. Since the blob can
become very big I want to use compression.

The class Sample contains a lot of properties. When I store the List
of samples only a few of the properties are used. Does it makes sense
to make a class sampleSimple that only contains the properties I use
for database storage and thus save storage space?

Is it possible to first compress the blob and then store it in the
database? Does .NET provides such functionality?

Many thanks in advance.

Stijn

[Rad [Visual C# MVP]]

On Mon, 20 Aug 2007 00:22:17 -0700, Tarscher <ta******@gmail.com>
wrote:

>Hi all,

My application uses a database to store big blobs (0.5 - 1 Gb). I have
a List of a class Sample that I store in a blob. Since the blob can
become very big I want to use compression.

The class Sample contains a lot of properties. When I store the List
of samples only a few of the properties are used. Does it makes sense
to make a class sampleSimple that only contains the properties I use
for database storage and thus save storage space?

Is it possible to first compress the blob and then store it in the
database? Does .NET provides such functionality?

Many thanks in advance.

Stijn

Well, first you might want to rethink your class design. If there are
so many properties that are not necessary for persistence, any
particular reason why they're there?

Also, like Cor has said, for some types of data like MP3, Image, Video
etc you are unlikely to get much savings from additional compression.
What sort of data are you storing in the blob?

--
http://bytes.thinkersroom.com

[Tarscher]

On 20 aug, 14:30, "Rad [Visual C# MVP]" <[e-mail]:

On Mon, 20 Aug 2007 00:22:17 -0700,Tarscher<[e-mail]>
wrote:

Hi all,

My application uses a database to store big blobs (0.5 - 1 Gb). I have
a List of a class Sample that I store in a blob. Since the blob can
become very big I want to use compression.

The class Sample contains a lot of properties. When I store the List
of samples only a few of the properties are used. Does it makes sense
to make a class sampleSimple that only contains the properties I use
for database storage and thus save storage space?

Is it possible to first compress the blob and then store it in the
database? Does .NET provides such functionality?

Many thanks in advance.

Stijn

Well, first you might want to rethink your class design. If there are
so many properties that are not necessary for persistence, any
particular reason why they're there?

Also, like Cor has said, for some types of data like MP3, Image, Video
etc you are unlikely to get much savings from additional compression.
What sort of data are you storing in the blob?

--http://bytes.thinkersroom.com

many thanks for all the replies.

There aren't actually that many properties but (4 in total) of which I
use 3 when stored in the database. Since I store a large number of
objects I think every byte I can save in serialization process will
eventuelly add up to the savings.

My object contains a string, an integer and a double. I create a list
of the object with about 1 million objects. This data I thus want to
compress