473,543 Members | 1,797 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

COMPRESS .. & VALUES COMPRESSION change request: RFC.

Well known, during "CREATE TABLE" we could optionally specify "COMPRESS SYSTEM DEFAULT" and "VALUE COMPRESSION" options. The main idea about these statements is : NOT to keep on disk the values that can be deduced from some means, isn't it ?

The small investigation ( on DB2/LINUX 8.2.3 ) shows these values are compressed:

- for numbers (integers,decim als): ZEROs and NULLs (if nullable).
- for strings (graphic) : sequences of spaces of any length (0-max) and NULLs (if nullable).

Nothing wrong with this implementation, but... The more natural behaviour is expected, like defined in DB2 Reference manual: "... default values used for the data types ... are to be stored using minimal space".

The change desirable: instead of skipping default value for "data type", do skip default value for COLUMN in table definition.

This change would greatly benefit applications which create huge sparse tables all filled almost single values (like JDE Peoplesoft ERP system).

Cheers,
--
Konstantin Andreev.
Jul 6 '06 #1
7 2376
Konstantin Andreev wrote:
Well known, during "CREATE TABLE" we could optionally specify "COMPRESS
SYSTEM DEFAULT" and "VALUE COMPRESSION" options. The main idea about
these statements is : NOT to keep on disk the values that can be deduced
from some means, isn't it ?

The small investigation ( on DB2/LINUX 8.2.3 ) shows these values are
compressed:

- for numbers (integers,decim als): ZEROs and NULLs (if nullable).
- for strings (graphic) : sequences of spaces of any length (0-max) and
NULLs (if nullable).

Nothing wrong with this implementation, but... The more natural
behaviour is expected, like defined in DB2 Reference manual: "...
default values used for the data types ... are to be stored using
minimal space".

The change desirable: instead of skipping default value for "data
type", do skip default value for COLUMN in table definition.

This change would greatly benefit applications which create huge sparse
tables all filled almost single values (like JDE Peoplesoft ERP system).
DB2 9 has table compression. Customers like VISA have seen huge (like
80%) reduction in size.

Cheers
Serge
--
Serge Rielau
DB2 Solutions Development
IBM Toronto Lab

IOD Conference
http://www.ibm.com/software/data/ond...ness/conf2006/
Jul 6 '06 #2
Serge Rielau wrote:
>The change desirable: instead of skipping default value for "data type", do skip default value for COLUMN in table definition.

This change would greatly benefit applications which create huge sparse tables all filled almost single values (like JDE Peoplesoft ERP system).
DB2 9 has table compression. Customers like VISA have seen huge (like 80%) reduction in size.
Excellent ! Waiting for production release of DB2 Viper...
--
Konstantin Andreev.
Jul 7 '06 #3
Serge Rielau<sr*****@ ca.ibm.com07/06/06 9:02 AM >>>
>DB2 9 has table compression. Customers like VISA have seen huge (like
80%) reduction in size.
You're probably not at liberty to say, but I'll ask anyway... Assuming you
know the answer, does Visa use DB2 to store all online transactions that
pass through VisaNet? I was under the impression they used z/TPF for their
transaction processing. (Or is there a special version of DB2 for TPF?).

Just wondering... we are a VisaNet processor endpoint, but we use VSAM for
all of our "card system related" files, for performance reasons. But if
Visa uses DB2 I can't imagine any reason we shouldn't also.

Frank
---
Frank Swarbrick
Senior Developer/Analyst - Mainframe Applications
FirstBank Data Corporation - Lakewood, CO USA
Jul 7 '06 #4
Frank Swarbrick wrote:
Serge Rielau<sr*****@ ca.ibm.com07/06/06 9:02 AM >>>
>DB2 9 has table compression. Customers like VISA have seen huge (like
80%) reduction in size.

You're probably not at liberty to say, but I'll ask anyway... Assuming you
know the answer, does Visa use DB2 to store all online transactions that
pass through VisaNet? I was under the impression they used z/TPF for their
transaction processing. (Or is there a special version of DB2 for TPF?).

Just wondering... we are a VisaNet processor endpoint, but we use VSAM for
all of our "card system related" files, for performance reasons. But if
Visa uses DB2 I can't imagine any reason we shouldn't also.
The data point I gave on compression relates to a data warehouse:
http://www.ibm.com/software/data/db2...11&S_CMP=VIPER
I don't know what they run beyond that on DB2 for LUW and you are
correct, if I knew I probably wouldn't be at liberty to say without a
quote to point to as in above link. Now that quote does have a name and
Ellen is a friendly person.

Cheers
Serge
--
Serge Rielau
DB2 Solutions Development
IBM Toronto Lab

IOD Conference
http://www.ibm.com/software/data/ond...ness/conf2006/
Jul 7 '06 #5
Serge Rielau wrote:
>The change desirable: instead of skipping default value for "data type", do skip default value for COLUMN in table definition.

This change would greatly benefit applications which create huge sparse tables all filled almost single values (like JDE Peoplesoft ERP system).
DB2 9 has table compression. Customers like VISA have seen huge (like 80%) reduction in size.
I've just read "Introducin g DB2 9, Part 1: Data compression in DB2 9" from developerWorks. The "table compression" is rather distinct from proposed "column default" compression. How would be interesting to compare the effectiveness.. .
--
Konstantin Andreev.
Jul 7 '06 #6
Serge Rielau<sr*****@ ca.ibm.com07/07/06 12:23 PM >>>
>Frank Swarbrick wrote:
>Serge Rielau<sr*****@ ca.ibm.com07/06/06 9:02 AM >>>
>>DB2 9 has table compression. Customers like VISA have seen huge (like
80%) reduction in size.

You're probably not at liberty to say, but I'll ask anyway... Assuming
you
>know the answer, does Visa use DB2 to store all online transactions that
pass through VisaNet? I was under the impression they used z/TPF for
their
>transaction processing. (Or is there a special version of DB2 for
TPF?).
>>
Just wondering... we are a VisaNet processor endpoint, but we use VSAM
for
>all of our "card system related" files, for performance reasons. But if
Visa uses DB2 I can't imagine any reason we shouldn't also.
The data point I gave on compression relates to a data warehouse:
http://www.ibm.com/software/data/db2...11&S_CMP=VIPER
I don't know what they run beyond that on DB2 for LUW and you are
correct, if I knew I probably wouldn't be at liberty to say without a
quote to point to as in above link. Now that quote does have a name and
Ellen is a friendly person.
Data warehouse makes sense. I can only imagine (in my nightmares!) how much
data they are storing!

Thanks for the info and the link. (Now is that a picture of you on that
page? <g>)

Frank
---
Frank Swarbrick
Senior Developer/Analyst - Mainframe Applications
FirstBank Data Corporation - Lakewood, CO USA
Jul 8 '06 #7
Frank Swarbrick wrote:
Data warehouse makes sense. I can only imagine (in my nightmares!) how much
data they are storing!
*chuckle*Now add Information Integration and Identity Analytics to the
mix and you're still playing catch up with reality.

Cheers
Serge
--
Serge Rielau
DB2 Solutions Development
IBM Toronto Lab

IOD Conference
http://www.ibm.com/software/data/ond...ness/conf2006/
Jul 8 '06 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

11
2470
by: Shiperton Henethe | last post by:
Dreamweaver 4 Hi Can anyone recommend a decent utility for compressing HTML that is safe - i.e. that *definitely* doesn't mess with the appearance in any browsers. I run a growing website whose pages are inexorably getting too "heavy".
1
6608
by: Herbert Wildmeister via .NET 247 | last post by:
Does anybody know a way to compress and decompress AVI videofiles in C# .NET using all installed video codecs? I've foundthe Video Compression Manager and a class wrapper for C++ but noCLS compliant equivalent. If anybody has a solution that doesthis in .NET please post it. Source and/or sample would begreat. Thanks. -----------------------...
8
6448
by: Jose L. Velazquez | last post by:
Hi all, I have made a webservice that returns an XML, but sometimes the connection is so slow and there is a lot of data to be returned. I would like to know if it is possible to send this data compressed, and how to do it. Thanks. Jose Luis.
6
2985
by: Champika Nirosh | last post by:
Hi, I have two machine where I needed to have a extended TCP/IP protocol to make the link between the two machines Mean,I need to write a application that compress every data the machine send out.. and also need to un compress at the receiving end.. futher explaing...
6
8799
by: Adriano | last post by:
Can anyone recommend a simple way to compress/decomress a String in .NET 1.1 ? I have a random string of 70 characters, the output from a DES3 encryption, and I wish to reduce the lengh of it, thanks,
6
6150
by: =?Utf-8?B?V2F5bmUgR29yZQ==?= | last post by:
Hi I want to achive 2 things. First I would like to compress an existing file on my harddrive. I can easily find out if a file is compressed or not by using "File.GetAttributes". But no matter what I do I simply can't set the compressed attribute using "SetAttributes". I have no problem setting the attributes "Read-Only", "Archive" or...
5
4909
by: zgh1970 | last post by:
Hi, Friends, default DB2 compression library. I am wondering if this option will have any new restriction on RESTORE in the following. (Can I used that backup imsage for restore at the follwoing compatible situations?) : There are operating system considerations when using the RESTORE DATABASE command:
4
2143
by: Lennart | last post by:
I heard a rumor that compress is only availible in Enterprise Edition in 9.5, can anyone confirm/reject this rumor? /Lennart
0
7354
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
7594
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
0
7746
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
1
7354
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
5888
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
0
4898
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3394
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3394
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1824
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.