validation testing -- use TPC-C? (or what?)

[z]

How can validation testing on DBMSs such as DB2 and Oracle be performed?

Is there a quick commonly used publicly available DBMS test that can be
used?

Could TPC-C be used? If so, where can TPC-C be obtained? (Is is publicly
available?)

Thanks.

[Larry Menard]

My TPC knowledge is a bit dated, so take it with a grain of salt.

First of all, make sure you are looking for the right benchmark for the
type of processing you are interested in. For example, TPC-C is a benchmark
that is indicative primarily of rapid transaction processing. There are
other benchmarks for workloads that are more DSS-intensive (TPC-H or TPC-D)
or web-intensive (TPC-W).

Second, I don't know of any publicly available TPC-C kits. It is my
understanding that the source code used for TPC-C benchmark applications are
closely-guarded secrets, since the application code itself might contain
code that accounts for significant performance improvements.
--
--------------------
Larry Menard
"Defender of Geese and of All Things Natural"
"z" <[e-mail]> wrote in message
news:6O*******************@newssvr27.news.prodigy. net...

How can validation testing on DBMSs such as DB2 and Oracle be performed?

Is there a quick commonly used publicly available DBMS test that can be
used?

Could TPC-C be used? If so, where can TPC-C be obtained? (Is is publicly
available?)

Thanks.

[Mark A]

"Larry Menard" <ro**@GoSpamYourself.com> wrote in message
news:To******************************@rogers.com.. .

My TPC knowledge is a bit dated, so take it with a grain of salt.

First of all, make sure you are looking for the right benchmark for the
type of processing you are interested in. For example, TPC-C is a
benchmark that is indicative primarily of rapid transaction processing.
There are other benchmarks for workloads that are more DSS-intensive
(TPC-H or TPC-D) or web-intensive (TPC-W).

Second, I don't know of any publicly available TPC-C kits. It is my
understanding that the source code used for TPC-C benchmark applications
are closely-guarded secrets, since the application code itself might
contain code that accounts for significant performance improvements.
--
--------------------
Larry Menard
"Defender of Geese and of All Things Natural"

The TPC benchmarks are published at www.tpc.org

I have reproduced the TPC-H (decision support) benchmark and the tpc.org
site provides you with everything you will need, including the test data
generator program. You may find it a bit difficult to compile the test data
generator program successfully, but I was able to get it to work after
changing a few compile parms. They also supply the schema and queries. You
will want to look at the vendor docs for each benchmark to pick up any
customization they have done to queries to allow for slight variations in
SQL syntax between the databases.

The TPC-C OLTP benchmark may be harder to reproduce as mentioned above, but
details of the benchmarks are published by each vendor on the website so you
could look there.

[Serge Rielau]

Larry Menard wrote:

My TPC knowledge is a bit dated, so take it with a grain of salt.

First of all, make sure you are looking for the right benchmark for the
type of processing you are interested in. For example, TPC-C is a benchmark
that is indicative primarily of rapid transaction processing. There are
other benchmarks for workloads that are more DSS-intensive (TPC-H or TPC-D)
or web-intensive (TPC-W).

Second, I don't know of any publicly available TPC-C kits. It is my
understanding that the source code used for TPC-C benchmark applications are
closely-guarded secrets, since the application code itself might contain
code that accounts for significant performance improvements.

Larry,

Actually the rules demand full disclosure. And everything should be
available rom the TPC website. Starting from the algorithms to generate
the data to the exact configuration and sourcecodee for any run.
In theory a competitor can validate a TPC result inhouse
(I doubt that this has ever been done though).
Due to this rule to expose these benchmarks can be challenged.
E.g. Oracle successfully challenged the first SQL Server 2000 TPC-C
benchmark using clustering (partitioned views) because SS2000 couldn't
update the partitioning key.

Cheers
Serge
--
Serge Rielau
DB2 SQL Compiler Development
IBM Toronto Lab

[Larry Menard]

I stand (or actually, sit) corrected.

--
--------------------
Larry Menard
"Defender of Geese and of All Things Natural"
"Serge Rielau" <sr*****@ca.ibm.com> wrote in message
news:40*************@individual.net...

Larry Menard wrote:

My TPC knowledge is a bit dated, so take it with a grain of salt.

First of all, make sure you are looking for the right benchmark for
the type of processing you are interested in. For example, TPC-C is a
benchmark that is indicative primarily of rapid transaction processing.
There are other benchmarks for workloads that are more DSS-intensive
(TPC-H or TPC-D) or web-intensive (TPC-W).

Second, I don't know of any publicly available TPC-C kits. It is my
understanding that the source code used for TPC-C benchmark applications
are closely-guarded secrets, since the application code itself might
contain code that accounts for significant performance improvements.

Larry,

Actually the rules demand full disclosure. And everything should be
available rom the TPC website. Starting from the algorithms to generate
the data to the exact configuration and sourcecodee for any run.
In theory a competitor can validate a TPC result inhouse
(I doubt that this has ever been done though).
Due to this rule to expose these benchmarks can be challenged.
E.g. Oracle successfully challenged the first SQL Server 2000 TPC-C
benchmark using clustering (partitioned views) because SS2000 couldn't
update the partitioning key.

Cheers
Serge
--
Serge Rielau
DB2 SQL Compiler Development
IBM Toronto Lab

[Matt Emmerton]

"z" <[e-mail]> wrote in message
news:6O*******************@newssvr27.news.prodigy. net...

How can validation testing on DBMSs such as DB2 and Oracle be performed?

Is there a quick commonly used publicly available DBMS test that can be
used?

Could TPC-C be used? If so, where can TPC-C be obtained? (Is is publicly
available?)

If you want to validate a database for use in your business environment, the
best thing to do is build a simple prototype of your business app and test
it on the databases you're considering. Both DB2 and Oracle come in
"express" editions, and should be full-featured enough to put a prototype
through it's paces. This way you can play with performance scaling, various
DDL options, backup/restore and other things that you're bound to do when in
production, and really get a feel for what works best in the intended
environment.

To echo someone else's comments -- using an off-the-shelf benchmark (such as
any of the TPC benchmarks) may not be anywhere near close to what your
business needs are, and thus you'd end up with something that isn't a good
fit. It's much like deciding to buy a sports car because it was the fastest
and flashiest, and then finding out that your wife is pregnant with triplets
and you should have bought a mini-van :)

--
Matt Emmerton

[DA Morgan]

Matt Emmerton wrote:

"z" <[e-mail]> wrote in message
news:6O*******************@newssvr27.news.prodigy. net...

How can validation testing on DBMSs such as DB2 and Oracle be performed?

Is there a quick commonly used publicly available DBMS test that can be
used?

Could TPC-C be used? If so, where can TPC-C be obtained? (Is is publicly
available?)

If you want to validate a database for use in your business environment, the
best thing to do is build a simple prototype of your business app and test
it on the databases you're considering.

Absolutely. TCP-? has absolutely nothing to do with the real world and
will provide you zero information related to your specific hardware,
your specific operating system, your specific configuration, your
specific application or anything else.
--
Daniel A. Morgan
http://www.psoug.org
da******@x.washington.edu
(replace x with u to respond)