473,763 Members | 1,883 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Very slow INSERTs - URGENT

DB2 V8.2 ICE Linux
An application is loading around 6 million rows in a table in database A
(Development environment).

The same application is taking ages to load the same table in database B
(UAT environment).
Database B is supposed to be faster since it is partitioned.
It has Referential integrity (RI) and indexes defined on it.
Does this have a major impact on INSERTs?

Is there anything else that needs to be looked at?

Thanks.

Cheers,
San.

Sep 16 '06 #1
10 3869
Here is a white paper that may help:

http://www-128.ibm.com/developerwork...m-0403wilkins/

Also, just because a database has been partitioned with DPF doesn't mean
it will automatically be faster. You have to ensure that it has the hw
resources that it needs. The hw configuration is very important.

Larry Edelstein

shsandeep wrote:
DB2 V8.2 ICE Linux
An application is loading around 6 million rows in a table in database A
(Development environment).

The same application is taking ages to load the same table in database B
(UAT environment).
Database B is supposed to be faster since it is partitioned.
It has Referential integrity (RI) and indexes defined on it.
Does this have a major impact on INSERTs?

Is there anything else that needs to be looked at?

Thanks.

Cheers,
San.
Sep 16 '06 #2
Thanks Larry!
This is exactly what I was looking for...

Cheers,
San.

Sep 16 '06 #3
San,

If you have a DPF production environment you need a DPF development
environment. Doesn't have to be the same number of nodes. But DPF it
ought to be.

Cheers
Serge
--
Serge Rielau
DB2 Solutions Development
IBM Toronto Lab

IOD Conference
http://www.ibm.com/software/data/ond...ness/conf2006/
Sep 16 '06 #4
Serge,
I totally agree with you.... In fact, you had mentioned this earlier as
well...
But whenever I put this point in front of the management, they ignore it
saying that we are not testing performance in the development
environment...
Are there any other benefits of partitioning the DEV database which I can
put forward to them??

Thanks.

Cheers,
San.

Sep 17 '06 #5
shsandeep wrote:
Serge,
I totally agree with you.... In fact, you had mentioned this earlier as
well...
But whenever I put this point in front of the management, they ignore it
saying that we are not testing performance in the development
environment...
Are there any other benefits of partitioning the DEV database which I can
put forward to them??
Money talks. Collect the amount of damage to the business because of
your current problem and any previous ones you may have encountered.
How are you supposed to track this problem, collect data, experiment if
your only system is production?
All you need are two logical nodes for development/test.
Sure you don't need DPF on the laptops of your app developers. But your
DBA's need access to a scaled down version of the original.

Cheers
Serge
--
Serge Rielau
DB2 Solutions Development
IBM Toronto Lab

IOD Conference
http://www.ibm.com/software/data/ond...ness/conf2006/
Sep 17 '06 #6
Serge is exactly right. My customers have suffered when they don't have
a test system (call it a QA system) that is representative of
production. It doesn't have to be an exact replica, but it should be
representative in some proportion.

Now ... let me tell you why you SHOULD be testing performance in a QA
system. It is not unheard of for maintenance (be it OS-related,
DB2-related, or even ISV-related) to affect performance. You want to
apply any changes ... any changes at all to an environment where you can
not only test functionality, but where you can test performance also. In
fact, one of my customers actually has a "benchmark" set of queries/txns
that they established a baseline for and then they run for EVERY SINGLE
change that is made to QA. If a change is made where the difference from
the baseline is significant and unexplained, research and analysis is
conducted to find out why. It could be as simple as an environment
variable that someone forgot to set ... or it could be a bug. You don't
want to find out about this in production ... you want to find out about
these kinds of things before any change every hits production. It
enhances quality of service for your end-users and minimizes outages ...
it is simply good systems management practice and reduces costs (in the
long run) to the business.

Such a QA system should be configured EXACTLY like production in terms
of sw and hw technology, but in terms of size, should be constucted in
some proportion. So for example, if production is a 16-way AIX 5.3
p-series POWER5 server with 8 logical partitions of DB2 DPF V8.2 @ fp 12
on it, perhaps QA should be a 4-way p-series server with the exact same
hw technology (if possible) with AIX 5.3 @ fp12 on it with maybe 2
logical partitions. The data in those 2 logical partitions should be
representative also to be about 1/4 of the total data in production.
That way, you can extrapolate performance results in QA.

Just my 2 cents.

Larry Edelstein
Serge Rielau wrote:
shsandeep wrote:
>Serge,
I totally agree with you.... In fact, you had mentioned this earlier as
well...
But whenever I put this point in front of the management, they ignore it
saying that we are not testing performance in the development
environment. ..
Are there any other benefits of partitioning the DEV database which I can
put forward to them??

Money talks. Collect the amount of damage to the business because of
your current problem and any previous ones you may have encountered.
How are you supposed to track this problem, collect data, experiment if
your only system is production?
All you need are two logical nodes for development/test.
Sure you don't need DPF on the laptops of your app developers. But your
DBA's need access to a scaled down version of the original.

Cheers
Serge
Sep 17 '06 #7
I went thru the same argument a few years back and between the
developers, QA, and the DBAs we finally managed to get ar least a
partial replication - similar to Larry's description - based on those
arguments. About 6 months after we got both the QA and Development
environments commissioned, we had a rash of one-of-a-kind hardware
failures in the production servers that took them off-line at a peak
demand cycle during a new campaign startup. The ability to rotate the
extra servers, even in limited capacities, made follow-on upgrade
requests a no brainer because the value of an additional partial
backup facility was immediately apparent to management (and us dumb
developers <g>).

On Sun, 17 Sep 2006 16:59:05 UTC Larry <la***@nospam.n etwrote:
Serge is exactly right. My customers have suffered when they don't have
a test system (call it a QA system) that is representative of
production. It doesn't have to be an exact replica, but it should be
representative in some proportion.

Now ... let me tell you why you SHOULD be testing performance in a QA
system. It is not unheard of for maintenance (be it OS-related,
DB2-related, or even ISV-related) to affect performance. You want to
apply any changes ... any changes at all to an environment where you can
not only test functionality, but where you can test performance also. In
fact, one of my customers actually has a "benchmark" set of queries/txns
that they established a baseline for and then they run for EVERY SINGLE
change that is made to QA. If a change is made where the difference from
the baseline is significant and unexplained, research and analysis is
conducted to find out why. It could be as simple as an environment
variable that someone forgot to set ... or it could be a bug. You don't
want to find out about this in production ... you want to find out about
these kinds of things before any change every hits production. It
enhances quality of service for your end-users and minimizes outages ...
it is simply good systems management practice and reduces costs (in the
long run) to the business.

Such a QA system should be configured EXACTLY like production in terms
of sw and hw technology, but in terms of size, should be constucted in
some proportion. So for example, if production is a 16-way AIX 5.3
p-series POWER5 server with 8 logical partitions of DB2 DPF V8.2 @ fp 12
on it, perhaps QA should be a 4-way p-series server with the exact same
hw technology (if possible) with AIX 5.3 @ fp12 on it with maybe 2
logical partitions. The data in those 2 logical partitions should be
representative also to be about 1/4 of the total data in production.
That way, you can extrapolate performance results in QA.

Just my 2 cents.

Larry Edelstein
Serge Rielau wrote:
shsandeep wrote:
Serge,
I totally agree with you.... In fact, you had mentioned this earlier as
well...
But whenever I put this point in front of the management, they ignore it
saying that we are not testing performance in the development
environment...
Are there any other benefits of partitioning the DEV database which I can
put forward to them??
Money talks. Collect the amount of damage to the business because of
your current problem and any previous ones you may have encountered.
How are you supposed to track this problem, collect data, experiment if
your only system is production?
All you need are two logical nodes for development/test.
Sure you don't need DPF on the laptops of your app developers. But your
DBA's need access to a scaled down version of the original.

Cheers
Serge

--
Will Honea
Sep 18 '06 #8
shsandeep wrote:
Are there any other benefits of partitioning the DEV database which I can
put forward to them??
There are some things to be aware of in DPF, which don't come into play in
non-DPF environments. Most notably, the fact that you have/need a
partitioning key on each column. Next, you may want to use replicated MQTs
in DPF when it makes sense - that question doesn't come up in non-DPF
either.

I would recommend that you use logical database partitions (all on the same
machine) in your dev environment. Then mgmt can't complain because (a)
they don't need to know it, and (b) you wouldn't need additional
hardware. ;-)

--
Knut Stolze
DB2 Information Integration Development
IBM Germany
Sep 18 '06 #9
Knut Stolze wrote:
I would recommend that you use logical database partitions (all on the same
machine) in your dev environment. Then mgmt can't complain because (a)
they don't need to know it, and (b) you wouldn't need additional
hardware. ;-)
a) Ehem, there is this small issue with licensing DPF.
--
Serge Rielau
DB2 Solutions Development
IBM Toronto Lab

IOD Conference
http://www.ibm.com/software/data/ond...ness/conf2006/
Sep 18 '06 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
8275
by: Robert Hooker | last post by:
Hi, I'm curious to know if I'm doing something wrong here, or if this is just mind-numbingly slow for a reason. In a simple WindowsFormsApplication: public Form1() { // Required for Windows Form Designer support
3
2091
by: Mario Soto | last post by:
Hi. i hava a postresql 7.4.2 in a production server. tha machine is a Pentium IV 2,6 GHZ AND 1 GB IN RAM with lINUX RH 9.0. The postresql.conf say: #--------------------------------------------------------------------------- # RESOURCE USAGE (except WAL) #---------------------------------------------------------------------------
50
5729
by: diffuser78 | last post by:
I have just started to learn python. Some said that its slow. Can somebody pin point the issue. Thans
5
4177
by: PH | last post by:
Hi guys; I got a single processor computer, running an application that launches 2 threads. Each of these threads listens for incoming connections in a specific port, so there is a Loop . Until inside each of them.
4
4774
by: sommes | last post by:
It's only happen on .asp website, what's the problem? Thank you
3
7572
by: cold80 | last post by:
I was just doing a performance test in order to see the benefit of using AJAX instead of doing a postback on the server. So I have a simple page that performs a request to the server and write a counter on the page when it receives the response from the server. I'm not changing the page actually, so the browser don't have to modify the internal DOM and to render the changes. I tried this simple page with IE 7.0, Firefox 2.0 and Opera 9.10....
1
2415
by: dumindaj | last post by:
I have problems running my Java Swing application remotely on a Linux machine. The application takes 20/30s to respond to events and painting takes considerable time. This is not a special case for my app. I copied SwingSet.jar demo over and this is also very slow to response. The command is ssh - X <remote-machine>. It is the bash shell. I have tried setting "ForwardedX11Trusted" in ssh/ssh_config file. Setting environment variables...
5
4248
by: lazyvlad | last post by:
Hi, I'm writing here because this issue is becoming more annoying with each day it passes. So I have a form, a dataset with a few table adapters (3 to be precise) and a datagridview.The datagridview has a lot of columns as checkboxes (30 columns are checkboxes per row). The problem is in the form drawing and painting, when first shown the form paints itself very slow and im running this on a relatively powerful computer (Pentium 2.4GHz with...
0
1454
by: Mohs | last post by:
Hi All, I am very new to DB2, and in a project we are using db2, 8.2 version. My problem is that one of the procedures that previously used to complete in 10 minutes is not getting completed now and keeps on executing. It simply inserts data in tables. We have used runstats to gather stats on table, and when we ran the procedure afterwards it ran fast initially inserting many rows, but later on became very slow, only inserting...
0
10149
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10003
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
9943
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8825
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7370
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6643
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5271
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
2
3529
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2797
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.