473,405 Members | 2,444 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,405 software developers and data experts.

Insert performance into DB2 using informatica

I need to speed up the inserts through informatica, and I came across
insert buf option with db2 packages for import statements and how we
can use that option to do the same in java programs. Are there any
packages to be bound for informatica powercenter, so that I can rebind
them with this option?
My informatica person says that Informatica does not bind any packages
to the database. Is he missing something. Any one who has worked in
informatica with db2, please help me guys,,.

Nov 7 '07 #1
2 5544
Ian
Arun Srinivasan wrote:
I need to speed up the inserts through informatica, and I came across
insert buf option with db2 packages for import statements and how we
can use that option to do the same in java programs. Are there any
packages to be bound for informatica powercenter, so that I can rebind
them with this option?
My informatica person says that Informatica does not bind any packages
to the database. Is he missing something. Any one who has worked in
informatica with db2, please help me guys,,.
He is correct. The native DB2 driver for Informatica uses DB2 CLI to
talk to the database.

Theoretically you can bind ALL of the CLI packages with buffered
inserts, but I would not recommend this (that would apply to pretty
much every application connecting to the database except those using
the JDBC Type 4 driver). This can create issues if the applications
have not been coded to handle errors that may occur asynchronously from
the actual call to insert the record.

The best options are using the parallel load option in Informatica
or using the bulk mode in Informatica.

However, one other option is just to have Informatica write to a flat
file and use LOAD. If you are doing multiple loads to the same table,
or have to deal with recoverable loads, then this is generally a better
idea than using bulk mode in Informatica idea because you can specify
load options that aren't available with bulk load mode.

One other thing you can do to enhance insert performance is to put your
tables in APPEND mode. Certain things will prevent this (MDC) or make
it a bad idea (if you have a clustering index and care about the tables'
cluster ratio).

Good luck,
Ian Bjorhovde

Nov 9 '07 #2
Ian wrote:
Arun Srinivasan wrote:
>I need to speed up the inserts through informatica, and I came across
insert buf option with db2 packages for import statements and how we
can use that option to do the same in java programs. Are there any
packages to be bound for informatica powercenter, so that I can rebind
them with this option?
My informatica person says that Informatica does not bind any packages
to the database. Is he missing something. Any one who has worked in
informatica with db2, please help me guys,,.

He is correct. The native DB2 driver for Informatica uses DB2 CLI to
talk to the database.

Theoretically you can bind ALL of the CLI packages with buffered
inserts, but I would not recommend this (that would apply to pretty
much every application connecting to the database except those using
the JDBC Type 4 driver). This can create issues if the applications
have not been coded to handle errors that may occur asynchronously from
the actual call to insert the record.

The best options are using the parallel load option in Informatica
or using the bulk mode in Informatica.

However, one other option is just to have Informatica write to a flat
file and use LOAD. If you are doing multiple loads to the same table,
or have to deal with recoverable loads, then this is generally a better
idea than using bulk mode in Informatica idea because you can specify
load options that aren't available with bulk load mode.

One other thing you can do to enhance insert performance is to put your
tables in APPEND mode. Certain things will prevent this (MDC) or make
it a bad idea (if you have a clustering index and care about the tables'
cluster ratio).

Good luck,
Ian Bjorhovde
Beware of load. You need a dedicated load slot as it can conflict with
queries and even backup (I believe that with V8 both Backup and load do
short locks on the history file). I have seen load force take out a backup.
If you do lots (perhapse 100) of loads per day you will regularly purge
the history file. you will also need to manage failed loads which are much
more unpleasant than failed inserts

Parallel inserts are much faster than a standard insert.

Colin Booth
Nov 10 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: Mark P | last post by:
Some time ago I posted here about inserting into a set with a hint: ...
16
by: robert | last post by:
been ruminating on the question (mostly in a 390/v7 context) of whether, and if so when, a row update becomes an insert/delete. i assume that there is a threshold on the number of columns of the...
7
by: jafastinger | last post by:
In our shop we have a 28,000,000 row insert that gets reloaded every night. I am not looking to change this process. I would like to know why in UDB Vs 7.2 we were getting an insert rate of 3300...
4
by: Mike Hnatt | last post by:
My goal is to get data from an XML file into a couple of tables in an Access database. The XML file is a little complex so I need control over what I do (I can't just read it into a dataset). ...
1
by: Vikas Rana | last post by:
Does anyone have experiences running postgres with Informatica tools? More specifically, are there ways to use Informatica tools with postgres? Informatica doesn't list postgres in its supported...
0
by: deepcrm | last post by:
Hi All, We have an Immediate Opening for DWH -Informatica Professionals, if you are looking out for a change, then plz send me your updated profile ASAP for further process. Location ...
4
by: Michel Esber | last post by:
Hello, Environment: db2 V8 FP 13 LUW Our application currently uses: insert into table values ('A'),('B'),...('Z') We have used CLI arrays inserts (1000 array and commit size) and...
1
by: greenMark | last post by:
Hi I'm really new to informatica and in a urgent need of learning it. I was able to download the software and install the client part. It asked me to enter the licence key when I tried to install the...
3
by: akdemirc | last post by:
i have a problem with large data import to a db in sql server.. Actually i have an application that collects data from an environment and dispatches this data to different csv files for sql server to...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.