473,385 Members | 1,620 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

Large Table Update/Merge problem

I have a temprary table with 5 million rows (A)
which needs to be appended with 90 million row table (B).

60% of the rows of the 5mil rows already exist in the big table

i need to update/merge the table A data with table B
Oracle version is 8.1.7

Please advice which method is the fastest.
i need to do it <30 mins
Jul 19 '05 #1
3 14020
Is the table B in another database? or on the same database? or in non
database format (flat file)? or non oracle database?

ra*************@yahoo.com (Raghu) wrote in message news:<45*************************@posting.google.c om>...
I have a temprary table with 5 million rows (A)
which needs to be appended with 90 million row table (B).

60% of the rows of the 5mil rows already exist in the big table

i need to update/merge the table A data with table B
Oracle version is 8.1.7

Please advice which method is the fastest.
i need to do it <30 mins

Jul 19 '05 #2
Do the update first.

update
(
select a.col1 acol1, b.col1 bcol1
from a, b
where a.key = b.key
)
set acol1 = bcol1

make sure you have unique key on the "key" columns of both tables.

Then go for the insert -

insert into b select * from a where not exists
(
select null from b where a.key = b.key
)

You may need to set statistics on the temporary tables to get an efficient plan.
(Cannot gather statistics on temporary tables)
Pratap Deshmukh
Cognizant Technology Solutions, India
Jul 19 '05 #3
ra*************@yahoo.com (Raghu) wrote in message news:<45*************************@posting.google.c om>...
I have a temprary table with 5 million rows (A)
which needs to be appended with 90 million row table (B).

60% of the rows of the 5mil rows already exist in the big table

i need to update/merge the table A data with table B
Oracle version is 8.1.7

Please advice which method is the fastest.
i need to do it <30 mins


Fastest method is using SQLLoader:
- export temp table in a flat file
- make sure a primary key exist on dest table
- allow SQLLoader accept 60% * 5mil = 3mil errors ( or adjust as you
prefer )
- load flat file via SQL Loader.

You can also write some PL/SQL to insert a row a time catching and
ignoring dup key exception. Commit every some thousand not to let
rollback seg grow too much

Insert 'where not exist' is terrible.

Bye
Cristian
Jul 19 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Spare Brain | last post by:
Hi Folks, I need to INSERT data into the table where the row may already be present. Can MERGE help me out? I'm limited to using SQL only, and thew DB is Oracle 9.2. The low-tech solution...
0
by: Johannes B. Ullrich | last post by:
--=-WKgoK98ejo9BZyGYc3N/ Content-Type: text/plain Content-Transfer-Encoding: quoted-printable I am having problems with MySQL 4.0.12 on RedHat Advanced Server 2.1 using a dual Xeon with 8...
0
by: Shane Niebergall | last post by:
Hi guys - I have a table that is approaching 4 gigs. I have optimized as much as I can with indexes so that select statements are ok, but updating entries seems to be taking a bit of time. I...
2
by: William Wisnieski | last post by:
Hi Everyone, Access 2000 I have some code behind a button that performs a word merge with a query data source. The merge works fine. But what I'd like to do somehow is after the merge is...
1
by: Jeff | last post by:
I'm actually stuck! Can't believe it... so I'd appreciate some help. What I'm doing is enabling users of an ASP.NET 1.1 Web application to update a table in a SQL Server 2000 database. To make...
16
by: UDBDBA | last post by:
Hi All: I need some clarification on a MERGE statement. The database is on V8 FP12 (AIX) 64bit. The source table is tableA. The target is a View "FACT" with UNION ALL because of the 512 Gig...
0
by: eduardasm | last post by:
Hello, I have a problem with XML schema update for one XML column (problem exists in both SP1 and SP2 for SQL Server 2005). 1. I have a table that looks like this: CREATE TABLE .( NOT NULL...
16
by: Jack | last post by:
I need to process large amount of data. The data structure fits well in a dictionary but the amount is large - close to or more than the size of physical memory. I wonder what will happen if I try...
3
by: Raghu | last post by:
I have a temprary table with 5 million rows (A) which needs to be appended with 90 million row table (B). 60% of the rows of the 5mil rows already exist in the big table i need to update/merge...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.