473,320 Members | 2,177 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Large Table Update/Merge problem

I have a temprary table with 5 million rows (A)
which needs to be appended with 90 million row table (B).

60% of the rows of the 5mil rows already exist in the big table

i need to update/merge the table A data with table B
Oracle version is 8.1.7

Please advice which method is the fastest.
i need to do it <30 mins
Jul 19 '05 #1
3 14017
Is the table B in another database? or on the same database? or in non
database format (flat file)? or non oracle database?

ra*************@yahoo.com (Raghu) wrote in message news:<45*************************@posting.google.c om>...
I have a temprary table with 5 million rows (A)
which needs to be appended with 90 million row table (B).

60% of the rows of the 5mil rows already exist in the big table

i need to update/merge the table A data with table B
Oracle version is 8.1.7

Please advice which method is the fastest.
i need to do it <30 mins

Jul 19 '05 #2
Do the update first.

update
(
select a.col1 acol1, b.col1 bcol1
from a, b
where a.key = b.key
)
set acol1 = bcol1

make sure you have unique key on the "key" columns of both tables.

Then go for the insert -

insert into b select * from a where not exists
(
select null from b where a.key = b.key
)

You may need to set statistics on the temporary tables to get an efficient plan.
(Cannot gather statistics on temporary tables)
Pratap Deshmukh
Cognizant Technology Solutions, India
Jul 19 '05 #3
ra*************@yahoo.com (Raghu) wrote in message news:<45*************************@posting.google.c om>...
I have a temprary table with 5 million rows (A)
which needs to be appended with 90 million row table (B).

60% of the rows of the 5mil rows already exist in the big table

i need to update/merge the table A data with table B
Oracle version is 8.1.7

Please advice which method is the fastest.
i need to do it <30 mins


Fastest method is using SQLLoader:
- export temp table in a flat file
- make sure a primary key exist on dest table
- allow SQLLoader accept 60% * 5mil = 3mil errors ( or adjust as you
prefer )
- load flat file via SQL Loader.

You can also write some PL/SQL to insert a row a time catching and
ignoring dup key exception. Commit every some thousand not to let
rollback seg grow too much

Insert 'where not exist' is terrible.

Bye
Cristian
Jul 19 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Spare Brain | last post by:
Hi Folks, I need to INSERT data into the table where the row may already be present. Can MERGE help me out? I'm limited to using SQL only, and thew DB is Oracle 9.2. The low-tech solution...
0
by: Johannes B. Ullrich | last post by:
--=-WKgoK98ejo9BZyGYc3N/ Content-Type: text/plain Content-Transfer-Encoding: quoted-printable I am having problems with MySQL 4.0.12 on RedHat Advanced Server 2.1 using a dual Xeon with 8...
0
by: Shane Niebergall | last post by:
Hi guys - I have a table that is approaching 4 gigs. I have optimized as much as I can with indexes so that select statements are ok, but updating entries seems to be taking a bit of time. I...
2
by: William Wisnieski | last post by:
Hi Everyone, Access 2000 I have some code behind a button that performs a word merge with a query data source. The merge works fine. But what I'd like to do somehow is after the merge is...
1
by: Jeff | last post by:
I'm actually stuck! Can't believe it... so I'd appreciate some help. What I'm doing is enabling users of an ASP.NET 1.1 Web application to update a table in a SQL Server 2000 database. To make...
16
by: UDBDBA | last post by:
Hi All: I need some clarification on a MERGE statement. The database is on V8 FP12 (AIX) 64bit. The source table is tableA. The target is a View "FACT" with UNION ALL because of the 512 Gig...
0
by: eduardasm | last post by:
Hello, I have a problem with XML schema update for one XML column (problem exists in both SP1 and SP2 for SQL Server 2005). 1. I have a table that looks like this: CREATE TABLE .( NOT NULL...
16
by: Jack | last post by:
I need to process large amount of data. The data structure fits well in a dictionary but the amount is large - close to or more than the size of physical memory. I wonder what will happen if I try...
3
by: Raghu | last post by:
I have a temprary table with 5 million rows (A) which needs to be appended with 90 million row table (B). 60% of the rows of the 5mil rows already exist in the big table i need to update/merge...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.