473,385 Members | 1,872 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

selecting the differences between 2 large tables

Hi all,
(Sorry for the cross-post, there doesn't appear to be much activity on
comp.database.oracle)I'm trying to get the last 300 rows from the difference
between 2 large
tables and my queries are taking at least 10 minutes to do this.

I'm running on quite a fast server and the tables are not very large,
3,000,000 to 30,000,000 rows.

I've tried the following:
(test is the primary key table and test2 and test3 have foreign keys to test
also, imagine I have the following unique indexes (test_id, sequence_no) and
(parent_id, test_id))

select test_id
from (select test_id from test2 where sequence_no = 0
minus
select test_id from test3 where parent_id = 581)
where rownum < 300

Explain Plan:
SELECT STATEMENT Optimizer=CHOOSE
COUNT (STOPKEY)
VIEW
MINUS
SORT (UNIQUE)
TABLE ACCESS (FULL) OF TEST2
SORT (UNIQUE)
INDEX (RANGE SCAN) OF TEST3_PAR_ID_TEST_ID_UNQ (UNIQUE)

second approach:

select test_id
from test2 t2, (select test_id from test3 where parent_id = 581) t3
where t2.test_id = t3.test_id(+)
and t2.sequence_no = 0
and t3.test_id is null
and rownum < 300

Explain Plan:
SELECT STATEMENT Optimizer=CHOOSE
COUNT (STOPKEY)
FILTER
NESTED LOOPS (OUTER)
TABLE ACCESS (FULL) OF TEST2
INDEX (UNIQUE SCAN) OF TEST3_PAR_ID_TEST_ID_UNQ (UNIQUE)

The table creates for the above tables are:
create table test (
test_id number primary key
);

create table test2 (
test2_id number primary key,
test_id number references test,
sequence_no number,
test_text varchar2(400)
);

create table test3 (
test3_id number primary key,
parent_id number,
test_id number references test,
data number
);

Can anybody think any way I can improve these?

Thanks,

Kevin
Jul 19 '05 #1
2 6793
"Kevin Crosbie" <ca**************@yahoo.com> wrote in message news:<48******************************@news.terane ws.com>...
Hi all,
(Sorry for the cross-post, there doesn't appear to be much activity on
comp.database.oracle)I'm trying to get the last 300 rows from the difference
between 2 large
tables and my queries are taking at least 10 minutes to do this. []
Can anybody think any way I can improve these?

Thanks,

Kevin


you picked another defunct group. Post one last time to
comp.databases.oracle.misc
and you'll get some suggestions.

I'll give you one quick comment here: the phrase "the last 300 rows"
is meaningless in a relational database. This isn't a file system,
rows have no location relative to each other. You'll do better to
eliminate such phrases from your vocabulary and describle what you
really mean or need.

See you in .misc
Ed
Jul 19 '05 #2
Thanks for the advice Ed,

I've decided to go a different route anyway, I'll do asynchronous calls
between my applications rather than send to databases and read again.

Thanks.
"Ed prochak" <ed********@magicinterface.com> wrote in message
news:4b**************************@posting.google.c om...
"Kevin Crosbie" <ca**************@yahoo.com> wrote in message

news:<48******************************@news.terane ws.com>...
Hi all,
(Sorry for the cross-post, there doesn't appear to be much activity on
comp.database.oracle)I'm trying to get the last 300 rows from the difference between 2 large
tables and my queries are taking at least 10 minutes to do this.

[]

Can anybody think any way I can improve these?

Thanks,

Kevin


you picked another defunct group. Post one last time to
comp.databases.oracle.misc
and you'll get some suggestions.

I'll give you one quick comment here: the phrase "the last 300 rows"
is meaningless in a relational database. This isn't a file system,
rows have no location relative to each other. You'll do better to
eliminate such phrases from your vocabulary and describle what you
really mean or need.

See you in .misc
Ed

Jul 19 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Good Man | last post by:
Hi there I'm developing a large web application. Part of this web application will be storing numerical chart data in a MySQL table - these numbers will be already calculated, and are just being...
57
by: Bing Wu | last post by:
Hi all, I am running a database containing large datasets: frames: 20 thousand rows, coordinates: 170 million row. The database has been implemented with: IBM DB2 v8.1
6
by: aaj | last post by:
Hi all I use a data adapter to read numerous tables in to a dataset. The dataset holds tables which in turn holds full details of the records i.e. keys, extra colums etc.. In some cases I...
1
by: sneha123 | last post by:
There will be some 20 questions and for each question there will be 4 choices.what i want to do is to select multiple answers by clicking the checkbox. i m using asp.net,vb.net pls help me we...
11
by: Mark Cubitt | last post by:
the reason I ask is I have always used postgres at home and work, but my new web host only has mysql :( I want to know what sort of differences I will have in regards to programming php/perl...
2
by: Kevin Crosbie | last post by:
Hi all, (Sorry for the cross-post, there doesn't appear to be much activity on comp.database.oracle)I'm trying to get the last 300 rows from the difference between 2 large tables and my queries...
7
by: Karol R | last post by:
Can anyone please explain me main differences between relational DB and warehouse (Point me to web site) ? Apart from theoretical differences I would like to know how Warehouse DB is updated ?...
25
by: tekctrl | last post by:
Anyone: I have a simple MSAccess DB which was created from an old ASCII flatfile. It works fine except for something that just started happening. I'll enter info in a record, save the record,...
10
by: nflacco | last post by:
I'm tinkering around with a data collection system, and have come up with a very hackish way to store my data- for reference, I'm anticipating collecting at least 100 million different dataId...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.