473,624 Members | 2,261 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

selecting the differences between 2 large tables

Hi all,
(Sorry for the cross-post, there doesn't appear to be much activity on
comp.database.o racle)I'm trying to get the last 300 rows from the difference
between 2 large
tables and my queries are taking at least 10 minutes to do this.

I'm running on quite a fast server and the tables are not very large,
3,000,000 to 30,000,000 rows.

I've tried the following:
(test is the primary key table and test2 and test3 have foreign keys to test
also, imagine I have the following unique indexes (test_id, sequence_no) and
(parent_id, test_id))

select test_id
from (select test_id from test2 where sequence_no = 0
minus
select test_id from test3 where parent_id = 581)
where rownum < 300

Explain Plan:
SELECT STATEMENT Optimizer=CHOOS E
COUNT (STOPKEY)
VIEW
MINUS
SORT (UNIQUE)
TABLE ACCESS (FULL) OF TEST2
SORT (UNIQUE)
INDEX (RANGE SCAN) OF TEST3_PAR_ID_TE ST_ID_UNQ (UNIQUE)

second approach:

select test_id
from test2 t2, (select test_id from test3 where parent_id = 581) t3
where t2.test_id = t3.test_id(+)
and t2.sequence_no = 0
and t3.test_id is null
and rownum < 300

Explain Plan:
SELECT STATEMENT Optimizer=CHOOS E
COUNT (STOPKEY)
FILTER
NESTED LOOPS (OUTER)
TABLE ACCESS (FULL) OF TEST2
INDEX (UNIQUE SCAN) OF TEST3_PAR_ID_TE ST_ID_UNQ (UNIQUE)

The table creates for the above tables are:
create table test (
test_id number primary key
);

create table test2 (
test2_id number primary key,
test_id number references test,
sequence_no number,
test_text varchar2(400)
);

create table test3 (
test3_id number primary key,
parent_id number,
test_id number references test,
data number
);

Can anybody think any way I can improve these?

Thanks,

Kevin
Jul 19 '05 #1
2 6811
"Kevin Crosbie" <ca************ **@yahoo.com> wrote in message news:<48******* *************** ********@news.t eranews.com>...
Hi all,
(Sorry for the cross-post, there doesn't appear to be much activity on
comp.database.o racle)I'm trying to get the last 300 rows from the difference
between 2 large
tables and my queries are taking at least 10 minutes to do this. []
Can anybody think any way I can improve these?

Thanks,

Kevin


you picked another defunct group. Post one last time to
comp.databases. oracle.misc
and you'll get some suggestions.

I'll give you one quick comment here: the phrase "the last 300 rows"
is meaningless in a relational database. This isn't a file system,
rows have no location relative to each other. You'll do better to
eliminate such phrases from your vocabulary and describle what you
really mean or need.

See you in .misc
Ed
Jul 19 '05 #2
Thanks for the advice Ed,

I've decided to go a different route anyway, I'll do asynchronous calls
between my applications rather than send to databases and read again.

Thanks.
"Ed prochak" <ed********@mag icinterface.com > wrote in message
news:4b******** *************** ***@posting.goo gle.com...
"Kevin Crosbie" <ca************ **@yahoo.com> wrote in message

news:<48******* *************** ********@news.t eranews.com>...
Hi all,
(Sorry for the cross-post, there doesn't appear to be much activity on
comp.database.o racle)I'm trying to get the last 300 rows from the difference between 2 large
tables and my queries are taking at least 10 minutes to do this.

[]

Can anybody think any way I can improve these?

Thanks,

Kevin


you picked another defunct group. Post one last time to
comp.databases. oracle.misc
and you'll get some suggestions.

I'll give you one quick comment here: the phrase "the last 300 rows"
is meaningless in a relational database. This isn't a file system,
rows have no location relative to each other. You'll do better to
eliminate such phrases from your vocabulary and describle what you
really mean or need.

See you in .misc
Ed

Jul 19 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
6196
by: Good Man | last post by:
Hi there I'm developing a large web application. Part of this web application will be storing numerical chart data in a MySQL table - these numbers will be already calculated, and are just being stored for reference. In this particular table, the stored data will never be deleted or changed. The only actions performed will be SELECTs and INSERTs. There will never be any DELETEs or UPDATEs.
57
25499
by: Bing Wu | last post by:
Hi all, I am running a database containing large datasets: frames: 20 thousand rows, coordinates: 170 million row. The database has been implemented with: IBM DB2 v8.1
6
2051
by: aaj | last post by:
Hi all I use a data adapter to read numerous tables in to a dataset. The dataset holds tables which in turn holds full details of the records i.e. keys, extra colums etc.. In some cases I need to use parts of the tables in datagrids, and here is where my problem lies
1
2911
by: sneha123 | last post by:
There will be some 20 questions and for each question there will be 4 choices.what i want to do is to select multiple answers by clicking the checkbox. i m using asp.net,vb.net pls help me we have written the code using radio button for selecting single item.but we want to replace it with checkbox to select multiple items. the code using radio button is given below .pls correct it with checkbox
11
9934
by: Mark Cubitt | last post by:
the reason I ask is I have always used postgres at home and work, but my new web host only has mysql :( I want to know what sort of differences I will have in regards to programming php/perl webpages using the database. I current use sequences in postgres and understand this will be different in mysql, so can you tell me what the difference will be and if there are any other differences I should look out for.
2
159
by: Kevin Crosbie | last post by:
Hi all, (Sorry for the cross-post, there doesn't appear to be much activity on comp.database.oracle)I'm trying to get the last 300 rows from the difference between 2 large tables and my queries are taking at least 10 minutes to do this. I'm running on quite a fast server and the tables are not very large, 3,000,000 to 30,000,000 rows. I've tried the following:
7
5567
by: Karol R | last post by:
Can anyone please explain me main differences between relational DB and warehouse (Point me to web site) ? Apart from theoretical differences I would like to know how Warehouse DB is updated ? If data needs to remain unchanged then what is happening for instance if in source relational DB address for a customer will change and record gets the update. How this is loaded to DB ?
25
20544
by: tekctrl | last post by:
Anyone: I have a simple MSAccess DB which was created from an old ASCII flatfile. It works fine except for something that just started happening. I'll enter info in a record, save the record, and try to move to another record and get an Access error "Record is too large". The record is only half filled, with many empty fields. If I remove the added data or delete some older data, then it saves ok and works fine again. Whenever I'm...
10
2856
by: nflacco | last post by:
I'm tinkering around with a data collection system, and have come up with a very hackish way to store my data- for reference, I'm anticipating collecting at least 100 million different dataId whatevers per year, possibly much more. ---366 data tables ( one for each day of the year ), each row being assigned a unique DataId ( unique across all 366 tables too ) ---100 data_map tables, table 0 having all DataIds ending in 00, table 99...
0
8233
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8675
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
8334
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8474
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7158
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6108
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5561
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
1
2604
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1784
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.