473,703 Members | 3,832 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

correlated delete with "in" and "left outer join"

I'm using postgresl 7.3.2 and have a query that executes very slowly.

There are 2 tables: Item and LogEvent. ItemID (an int4) is the primary key
of Item, and is also a field in LogEvent. Some ItemIDs in LogEvent do not
correspond to ItemIDs in Item, and periodically we need to purge the
non-matching ItemIDs from LogEvent.

The query is:

delete from LogEvent where EventType != 'i' and ItemID in
(select distinct e.ItemID from LogEvent e left outer join Item i
on e.ItemID = i.ItemID where e.EventType != 'i' and i.ItemID is null);

I understand that using "in" is not very efficient.

Is there some other way to write this query without the "in"?

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to ma*******@postg resql.org)

Nov 22 '05 #1
14 5712
On Feb 27, 2004, at 11:26 AM, <mi**@linkify.c om> wrote:
I'm using postgresl 7.3.2 and have a query that executes very slowly.
<snip />
I understand that using "in" is not very efficient.

Is there some other way to write this query without the "in"?


NOT EXISTS ( ) is sometimes more efficient. If at all possible, upgrade
to 7.4.1. One of the many things that have improved since 7.3.2 is the
efficiency of queries using IN.

Michael Glaesemann
grzm myrealbox com
---------------------------(end of broadcast)---------------------------
TIP 8: explain analyze is your friend

Nov 22 '05 #2
On Thu, 26 Feb 2004 mi**@linkify.co m wrote:
I'm using postgresl 7.3.2 and have a query that executes very slowly.

There are 2 tables: Item and LogEvent. ItemID (an int4) is the primary key
of Item, and is also a field in LogEvent. Some ItemIDs in LogEvent do not
correspond to ItemIDs in Item, and periodically we need to purge the
non-matching ItemIDs from LogEvent.

The query is:

delete from LogEvent where EventType != 'i' and ItemID in
(select distinct e.ItemID from LogEvent e left outer join Item i
on e.ItemID = i.ItemID where e.EventType != 'i' and i.ItemID is null);

I understand that using "in" is not very efficient.

Is there some other way to write this query without the "in"?


Perhaps
delete from LogEvent where EventType != 'i' and not exists
(select * from Item i where i.ItemID=LogEve nt.ItemID);

---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

Nov 22 '05 #3
Stephan Szabo wrote:
On Thu, 26 Feb 2004 mi**@linkify.co m wrote:

I'm using postgresl 7.3.2 and have a query that executes very slowly.

There are 2 tables: Item and LogEvent. ItemID (an int4) is the primary key
of Item, and is also a field in LogEvent. Some ItemIDs in LogEvent do not
correspond to ItemIDs in Item, and periodically we need to purge the
non-matching ItemIDs from LogEvent.

The query is:

delete from LogEvent where EventType != 'i' and ItemID in
(select distinct e.ItemID from LogEvent e left outer join Item i
on e.ItemID = i.ItemID where e.EventType != 'i' and i.ItemID is null);

I understand that using "in" is not very efficient.

Is there some other way to write this query without the "in"?

Perhaps
delete from LogEvent where EventType != 'i' and not exists
(select * from Item i where i.ItemID=LogEve nt.ItemID);


Maybe I'm not reading his subquery correctly, but the left outer
join will produce a row from LogEvent regardless of whether or not a
matching row exists in Item, correct? So doesn't it reduce to:

DELETE FROM LogEvent WHERE EventType <> 'i';

???

Mike Mascari

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postg resql.org

Nov 22 '05 #4
Mike Mascari wrote:
Stephan Szabo wrote:
On Thu, 26 Feb 2004 mi**@linkify.co m wrote:
I'm using postgresl 7.3.2 and have a query that executes very slowly.

There are 2 tables: Item and LogEvent. ItemID (an int4) is the
primary key
of Item, and is also a field in LogEvent. Some ItemIDs in LogEvent
do not
correspond to ItemIDs in Item, and periodically we need to purge the
non-matching ItemIDs from LogEvent.


Perhaps
delete from LogEvent where EventType != 'i' and not exists
(select * from Item i where i.ItemID=LogEve nt.ItemID);

Maybe I'm not reading his subquery correctly, but the left outer join
will produce a row from LogEvent regardless of whether or not a matching
row exists in Item, correct? So doesn't it reduce to:

DELETE FROM LogEvent WHERE EventType <> 'i';


I failed to read what he was trying to accomplish and assumed the
original query was precisely what he intended. My apologies...

Mike Mascari

---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 22 '05 #5
The subquery will always return a row from LogEvent, but that row's itemID
will be null if theitemID doesn't match a row from Item. That's why the subquery has the
"and i.ItemID is null".
Stephan Szabo wrote:
On Thu, 26 Feb 2004 mi**@linkify.co m wrote:

I'm using postgresl 7.3.2 and have a query that executes very slowly.

There are 2 tables: Item and LogEvent. ItemID (an int4) is the
primary key of Item, and is also a field in LogEvent. Some ItemIDs in
LogEvent do not correspond to ItemIDs in Item, and periodically we
need to purge the non-matching ItemIDs from LogEvent.

The query is:

delete from LogEvent where EventType != 'i' and ItemID in
(select distinct e.ItemID from LogEvent e left outer join Item i on
e.ItemID = i.ItemID where e.EventType != 'i' and i.ItemID is null);

I understand that using "in" is not very efficient.

Is there some other way to write this query without the "in"?

Perhaps
delete from LogEvent where EventType != 'i' and not exists
(select * from Item i where i.ItemID=LogEve nt.ItemID);


Maybe I'm not reading his subquery correctly, but the left outer
join will produce a row from LogEvent regardless of whether or not a
matching row exists in Item, correct? So doesn't it reduce to:

DELETE FROM LogEvent WHERE EventType <> 'i';

???

Mike Mascari


---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 22 '05 #6
mi**@linkify.co m wrote:
The subquery will always return a row from LogEvent, but that row's itemID
will be null if the itemID doesn't match a row from Item.
That's why the subquery has the "and i.ItemID is null".


You lost me.

[test@lexus] \d foo
Table "public.foo "
Column | Type | Modifiers
--------+---------+-----------
key | integer |

[test@lexus] \d bar
Table "public.bar "
Column | Type | Modifiers
--------+---------+-----------
key | integer |
value | text |

[test@lexus] select * from foo;
key
-----
1
3
(2 rows)

[test@lexus] select * from bar;
key | value
-----+-------
1 | Mike
2 | Joe
(2 rows)

[test@lexus] select f.key from foo f left outer join bar b on f.key
= b.key and b.key is null;
key
-----
1
3
(2 rows)

To do what I think you believe to be happening w.r.t. outer joins,
you'd have to have a subquery like:

[test@lexus] select a.fookey
test-# FROM
test-# (SELECT foo.key AS fookey, bar.key as barkey FROM foo LEFT
OUTER JOIN bar ON foo.key = bar.key) AS a
test-# WHERE a.barkey IS NULL;
fookey
--------
3
(1 row)

Nevertheless, Stephan's solution matches your description of the
problem and excutes the logical equivalent of the above much more
rapidly...

Mike Mascari



---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 22 '05 #7
On Thu, Feb 26, 2004 at 06:26:19PM -0800, mi**@linkify.co m wrote:
I'm using postgresl 7.3.2 and have a query that executes very slowly.

There are 2 tables: Item and LogEvent. ItemID (an int4) is the
primary key
of Item, and is also a field in LogEvent. Some ItemIDs in LogEvent do
not
correspond to ItemIDs in Item, and periodically we need to purge the
non-matching ItemIDs from LogEvent.


delete from LogEvent where EventType!='i' and
ItemID not in (select ItemID from Item);

delete from LogEvent where EventType!='i' and
not exists (select * from Item where Item.ItemID=Log Event.ItemID);

You might also use a foreign key, cascading delete, etc. As for the
query style, I've had cases with the latest 7.4 where the "in" style
wasn't optimized but the "exists" style was. It's the exact same query,
and technically the optimizer should figure that out. Use "explain" to
see if it's being optimized to use indexes or if it's just doing table
scans.

Michael
--
Michael Darrin Chaney
md******@michae lchaney.com
http://www.michaelchaney.com/

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 22 '05 #8

On Fri, 27 Feb 2004, Mike Mascari wrote:
To do what I think you believe to be happening w.r.t. outer joins,
you'd have to have a subquery like:

[test@lexus] select a.fookey
test-# FROM
test-# (SELECT foo.key AS fookey, bar.key as barkey FROM foo LEFT
OUTER JOIN bar ON foo.key = bar.key) AS a
test-# WHERE a.barkey IS NULL;


This AFAICS is pretty much what he did, except that he didn't alias the
join which is okay I believe. He had one condition in on and two
conditions in where.

The original subquery looked like:
select distinct e.ItemID from LogEvent e left outer join Item i
on e.ItemID = i.ItemID where e.EventType != 'i' and i.ItemID is null
---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 22 '05 #9
Stephan Szabo wrote:
On Fri, 27 Feb 2004, Mike Mascari wrote:
To do what I think you believe to be happening w.r.t. outer joins,
you'd have to have a subquery like:

[test@lexus] select a.fookey
test-# FROM
test-# (SELECT foo.key AS fookey, bar.key as barkey FROM foo LEFT
OUTER JOIN bar ON foo.key = bar.key) AS a
test-# WHERE a.barkey IS NULL;


This AFAICS is pretty much what he did, except that he didn't alias the
join which is okay I believe. He had one condition in on and two
conditions in where.

The original subquery looked like:
select distinct e.ItemID from LogEvent e left outer join Item i
on e.ItemID = i.ItemID where e.EventType != 'i' and i.ItemID is null


That is indeed the original subquery. But the 'i.ItemID is null'
condition doesn't change the IN list one iota. He was somehow
expecting the subquery to yield records internally like:

1 NULL
2 NULL
3 3

and simultaneously have the condition 'i.ItemID is null' eliminate
the third tuple. But that is not how the left outer join executes.
The 'i.ItemID is null' condition is evaluated, probably always to
false, which ensures that the left outer join will never find a
matching row from the 'Item' relation and, if queried not as a
subquery but stand-alone as:

select distinct e.ItemID, i.ItemID
from LogEvent e left outer join Item i on e.ItemID = i.ItemID
where e.EventType != 'i' and i.ItemID is null

would always yield a relation of the form:

e.ItemID NULL

for every e.ItemID whose e.EventType != 'i'. That ain't right.

Another example:

[test@lexus] select * from foo;
key
-----
1
3
(2 rows)

[test@lexus] select * from bar;
key | value
-----+-------
1 | Mike
2 | Joe
(2 rows)

[test@lexus] select foo.key, bar.key from foo left outer join bar on
foo.key = bar.key and bar.key is null;
key | key
-----+-----
1 |
3 |
(2 rows)

[test@lexus] select foo.key, bar.key from foo left outer join bar on
foo.key = bar.key;
key | key
-----+-----
1 | 1
3 |
(2 rows)

[test@lexus] select a.fookey, a.barkey from (select foo.key as
fookey, bar.key as barkey from foo left outer join bar on foo.key =
bar.key) as a where a.barkey is null;
fookey | barkey
--------+--------
3 |
(1 row)
Mike Mascari
---------------------------(end of broadcast)---------------------------
TIP 7: don't forget to increase your free space map settings

Nov 22 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
10052
by: Dam | last post by:
Using SqlServer : Query 1 : SELECT def.lID as IdDefinition, TDC_AUneValeur.VALEURDERETOUR as ValeurDeRetour FROM serveur.Data_tblDEFINITIONTABLEDECODES def, serveur.Data_tblTABLEDECODEAUNEVALEUR TDC_AUneValeur where def.TYPEDETABLEDECODES = 4
1
4214
by: Steve | last post by:
I have a SQL query I'm invoking via VB6 & ADO 2.8, that requires three "Left Outer Joins" in order to return every transaction for a specific set of criteria. Using three "Left Outer Joins" slows the system down considerably. I've tried creating a temp db, but I can't figure out how to execute two select commands. (It throws the exception "The column prefix 'tempdb' does not match with a table name or alias name used in the query.")
7
31560
by: Steve | last post by:
I have a SQL query I'm invoking via VB6 & ADO 2.8, that requires three "Left Outer Joins" in order to return every transaction for a specific set of criteria. Using three "Left Outer Joins" slows the system down considerably. I've tried creating a temp db, but I can't figure out how to execute two select commands. (It throws the exception "The column prefix 'tempdb' does not match with a table name or alias name used in the query.")
3
19476
by: Martin | last post by:
Hello everybody, I have the following question. As a join clause on Oracle we use " table1.field1 = table2.field1 (+) " On SQL Server we use " table1.field1 *= table2.field1 " Does DB2 have the same type of operator, without using the OUTER JOIN syntax ?
77
4006
by: M.B | last post by:
Guys, Need some of your opinion on an oft beaten track We have an option of using "goto" in C language, but most testbooks (even K&R) advice against use of it. My personal experience was that goto sometimes makes program some more cleaner and easy to understand and also quite useful (in error handling cases). So why goto is outlawed from civilized c programmers community. is there any technical inefficiency in that.
0
1558
by: SKB | last post by:
Hi, I want to implement the "outer join" functionality in Java. Can somebody explain the pseudo code for the same. OR what needs to be done to extend the hash-join Java code of equijoin(I have the hash join code for equijoin with me). Regds
4
2324
by: saniac | last post by:
I am working on a little project using pysqlite. It's going to be exposed on the web, so I want to make sure I quote all incoming data correctly. However, I've run into a brick wall trying to use parameters to populate a query of the form "select * where col1 in ( ? )" The naive approach doesn't work: values= sql = """select * where value in (?)""" cu = cx.cursor()
3
3237
by: Kevin Chambers | last post by:
Hi all-- Quick question: has anyone come up with an easy way to take an array and use its elements as part of a WHERE clause? For example: <This obviously doesn't work> SELECT * FROM Table1 WHERE Field1 IN Array(1,2,3) The only solution I can think of is putting the values in a temp table
1
3275
by: nico3334 | last post by:
I have a query that currently pulls data from a main table and a second table using LEFT OUTER JOIN. I know how to do make another LEFT OUTER JOIN with the main table, but I want to add another LEFT OUTER JOIN to the second table. So I want the third table to be joined through the second table, not the main table. Here is my original code that joins the main table and the second table SELECT t1.supply, t2.inventory, FROM MAIN_TABLE...
0
9262
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9125
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8970
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7876
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6595
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5923
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4690
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
2463
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2070
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.