473,847 Members | 1,500 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Delete very slow after deletion of many rows in dependent table

Hi,

we saw the following problem:

We deleted all rows from a table B referencing table A (~500000 records). No
problem, but the following try to delete all records from table A (~180000) lead
to a "never ending" statement. We found out, that vacuuming table B after delete
did the trick.

It seems to us the database has to do scan thru deleted records on B while
deleting from A. Why did it last so long? An index on B.a_fk did not lead to
imporvements. The query plan did not help.

Could anybody explain the problem to me? Does anybody know a different solution
than vacuuming the table after big deletes? (We already vaccuum the database
periodically.)

Below the statements to reproduce the problem.

Regards
Cornelius

=============== =============== =============== ===============
-- create two tables, b references a through a_fk
create table table_a ( a_pk int8 primary key );

create table table_b ( b_pk int8 primary key, a_fk int8 not null references
table_a ( a_pk ) );

-- fill a and b with 100000 records
create or replace function fill_table_a() returns int8 as '
begin
for i in 1..100000 loop
insert into table_a values ( i );
end loop;
return 1;
end' language plpgsql;

select fill_table_a();

insert into table_b ( select a_pk, a_pk from table_a );

commit;

-- delete records from b, so records from a can be also be deleted
delete from table_b;
commit;

-- delete records from a;
-- this delete needs a VERY long time
delete from table_a;
-- we do it again, but vacuum table_b first

-- delete records from b, so records from a can be also be deleted
vacuum table_b;

-- delete records from a;
delete from table_a;
commit;

--
_______________ _______________ _______________ ___________

Cornelius Buschka

arcus(x) GmbH
Hein-Hoyer-Straße 75 fon: +49 (0)40.333 102 92
D-20359 Hamburg fax: +49 (0)40.333 102 93

http://www.arcusx.com c.buschka AT arcusx DOT com
_______________ _______________ _______________ ___________


---------------------------(end of broadcast)---------------------------
TIP 9: the planner will ignore your desire to choose an index scan if your
joining column's datatypes do not match

Nov 23 '05 #1
2 8854

On Sun, 21 Nov 2004, Cornelius Buschka wrote:
Hi,

we saw the following problem:

We deleted all rows from a table B referencing table A (~500000 records). No
problem, but the following try to delete all records from table A (~180000) lead
to a "never ending" statement. We found out, that vacuuming table B after delete
did the trick.

It seems to us the database has to do scan thru deleted records on B while
deleting from A. Why did it last so long? An index on B.a_fk did not lead to
imporvements. The query plan did not help.


An index seems to help for me. It's still kinda slow, but the real time
for the delete on A goes from more minutes than I was willing to wait to
about 19s.

However, if you'd already run the key without the index, refilled the
table, made the index and tried it again, it probably wouldn't have used
the index because it tries to cache the plan on first use in each session
(you'd need to start a new session to try again).

---------------------------(end of broadcast)---------------------------
TIP 5: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faqs/FAQ.html

Nov 23 '05 #2
Hi Stephan,

caching of the execution plan is a good hint. We'll try it in a new connection.

Best Regards
Cornelius

Stephan Szabo wrote:
On Sun, 21 Nov 2004, Cornelius Buschka wrote:

Hi,

we saw the following problem:

We deleted all rows from a table B referencing table A (~500000 records). No
problem, but the following try to delete all records from table A (~180000) lead
to a "never ending" statement. We found out, that vacuuming table B after delete
did the trick.

It seems to us the database has to do scan thru deleted records on B while
deleting from A. Why did it last so long? An index on B.a_fk did not lead to
imporvement s. The query plan did not help.

An index seems to help for me. It's still kinda slow, but the real time
for the delete on A goes from more minutes than I was willing to wait to
about 19s.

However, if you'd already run the key without the index, refilled the
table, made the index and tried it again, it probably wouldn't have used
the index because it tries to cache the plan on first use in each session
(you'd need to start a new session to try again).

--
_______________ _______________ _______________ ___________

Cornelius Buschka

arcus(x) GmbH
Hein-Hoyer-Straße 75 fon: +49 (0)40.333 102 92
D-20359 Hamburg fax: +49 (0)40.333 102 93

http://www.arcusx.com mailto:c.****** *@arcusx.com
_______________ _______________ _______________ ___________
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to ma*******@postg resql.org)

Nov 23 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
1774
by: Vic | last post by:
Hi all, When I test the Delete multi table function in MySQL, DELETE table_name ...] FROM table-references I accidentally delete all data in one table. All data in that table are gone when I try to select them out in Control Center. But when I go into the /mysql/data/mydatabase/, I see a MYD, MYI, frm for that table. And it seems that data is still inside the MYD, althought it's
2
8809
by: Ryan | last post by:
I have a table in my database on SQL Server which holds a file name that refers to a file that is stored on the server. I would like to create a trigger to delete this file from the server if the row in the table is deleted. I have been trying to use this command in a trigger (<filename> is the name and path of the file): xp_cmdshell "delete <filename>" If some one could please help I would appreciate it very much. I would love a...
2
11806
by: Bob Ganger | last post by:
Hello, I am working on a project using SQL Server 2000 with a database containing about 10 related tables with a lot of columns containing text. The total current size of the database is about 2 Gig. When I delete data from the database, it takes a lot of system resources and monopolizes the database so that all other query requests are slow as mud! Ideally, I would like to be able to issue delete commands to the database on a...
9
10693
by: Robert Schneider | last post by:
Hi to all, I don't understand that: I try to delete a record via JDBC. But I always get the error SQL7008 with the error code 3. It seems that this has something to do with journaling, since the table from which I want to delete has two foreign keys that references two other tables and it is also referenced by another table. But this shouldn't be a problem, since I set the commit mode to none (or *none) at all places where this makes...
16
3884
by: robert | last post by:
been ruminating on the question (mostly in a 390/v7 context) of whether, and if so when, a row update becomes an insert/delete. i assume that there is a threshold on the number of columns of the table, or perhaps bytes, being updated where the engine just decides, screw it, i'll just make a new one. surfed this group and google, but couldn't find anything. the context: we have some java folk who like to parametize/
14
9664
by: Michel Esber | last post by:
Linux RH 4.0 running DB2 V8 FP 11. I have a table with ~ 11M rows and running DELETE statements is really slow. Deleting 1k rows takes more than 3 minutes. If I run select statements on the same table, I usually fetch rows in a reasonable time. The table has the following description: MACHINE_ID VARCHAR (24)
2
7688
by: nekiv90 | last post by:
Greetings, I have to delete older policies and its related records in other tables. The deletion from the parent table will trigger the deletion of relevant records from about 30 something tables that are defined with DELETE CASCADE. The delete SQL goes like this:
5
3116
by: rn5a | last post by:
The .NET 2.0 documentation states the following: When using a DataSet or DataTable in conjunction with a DataAdapter & a relational data source, use the Delete method of the DataRow to remove the row. The Delete method marks the row as Deleted in the DataSet or DataTable but does not remove it. Instead when the DataAdapter encounters a row marked as Deleted, it executes its DeleteCommand method to delete the row at the data source. The...
3
12350
by: Michel Esber | last post by:
Hello, Environment: DB2 LUW v8 FP15 / Linux I have a table with 50+ Million rows. The table structure is basically (ID - Timestamp). I have two main applications - one inserting rows, and the other reading/deleting rows. The 'deleter' application runs a MIN/MAX (timestamp) for each ID and, if the difference between min/max is greater than 1h, it reads all
0
9892
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9734
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10991
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10653
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
10347
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9490
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5915
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4540
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4129
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.