473,809 Members | 2,775 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Memory exhausted on DELETE.

I have a table with about 1,400,000 rows in it. Each DELETE cascades to
about 7 tables. When I do a 'DELETE FROM events' I get the following
error:

ERROR: Memory exhausted in AllocSetAlloc(8 4)

I'm running a default install. What postgres options to I need
to tweak to get this delete to work?

Also, if my tables grows to 30,000,000 rows will the same tweaks
still work? Or do I have to use a different delete strategy, such
as deleting 1000 rows at a time.

Thanks.

Nov 23 '05 #1
1 2502
<jb****@yahoo.c om> writes:
I have a table with about 1,400,000 rows in it. Each DELETE cascades to
about 7 tables. When I do a 'DELETE FROM events' I get the following
error: ERROR: Memory exhausted in AllocSetAlloc(8 4) I'm running a default install. What postgres options to I need
to tweak to get this delete to work?
It isn't a Postgres tweak. The only way you can fix it is to allow the
backend process to grow larger, which means increasing the kernel limits
on process data size. This might be as easy as tweaking "ulimit" in the
postmaster's environment, or it might be painful, depending on your OS.
You might also have to increase swap space.

There's a TODO item to allow the list of pending trigger events (which
is, I believe, what's killing you) to be pushed out to temp files when
it gets too big. However, that will have negative performance
implications of its own, so...
Also, if my tables grows to 30,000,000 rows will the same tweaks
still work? Or do I have to use a different delete strategy, such
as deleting 1000 rows at a time.


On the whole, deleting a few thousand rows at a time might be your best
bet.

BTW: make sure you have indexes on the referencing columns, or this will
take a REALLY long time.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to ma*******@postg resql.org)

Nov 23 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
12371
by: Paul | last post by:
Hi, I'm using TikiWiki and was using its file upload feature, trying to upload a 2MB file. I get this error: Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 1677706 bytes) in /var/html/www/tiki/lib/filegals/filegallib.php on line 30 Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to
5
2216
by: Aaron Birkland | last post by:
I have a long but straightforward query (on some very large tables) that always ends in 'Memory exhausted in AllocSetAlloc(108)'. Even stranger are some messages that appear in the logfile, such as the following (edited for length, repetitions, etc): TopMemoryContext: 32792 total in 4 blocks; 9712 free (2 chunks); 23080 used TopTransactionContext: 8192 total in 1 blocks; 8176 free (0 chunks); 16 used DeferredTriggerXact: 0 total in 0...
0
1916
by: Mattia | last post by:
************************************************** Manage image without exhausted memory ************************************************** Hi; I have a big problem. I must create a script that upload an image an then resize it, if width or height are more than 250px. Now, after upload an image (in this example I suppose that it's a JPEG image):
3
1290
by: cmills28 | last post by:
Hi. I'm developing a php 4.3.10 script, extracting data from webpages. Using string functions (strpos, substr) alot, some arrays (both static & dynamic) also data is going to a MySQL 5 DB. Am using mysql_free_result() function to free memory, but I'm still having memory problems -- keep getting the memory exhausted error. I'm developing this on Windows XP, but the live site is on Linux. Would this problem be a glitch in my code, in...
1
2978
by: Kimmo Laine | last post by:
Hi! We've encountered a strange problem concerning memory usage. In the previous install the maximum memory amount per page was limited to 8 MB and it was never reached. Now, after upgrading both hardware and software, running the same scripts that used to be fine with 8 MB hit the roof all the time. The maximum memory allocation was set to 128 MB and still PHP encounters fatal errors with allowed memory size exhausted. Pardon my french,...
2
4145
by: Sendil kumar | last post by:
Hi All, My application calls some java functionalities using C++. While running, my application crashes after 1 hour. The reason it saya is "Java: Heap memory exhausted", it couldn't allocate heap memory for further operations as it was exhausted. I think, java uses garbage collector to release memory when ever the objects goes out of scope. I even tried to catch the exception,but, It was not caught. Could any one give me solutions to catch...
5
24841
by: kumarmdb2 | last post by:
Hi guys, For last few days we are getting out of private memory error. We have a development environment. We tried to figure out the problem but we believe that it might be related to the OS (I am new to Windows so not sure). We are currently bouncing the instance to overcome this error. This generally happen at the end of business day only (So maybe memory might be getting used up?). We have already increased the statement heap & ...
1
2939
by: RYKLOU | last post by:
I am kinda new to php, but i do know what i am doing kinda, but i came across this error when i am trying to upload a file to my website. Fatal error: Allowed memory size of 8388608 bytes exhausted (tried to allocate 3714000 bytes) in /opt/lampp/htdocs/tutorials/php-mysql-tutorial/admin/image-gallery/library/functions.php on line 104 Platform: Ubuntu 8.04 LST (where i make my programs, and test them before i upload them), Using XAMPP for...
3
3237
by: jitendrarathod | last post by:
Hello, I have a one PHP script which crawled the URLs and get the job from the URLs. My logic is so simple. First take the URL from database(total 25000 URLs). Then start to crawl the URLs. take the job and check in the database that job exist in database or not. If existed then update the visited date for that job. otherwise add new job record in database. Currently our database has 11422185 job records. And all are in one table. but...
0
9721
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9601
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10637
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9199
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
6881
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5550
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
4332
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
3861
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3014
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.