473,594 Members | 2,839 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

postmaster out of memory....

Hello all!

when asking postgres to aggregate totals accross 4.5 or so Million records. The visible effect is that the postmaster will grow to the 3GB process limit and die without a core :(.
I have seen this same behaviour discussed back in 6.5 archives in the thread with subject "[SQL] How to avoid "Out of memory" using aggregate functions? ". Is this fixed? Why is the postmaster exceeding it's 102MB sort mem size when doing these queries and not paging out the data?

-Joe Maldonado

--
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Nov 23 '05 #1
1 2258
On Fri, Feb 27, 2004 at 01:45:07PM -0500, Joe Maldonado wrote:
Hello all!

when asking postgres to aggregate totals accross 4.5 or so Million
records. The visible effect is that the postmaster will grow to the
3GB process limit and die without a core :(.
I have seen this same behaviour discussed back in 6.5 archives in the
thread with subject "[SQL] How to avoid "Out of memory" using aggregate
functions? ". Is this fixed? Why is the postmaster exceeding it's 102MB
sort mem size when doing these queries and not paging out the data?
What version? We aggregate million of value vairly often. What
aggregate?

Have you run ANALYZE recently?
--
Martijn van Oosterhout <kl*****@svana. org> http://svana.org/kleptog/ If the Catholic church can survive the printing press, science fiction
will certainly weather the advent of bookwarez.
http://craphound.com/ebooksneitherenorbooks.txt - Cory Doctorow


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQFAQRUwY5T wig3Ge+YRAvwCAJ 0bZWxQ2iwvBAfxN +RzERyHCV1apgCg xq6m
NAAaaFgdyVWoxnX c0kgisEw=
=+Gr3
-----END PGP SIGNATURE-----

Nov 23 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
5879
by: Chris Webster | last post by:
I have one process which writes a single float into 300 columns once per second. I then run 4 process, from remote computers, to query a small subset of the latest row. I have even commented out everything in the query programs, all they do is sleep, and the associated postmaster still sucks up 15% - 20% CPU. Computer is a P4 /w 1Gig memory, all disk access is local. RH9 /w stock postgresql-7.3 installed.
2
1536
by: psql-mail | last post by:
Help! Postmaster is segfaulting i think. I am running RH_AS_3beta, postgresql 7.3.4 compiled from postgresql-7.3. 4-1PGDG.src.rpm on ia64 arch. Tsearch2 compiled from tsearch-v2-stable.tar.gz I have no idea how to fix it, all help gratefully received! Potentially useful output below:
2
2109
by: Andy Harrison | last post by:
-----BEGIN PGP SIGNED MESSAGE----- Could anyone explain about the max_connections in postgres? We ran into the connection limit on one of our production servers. One reason is that we have more apps that use postgres running on it (RT3 and Webcollab were added not long ago). Webcollab, for instance, seems to keep 7 postmaster processes all to itself. Is this normal? Does postgres work similarly to how apache handles spawning spare...
0
2406
by: Jeremiah Elliott | last post by:
I have a java application that pulls data off db2/400 an inserts it into a postgres(7.3.2) database. The jdbc errors out with this message: ConnectionBean: Driver not loaded Something unusual has occured to cause the driver to fail. Please report this exception: Exception: java.sql.SQLException: FATAL: The database system is starting up Stack Trace: java.sql.SQLException: FATAL: The database system is starting up
0
1374
by: Melanie Bergeron | last post by:
Hi all! I have a big problem installing postmaster as a service under Windows XP with cygwin. I already installed postgresql on 2 machines without problems but this machine really seem to don't like postgresql ;) I do exactly the procedure indicated in the README file but I always got the same problem with this machine. When I try to start postmaster who was previously installed as a service with the command , I always got the same...
8
3152
by: Lincoln Yeoh | last post by:
Hi, I recently upgraded to postgresql 7.4 and I am having a problem with postmaster using lots of memory for a query (keeps growing even up to 400MB+ till I stop postgresql ). I don't recall this ever happening with 7.3 with the exact same query but on different data (just as much data tho, or even more). Happened on 7.4 so I upgraded to 7.4.1. Is HashAggregate something new in 7.4? If I have time and disk space I'll downgrade to 7.3.3...
2
1537
by: Lincoln Yeoh | last post by:
Sorry for the repost. Just wondering if anyone has a workaround for 7.4? At 11:25 AM 1/27/2004 -0500, Tom Lane wrote: >Martijn van Oosterhout <kleptog@svana.org> writes: > > I'm afraid I'll have to defer to someone else (Tom?) as why the > estimate was > > out by three orders of magnitude. > >I'd like to know that, too. >
8
1717
by: Thomas Hallgren | last post by:
I'm writing a small test harness. I have two threads. One that starts the postmaster and another that does all the testing and finally stops the postmaster with a pg_ctl stop. At present, the second thread starts with a sleep sufficient to ensure that the postmaster is running. Is there a proper way to test when the postmaster is ready to receive commands? Kind regards, Thomas Hallgren
2
5904
by: Gregory S. Williamson | last post by:
Dear peoples, We had an oddness today with one of of postgres servers (Dell 2 CPU box running linux) and postgres 7.4. The server was under heavy load (50+ for a 1 minutes spike; about 20 for the 15 minute average) with about 250 connections (we still don't understand the heavy load itself). Looking in the logs I see: 2004-11-13 13:30:28 LOG: unexpected EOF on client connection 2004-11-13 13:30:40 LOG: unexpected EOF on client...
0
7946
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
7877
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
8253
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8240
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
5739
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
3867
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
3903
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1482
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
0
1216
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.