473,326 Members | 2,099 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,326 software developers and data experts.

postmaster out of memory....

Hello all!

when asking postgres to aggregate totals accross 4.5 or so Million records. The visible effect is that the postmaster will grow to the 3GB process limit and die without a core :(.
I have seen this same behaviour discussed back in 6.5 archives in the thread with subject "[SQL] How to avoid "Out of memory" using aggregate functions? ". Is this fixed? Why is the postmaster exceeding it's 102MB sort mem size when doing these queries and not paging out the data?

-Joe Maldonado

--
Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/

---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Nov 23 '05 #1
1 2241
On Fri, Feb 27, 2004 at 01:45:07PM -0500, Joe Maldonado wrote:
Hello all!

when asking postgres to aggregate totals accross 4.5 or so Million
records. The visible effect is that the postmaster will grow to the
3GB process limit and die without a core :(.
I have seen this same behaviour discussed back in 6.5 archives in the
thread with subject "[SQL] How to avoid "Out of memory" using aggregate
functions? ". Is this fixed? Why is the postmaster exceeding it's 102MB
sort mem size when doing these queries and not paging out the data?
What version? We aggregate million of value vairly often. What
aggregate?

Have you run ANALYZE recently?
--
Martijn van Oosterhout <kl*****@svana.org> http://svana.org/kleptog/ If the Catholic church can survive the printing press, science fiction
will certainly weather the advent of bookwarez.
http://craphound.com/ebooksneitherenorbooks.txt - Cory Doctorow


-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.6 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQFAQRUwY5Twig3Ge+YRAvwCAJ0bZWxQ2iwvBAfxN+RzER yHCV1apgCgxq6m
NAAaaFgdyVWoxnXc0kgisEw=
=+Gr3
-----END PGP SIGNATURE-----

Nov 23 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Chris Webster | last post by:
I have one process which writes a single float into 300 columns once per second. I then run 4 process, from remote computers, to query a small subset of the latest row. I have even commented...
2
by: psql-mail | last post by:
Help! Postmaster is segfaulting i think. I am running RH_AS_3beta, postgresql 7.3.4 compiled from postgresql-7.3. 4-1PGDG.src.rpm on ia64 arch. Tsearch2 compiled from tsearch-v2-stable.tar.gz ...
2
by: Andy Harrison | last post by:
-----BEGIN PGP SIGNED MESSAGE----- Could anyone explain about the max_connections in postgres? We ran into the connection limit on one of our production servers. One reason is that we have...
0
by: Jeremiah Elliott | last post by:
I have a java application that pulls data off db2/400 an inserts it into a postgres(7.3.2) database. The jdbc errors out with this message: ConnectionBean: Driver not loaded Something unusual has...
0
by: Melanie Bergeron | last post by:
Hi all! I have a big problem installing postmaster as a service under Windows XP with cygwin. I already installed postgresql on 2 machines without problems but this machine really seem to don't...
8
by: Lincoln Yeoh | last post by:
Hi, I recently upgraded to postgresql 7.4 and I am having a problem with postmaster using lots of memory for a query (keeps growing even up to 400MB+ till I stop postgresql ). I don't recall...
2
by: Lincoln Yeoh | last post by:
Sorry for the repost. Just wondering if anyone has a workaround for 7.4? At 11:25 AM 1/27/2004 -0500, Tom Lane wrote: >Martijn van Oosterhout <kleptog@svana.org> writes: > > I'm afraid I'll have...
8
by: Thomas Hallgren | last post by:
I'm writing a small test harness. I have two threads. One that starts the postmaster and another that does all the testing and finally stops the postmaster with a pg_ctl stop. At present, the...
2
by: Gregory S. Williamson | last post by:
Dear peoples, We had an oddness today with one of of postgres servers (Dell 2 CPU box running linux) and postgres 7.4. The server was under heavy load (50+ for a 1 minutes spike; about 20 for the...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.