473,400 Members | 2,163 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,400 software developers and data experts.

Stored Procedures woes

Hello,

We are using a number of stored procedures that are called often from our
client programs. I include one here as an example. The problem we are seeing
is that when executing some of these that deal with a large number of
records, they begin execution and never return. The process handling the
request chews 97% of CPU resources and must be cancelled before it will
release. We have tried examining where in the process it stops via the debug
output, but this has proven to be fruitless as A. the problem doesn't happen
with all the SP's (some other SP's deal with far more data but don't have
the problem), and B. doesn't always happen consistently with the SP's that
seem to cause problems. What we do know is that the machine doesn't seem to
be low on memory, never returns any error, and happens regardless of machine
or PG version (we've tried 7.4.1, and 7.4.2).

We must be doing something fundamentally wrong , so if anyone can shed any
light on why this might be happening and how I can track it down, would be
great. We don't have a whole lot of experience with PG, so there may very
likely be something we're doing wrong.

Thanks,

Andrew.

CREATE OR REPLACE FUNCTION updateUserSessionCost(INTEGER,DOUBLE precision)
RETURNS INTEGER AS '
DECLARE
groupId ALIAS FOR $1;
rate ALIAS FOR $2;
user_rec RECORD;
us_rec RECORD;
userId text;
costIn DOUBLE precision;
costOut DOUBLE precision;
BEGIN
-- get all user belong to that group
FOR user_rec IN SELECT distinct user_id FROM user_session where
group_id=groupId LOOP
userId = user_rec.user_id;
-- loop all session record for that user
FOR us_rec IN SELECT
session_id,hit_bytes_in,miss_bytes_in,hit_bytes_ou t,miss_bytes_out FROM
user_session where user_id=userId and group_id=groupId LOOP
costIn :=
(us_rec.hit_bytes_in+us_rec.miss_bytes_in)*rate/1048576;
costOut :=
(us_rec.hit_bytes_out+us_rec.miss_bytes_out)*rate/1048576;
if (costIn < 0.0001) then
costIn := 0;
end if;
if (costOut < 0.0001) then
costOut := 0;
end if;

update user_session set cost_bytes_in=costIn,
cost_bytes_out=costOut WHERE user_id=userId AND
session_id=us_rec.session_id;

END LOOP;
END LOOP;
return 1;
END;
' LANGUAGE 'plpgsql'
;
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddressHere" to ma*******@postgresql.org)

Nov 23 '05 #1
3 1356
On Thu, 2004-08-19 at 10:53, Andrew Hall wrote:
Hello,

We are using a number of stored procedures that are called often from our
client programs. I include one here as an example. The problem we are seeing
is that when executing some of these that deal with a large number of
records, they begin execution and never return. ....
CREATE OR REPLACE FUNCTION updateUserSessionCost(INTEGER,DOUBLE precision)
RETURNS INTEGER AS '
DECLARE .... BEGIN
-- get all user belong to that group
FOR user_rec IN SELECT distinct user_id FROM user_session where
group_id=groupId LOOP
userId = user_rec.user_id;
-- loop all session record for that user
FOR us_rec IN SELECT
session_id,hit_bytes_in,miss_bytes_in,hit_bytes_ou t,miss_bytes_out FROM
user_session where user_id=userId and group_id=groupId LOOP .... update user_session set cost_bytes_in=costIn,
cost_bytes_out=costOut WHERE user_id=userId AND
session_id=us_rec.session_id;


Are you somehow setting off an infinite recursion? How is this being
called?

Try putting RAISE NOTICE statements in to track the progress of the
code.
Oliver Elphick
---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postgresql.org so that your
message can get through to the mailing list cleanly

Nov 23 '05 #2
Oliver Elphick <ol**@lfix.co.uk> writes:
We are using a number of stored procedures that are called often from our
client programs.
Are you somehow setting off an infinite recursion? How is this being
called?
I doubt it would be an infinite-recursion problem, as that would soon
lead to stack overflow and core dump (or at least very obvious memory
bloat). I'm wondering about inefficient plans being generated for the
queries in the procedure.
Try putting RAISE NOTICE statements in to track the progress of the
code.


Right. See also recent discussion on pgsql-performance about
investigating plans used for plpgsql queries.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postgresql.org

Nov 23 '05 #3
"Andrew Hall" wrote:

We are using a number of stored procedures that are called often from our
client programs. I include one here as an example. The problem we are seeing
is that when executing some of these that deal with a large number of
records, they begin execution and never return. The process handling the
request chews 97% of CPU resources and must be cancelled before it will
release. We have tried examining where in the process it stops via the debug
output, but this has proven to be fruitless as A. the problem doesn't happen
with all the SP's (some other SP's deal with far more data but don't have
the problem), and B. doesn't always happen consistently with the SP's that
seem to cause problems. What we do know is that the machine doesn't seem to
be low on memory, never returns any error, and happens regardless of machine
or PG version (we've tried 7.4.1, and 7.4.2).


Wouldn't the following query be functionally the same as the
procedure you posted (if you fix the rate and the groupid)? If
so, does it perform better and how does the explain look?

UPDATE user_sessions
SET cost_bytes_in = a.costIn,
cost_bytes_out = a.costOut
FROM (
SELECT
session_id,
CASE
WHEN
(us_rec.hit_bytes_in+us_rec.miss_bytes_in)*$rate/1048576 < 0.0001
THEN 0
ELSE
(us_rec.hit_bytes_in+us_rec.miss_bytes_in)*$rate/1048576 END
AS costIn,
CASE
WHEN
(us_rec.hit_bytes_out+us_rec.miss_bytes_out)*$rate/1048576 <
0.0001 THEN 0
ELSE
(us_rec.hit_bytes_out+us_rec.miss_bytes_out)*$rate/1048576 END
AS costOut
FROM user_session
WHERE group_id = $groupId
) a
WHERE group_id = $groupId AND user_id = a.user_id;

Jochem
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Nov 23 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: The Collective | last post by:
Hello, Decided (Finally!!!) to start testing out some of the new features 5.x has to offer. More specifically stored procedures and functions. I have tried to create a procedure but I continue to...
11
by: jrefactors | last post by:
I want to know the differences between SQL Server 2000 stored procedures and oracle stored procedures? Do they have different syntax? The concept should be the same that the stored procedures...
2
by: scott | last post by:
Hi, Just wondering what sort of problems and advantages people have found using stored procedures. I have an app developed in VB6 & VB.NET and our developers are starting to re-write some of the...
2
by: Kent Lewandowski | last post by:
hi all, Recently I wrote some stored procedures using java jdbc code (admittedly my first stab) and then tried to implement the same within java packages (for code reuse). I encountered...
5
by: Tim Marshall | last post by:
I was following the thread "Re: Access Treeview - Is it Safe Yet?" with interest and on reading the post describing Lauren Quantrell's SmartTree, I've run into something I don't understand: Stored...
2
by: Eli | last post by:
Hi all We currently have a strange problem with calling a Stored Procedure (SQL Database) in our C# Project. The only error I get is "System error" which says a lot :) Background: We have...
45
by: John | last post by:
Hi When developing vb.bet winform apps bound to sql server datasource, is it preferable to use SELECTs or stored procedure to read and write data from/to SQL Server? Why? Thanks Regards
28
by: mooreit | last post by:
The purpose for my questions is accessing these technologies from applications. I develop both applications and databases. Working with Microsoft C#.NET and Microsoft SQL Server 2000 Production and...
11
by: peter | last post by:
I am trying to get a SQL stored procedure to use user maintained MQT implicitly which raises questions on when they are used or not used. In theory you would expect the stored procedure to pick up...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.