473,787 Members | 2,798 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Stored Procedures woes

Hello,

We are using a number of stored procedures that are called often from our
client programs. I include one here as an example. The problem we are seeing
is that when executing some of these that deal with a large number of
records, they begin execution and never return. The process handling the
request chews 97% of CPU resources and must be cancelled before it will
release. We have tried examining where in the process it stops via the debug
output, but this has proven to be fruitless as A. the problem doesn't happen
with all the SP's (some other SP's deal with far more data but don't have
the problem), and B. doesn't always happen consistently with the SP's that
seem to cause problems. What we do know is that the machine doesn't seem to
be low on memory, never returns any error, and happens regardless of machine
or PG version (we've tried 7.4.1, and 7.4.2).

We must be doing something fundamentally wrong , so if anyone can shed any
light on why this might be happening and how I can track it down, would be
great. We don't have a whole lot of experience with PG, so there may very
likely be something we're doing wrong.

Thanks,

Andrew.

CREATE OR REPLACE FUNCTION updateUserSessi onCost(INTEGER, DOUBLE precision)
RETURNS INTEGER AS '
DECLARE
groupId ALIAS FOR $1;
rate ALIAS FOR $2;
user_rec RECORD;
us_rec RECORD;
userId text;
costIn DOUBLE precision;
costOut DOUBLE precision;
BEGIN
-- get all user belong to that group
FOR user_rec IN SELECT distinct user_id FROM user_session where
group_id=groupI d LOOP
userId = user_rec.user_i d;
-- loop all session record for that user
FOR us_rec IN SELECT
session_id,hit_ bytes_in,miss_b ytes_in,hit_byt es_out,miss_byt es_out FROM
user_session where user_id=userId and group_id=groupI d LOOP
costIn :=
(us_rec.hit_byt es_in+us_rec.mi ss_bytes_in)*ra te/1048576;
costOut :=
(us_rec.hit_byt es_out+us_rec.m iss_bytes_out)* rate/1048576;
if (costIn < 0.0001) then
costIn := 0;
end if;
if (costOut < 0.0001) then
costOut := 0;
end if;

update user_session set cost_bytes_in=c ostIn,
cost_bytes_out= costOut WHERE user_id=userId AND
session_id=us_r ec.session_id;

END LOOP;
END LOOP;
return 1;
END;
' LANGUAGE 'plpgsql'
;
---------------------------(end of broadcast)---------------------------
TIP 2: you can get off all lists at once with the unregister command
(send "unregister YourEmailAddres sHere" to ma*******@postg resql.org)

Nov 23 '05 #1
3 1387
On Thu, 2004-08-19 at 10:53, Andrew Hall wrote:
Hello,

We are using a number of stored procedures that are called often from our
client programs. I include one here as an example. The problem we are seeing
is that when executing some of these that deal with a large number of
records, they begin execution and never return. ....
CREATE OR REPLACE FUNCTION updateUserSessi onCost(INTEGER, DOUBLE precision)
RETURNS INTEGER AS '
DECLARE .... BEGIN
-- get all user belong to that group
FOR user_rec IN SELECT distinct user_id FROM user_session where
group_id=groupI d LOOP
userId = user_rec.user_i d;
-- loop all session record for that user
FOR us_rec IN SELECT
session_id,hit_ bytes_in,miss_b ytes_in,hit_byt es_out,miss_byt es_out FROM
user_session where user_id=userId and group_id=groupI d LOOP .... update user_session set cost_bytes_in=c ostIn,
cost_bytes_out= costOut WHERE user_id=userId AND
session_id=us_r ec.session_id;


Are you somehow setting off an infinite recursion? How is this being
called?

Try putting RAISE NOTICE statements in to track the progress of the
code.
Oliver Elphick
---------------------------(end of broadcast)---------------------------
TIP 3: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to ma*******@postg resql.org so that your
message can get through to the mailing list cleanly

Nov 23 '05 #2
Oliver Elphick <ol**@lfix.co.u k> writes:
We are using a number of stored procedures that are called often from our
client programs.
Are you somehow setting off an infinite recursion? How is this being
called?
I doubt it would be an infinite-recursion problem, as that would soon
lead to stack overflow and core dump (or at least very obvious memory
bloat). I'm wondering about inefficient plans being generated for the
queries in the procedure.
Try putting RAISE NOTICE statements in to track the progress of the
code.


Right. See also recent discussion on pgsql-performance about
investigating plans used for plpgsql queries.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: subscribe and unsubscribe commands go to ma*******@postg resql.org

Nov 23 '05 #3
"Andrew Hall" wrote:

We are using a number of stored procedures that are called often from our
client programs. I include one here as an example. The problem we are seeing
is that when executing some of these that deal with a large number of
records, they begin execution and never return. The process handling the
request chews 97% of CPU resources and must be cancelled before it will
release. We have tried examining where in the process it stops via the debug
output, but this has proven to be fruitless as A. the problem doesn't happen
with all the SP's (some other SP's deal with far more data but don't have
the problem), and B. doesn't always happen consistently with the SP's that
seem to cause problems. What we do know is that the machine doesn't seem to
be low on memory, never returns any error, and happens regardless of machine
or PG version (we've tried 7.4.1, and 7.4.2).


Wouldn't the following query be functionally the same as the
procedure you posted (if you fix the rate and the groupid)? If
so, does it perform better and how does the explain look?

UPDATE user_sessions
SET cost_bytes_in = a.costIn,
cost_bytes_out = a.costOut
FROM (
SELECT
session_id,
CASE
WHEN
(us_rec.hit_byt es_in+us_rec.mi ss_bytes_in)*$r ate/1048576 < 0.0001
THEN 0
ELSE
(us_rec.hit_byt es_in+us_rec.mi ss_bytes_in)*$r ate/1048576 END
AS costIn,
CASE
WHEN
(us_rec.hit_byt es_out+us_rec.m iss_bytes_out)* $rate/1048576 <
0.0001 THEN 0
ELSE
(us_rec.hit_byt es_out+us_rec.m iss_bytes_out)* $rate/1048576 END
AS costOut
FROM user_session
WHERE group_id = $groupId
) a
WHERE group_id = $groupId AND user_id = a.user_id;

Jochem
---------------------------(end of broadcast)---------------------------
TIP 4: Don't 'kill -9' the postmaster

Nov 23 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
1363
by: The Collective | last post by:
Hello, Decided (Finally!!!) to start testing out some of the new features 5.x has to offer. More specifically stored procedures and functions. I have tried to create a procedure but I continue to get an error and I think my syntax is correct..... Here is the code: CREATE PROCEDURE usp_GetPermissionGroups
11
10758
by: jrefactors | last post by:
I want to know the differences between SQL Server 2000 stored procedures and oracle stored procedures? Do they have different syntax? The concept should be the same that the stored procedures execute in the database server with better performance? Please advise good references for Oracle stored procedures also. thanks!!
2
2821
by: scott | last post by:
Hi, Just wondering what sort of problems and advantages people have found using stored procedures. I have an app developed in VB6 & VB.NET and our developers are starting to re-write some of the code in stored procedures (im advocating encryption of them). When deploying an application however stored procedure seem to add another level of complexity to installation. In future we also plan to have an basic ASP app with some of the...
2
9245
by: Kent Lewandowski | last post by:
hi all, Recently I wrote some stored procedures using java jdbc code (admittedly my first stab) and then tried to implement the same within java packages (for code reuse). I encountered problems doing this. I wanted to implemented a generic "Helper" class like this: /** * Helper
5
3486
by: Tim Marshall | last post by:
I was following the thread "Re: Access Treeview - Is it Safe Yet?" with interest and on reading the post describing Lauren Quantrell's SmartTree, I've run into something I don't understand: Stored Procedures. I thought stored pricedures were an Oracle/MS SQL Server thing and don't know how they work with Access Jet. I've looked at some of the help on stored procedures in A2003, but really don't understand what's going on. Can someone...
2
3335
by: Eli | last post by:
Hi all We currently have a strange problem with calling a Stored Procedure (SQL Database) in our C# Project. The only error I get is "System error" which says a lot :) Background: We have several stored procedures to Insert and update datas in our SQL database. Some stored procedures are smaller (insert datas in only one table) and some of them are quite big (insert datas in several
45
3415
by: John | last post by:
Hi When developing vb.bet winform apps bound to sql server datasource, is it preferable to use SELECTs or stored procedure to read and write data from/to SQL Server? Why? Thanks Regards
28
72559
by: mooreit | last post by:
The purpose for my questions is accessing these technologies from applications. I develop both applications and databases. Working with Microsoft C#.NET and Microsoft SQL Server 2000 Production and 2005 Test Environments. What is the purpose of a view if I can just copy the vode from a view and put it into a stored procedure? Should I be accessing views from stored procedures?
11
3445
by: peter | last post by:
I am trying to get a SQL stored procedure to use user maintained MQT implicitly which raises questions on when they are used or not used. In theory you would expect the stored procedure to pick up the MQT at the time it is bound on the creation of the static SQL. This raises the question on how you stop it or start it using a MQT as there is no option on the bind. What happens when it is rebound? What happens if the plan is made invalid...
0
10363
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10169
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
9964
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8993
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7517
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6749
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5398
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5534
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
3
2894
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.