473,324 Members | 2,246 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,324 software developers and data experts.

parsing dynamic sql and removing constants

Hello

I need to process a SQL monitoring log stored in a table to group the
statements that change just the constants in it. Ex:

select a,b,c from table where (a = 'xyz' and b = 123 and c !=
'2004-10-10') or ( a like 'def%' )
select a,b,c from table where (a = 'zyx' and b = 321 and c !=
'2004-01-01') or ( a like '%fed' )

using an udf to remove constants:

select a,b,c from table where (a = '' and b = and c != '' ) or ( a
like '' )
or
select a,b,c from table where (a = %cconstant% and b = %nconstant% and
c != %dconstant% ) or ( a like %cconstant% )

I'm building a simple udf to do this, but i found this a little more
trickier.
It seems that i could use yacc / lex to do this, but i didnt touch
anything related yet
(i wasn't willing to spend so much time)

Anybody knowns any clever algorithm/udf that i could use to do this?
Thanks
Nov 12 '05 #1
3 2791
This is a much much more complex process than your two statements make
it appear to be. I believe that there are some purchasable performance
tools from third party vendors that do this bit I've never used any of
them and don't know how well they work.

I wrote a Java program to do this directly from a statement trace log
from UDB V7. I basically had to write an SQL statement parser then
rebuild the SQL statement. I'd be interested in "a clever algorithm" too.

Phil Sherman
Alexandre H. Guerra wrote:
Hello

I need to process a SQL monitoring log stored in a table to group the
statements that change just the constants in it. Ex:

select a,b,c from table where (a = 'xyz' and b = 123 and c !=
'2004-10-10') or ( a like 'def%' )
select a,b,c from table where (a = 'zyx' and b = 321 and c !=
'2004-01-01') or ( a like '%fed' )

using an udf to remove constants:

select a,b,c from table where (a = '' and b = and c != '' ) or ( a
like '' )
or
select a,b,c from table where (a = %cconstant% and b = %nconstant% and
c != %dconstant% ) or ( a like %cconstant% )

I'm building a simple udf to do this, but i found this a little more
trickier.
It seems that i could use yacc / lex to do this, but i didnt touch
anything related yet
(i wasn't willing to spend so much time)

Anybody knowns any clever algorithm/udf that i could use to do this?
Thanks


Nov 12 '05 #2
Ian
Alexandre H. Guerra wrote:
Hello

I need to process a SQL monitoring log stored in a table to group the
statements that change just the constants in it. Ex:

select a,b,c from table where (a = 'xyz' and b = 123 and c !=
'2004-10-10') or ( a like 'def%' )
select a,b,c from table where (a = 'zyx' and b = 321 and c !=
'2004-01-01') or ( a like '%fed' )

using an udf to remove constants:

select a,b,c from table where (a = '' and b = and c != '' ) or ( a
like '' )
or
select a,b,c from table where (a = %cconstant% and b = %nconstant% and
c != %dconstant% ) or ( a like %cconstant% )

I did this once using perl regular expressions, it was pretty effective.
(but I used 3 regexes to completely parameterize a statement).

There was an article on DB2DD to build a UDF that exposed the perl regex
library to DB2, so I suppose this would be possible to do within DB2.
(The script I wrote was reading SQL from a flat file).


----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= East/West-Coast Server Farms - Total Privacy via Encryption =---
Nov 12 '05 #3
Hi,

i just wrote an UDF in C++ which uses the Perl Library to format the
constants in SQL-Statements. We are going to use this UDF in our
performance monitoring tool so i am not allowed to post it here. But
you can use the following links to implement your own UDF (1st
scenario).

1st scenario: CPerlString, Classdefinitions to integrate Perl Library
http://www.codeproject.com/string/cperlstring.asp
and integrate CPerlString in your UDF

2nd scenario: Knut Stolze (this guy knows what he does) uses PCRE
http://www-106.ibm.com/developerwork...301stolze.html

Using Lexx and yacc could be a problem (non deterministic, not context
sensitive grammar ....)

sample Perl REGEX to replace any number in a string with
'HERE_WAS_A_NUMBER':
$parse =~ s/(\W)(\d+)/$1HERE_WAS_A_NUMBER/gi;
be careful, this also make out of 123.456
HERE_WAS_A_NUMBER.HERE_WAS_A_NUMBER
I wrote 4 regular expressions in my UDF to remove the constants and it
works for almost 90% of the sqls i tested with. Let me know if you
need some hints for the regular expressions, a sample c-code or
compiling DB2 UDFs for WIN32 (took me 2 days to set up the right
environment)

best wishes
Florian
Ian <ia*****@mobileaudio.com> wrote in message news:<41********@127.0.0.1>...
Alexandre H. Guerra wrote:
Hello

I need to process a SQL monitoring log stored in a table to group the
statements that change just the constants in it. Ex:

select a,b,c from table where (a = 'xyz' and b = 123 and c !=
'2004-10-10') or ( a like 'def%' )
select a,b,c from table where (a = 'zyx' and b = 321 and c !=
'2004-01-01') or ( a like '%fed' )

using an udf to remove constants:

select a,b,c from table where (a = '' and b = and c != '' ) or ( a
like '' )
or
select a,b,c from table where (a = %cconstant% and b = %nconstant% and
c != %dconstant% ) or ( a like %cconstant% )

I did this once using perl regular expressions, it was pretty effective.
(but I used 3 regexes to completely parameterize a statement).

There was an article on DB2DD to build a UDF that exposed the perl regex
library to DB2, so I suppose this would be possible to do within DB2.
(The script I wrote was reading SQL from a flat file).


----== Posted via Newsfeeds.Com - Unlimited-Uncensored-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! >100,000 Newsgroups
---= East/West-Coast Server Farms - Total Privacy via Encryption =---

Nov 12 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Emre Sevinc | last post by:
I'm using classical ASP on IIS 6.0 (Windows 2003 Server). What I want to do is to serve the same pages in two different languages without using two different folders and doubling my same ASP code...
9
by: {AGUT2}=IWIK= | last post by:
Hello all, It's my fisrt post here and I am feeling a little stupid here, so go easy.. :) (Oh, and I've spent _hours_ searching...) I am desperately trying to read in an ASCII...
8
by: Drew | last post by:
I am trying to build a small app that shows a Course Title from the database, then displays a dropdown full of categories for the user to choose one... I thought a loop would be the best way to...
3
by: Arsalan Ahmad | last post by:
Hi, I have recently uploaded a file in a webserver. When i try to access it via browser, I get following error: This is working fine in my local copy. Any idea what could be wrong. Server Error...
23
by: Peter Row | last post by:
Hi, I am currently working on a VB.NET project that has been going for quite a while. In the past it has been developed with a lot of compatibility VB6 functions and constants, e.g Left(),...
1
by: chixor1 | last post by:
I have been charged with Parsing the data from many Abstract files, and then inputing this information into a SQL Database. The file format is rather unusual and certainly not delimited in any...
2
by: python | last post by:
I'm parsing a text file for a proprietary product that has the following 2 directives: #include <somefile> #define <name<value> Defined constants are referenced via <#name#syntax. I'm...
6
by: =?ISO-8859-1?Q?Tim_B=FCthe?= | last post by:
Hi, we are building a Java webapplication using JSF, running on websphere, querying a DB2 9 on Suse Enterprise 10. The app uses JDBC and PreparedStatements only (aka dynamic SQL). Every night,...
1
by: hd95 | last post by:
In a perfect world my xml feed source would produce perfect xml ..that is not the case I am parsing an XML feed that sometimes has ampersands and dashes in the content that messes up my parsing. ...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.