473,394 Members | 1,765 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,394 software developers and data experts.

Converting Perl Web Report to Python

I enjoyed Paul Barry's September article in Linux Journal entitled,
"Web Reporting with MySQL, CSS and Perl". It provides a simple,
elegant way to use HTML to display database content without any sql
markup in the cgi script. The cgi script simply calls the Mysql
command line with the HTML option (-H) and the SQL script file directed
to that command. This provides complete separation of the markup from
the sql code. The plain vanila HTML output can be spruced up with CSS
to provide more color and size control of the HTML.

Could this script be easily converted to Python? How would you execute
the Msql command line and direct the output to a variable for display
in the cgi script? Would it be possible to easily enhance this script
by allowing the user to pass in an SQL query parameter to the sql
script? I attempted this in Perl by substituting the string "p_1" in
the where clause of the sql code but I could not substitute this string
with the value in the cgi code (ie. $query =~ s/p_1/value_variable/;).
Perhaps it would be easier in Python?

Also, would the user supplied parameter be a security issue?
Thanks!

Below is a link to the article:
http://delivery.acm.org/10.1145/1160...FTOKEN=6184618

Sep 23 '06 #1
4 1757
Dennis,
Wow! Talk about RAD development... Nice job... Errr, real nice job. I
will test your concepts and see how it works.
Thanks a ton. You far exceeded my expectations!
Pat

Dennis Lee Bieber wrote:
On 23 Sep 2006 06:04:16 -0700, "pm******@gmail.com" <pm******@gmail.com>
declaimed the following in comp.lang.python:

Answering bottom up...

Could this script be easily converted to Python? How would you execute
the Msql command line and direct the output to a variable for display
in the cgi script? Would it be possible to easily enhance this script
by allowing the user to pass in an SQL query parameter to the sql
script? I attempted this in Perl by substituting the string "p_1" in
the where clause of the sql code but I could not substitute this string
with the value in the cgi code (ie. $query =~ s/p_1/value_variable/;).
Perhaps it would be easier in Python?

Also, would the user supplied parameter be a security issue?

Unless you duplicate the type of checking a db-api module does for
parameterized queries -- indubitably...

I don't do PERL, but from what I see, they are essentially doing the
equivalent of a popen() call.

-=-=-=-=-=-=-=-=- script1.py
import os

USERID = "BestiariaCP" #this account is a read-only, no password
DBNAME = "bestiaria" #used for CherryTemplate page generation

# lacking a CGI interface, I'm just going to put in a few assignments
query = "script1.sql"
title = "This is demonstrably nonsense"

cmdline = "mysql -H -u %s %s < %s" % (USERID, DBNAME, query)

# I don't know what header and start_html() do, so just a dummy here
print """
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

<html>
<head>
<title>%s</title>
</head>

<body>
""" % title

rpipe = os.popen(cmdline, "r")
results = rpipe.readlines()
rpipe.close()

print '<h3 align="center">%s</h3>' % title
print "".join(results)
print '<br><h4 align="center">This was a sample</h4>'
print "</body>\n</html>"
-=-=-=-=-=-=-=-=-=- script1.sql
select name, URL, description, occurs
from comics
order by name;
-=-=-=-=-=-=-=-=-=- script1.html (excluded middle)

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

<html>
<head>
<title>This is demonstrably nonsense</title>
</head>

<body>

<h3 align="center">This is demonstrably nonsense</h3>
<TABLE
BORDER=1><TR><TH>name</TH><TH>URL</TH><TH>description</TH><TH>occurs</TH></TR><TR><TD>A
Doemain of Our Own</TD><TD>http://www.doemain.com/</TD><TD>The new
family on the block.</TD><TD>Typically Friday, though some Monday and
Wednesday
updates</TD></TR><TR><TD>Aford</TD><TD>http://www.afordturtle.com/</TD><TD>A
turtle, a snake, and a robin; and their life in the
forest.</TD><TD>Daily</TD></TR>

<TR><TD>Virtual Comix</TD><TD>http://comix.keenspace.com/</TD><TD>He's
the insane rabbit next door.</TD><TD>No updates since March of
2003</TD></TR><TR><TD>West Corner of the
Park</TD><TD>http://www.graphxpress.com/</TD><TD>Inspired by FurryMUCK,
and done by Jim Groat, the creator of <i>Red
Shetland</i>.</TD><TD>Usually late Sunday, unless a Furry Convention
happens</TD></TR><TR><TD>Whatever
USA</TD><TD>http://whateverusa.keenspace.com/</TD><TD>A precocious
porcupine pup and his pals.</TD><TD>Sporadic</TD></TR><TR><TD>Wild
Angels</TD><TD>http://www.ottercomics.com/angels/</TD><TD>It'll put the
fur of God in you.</TD><TD>Irregular (between two and seven times each
month)</TD></TR><TR><TD>Wyldfire</TD><TD>http://www.morgankeithstudios.com/projects_wyldfire.html</TD><TD>What
if the cat stuck up a tree <i>is</ithe fireman?</TD><TD>Archives
available</TD></TR></TABLE>
<br><h4 align="center">This was a sample</h4>
</body>
</html>

Seems like a lot of hassle to go through when a Python function can
generate similar results without losing the safety of parameterized
queries (I do hope the SQL files the CGI is specifying were pre-defined,
and not something the user uploads <G>)
-=-=-=-=-=-=-=- script2.py
import MySQLdb

USERID = "BestiariaCP" #this account is a read-only, no password
DBNAME = "bestiaria" #used for CherryTemplate page generation

# lacking a CGI interface, I'm just going to put in a few assignments
queryfile = "script1.sql"
title = "This is demonstrably nonsense"

cn = MySQLdb.connect(host="localhost", user=USERID, db=DBNAME)
cr = cn.cursor()

# I don't know what header and start_html() do, so just a dummy here
print """
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

<html>
<head>
<title>%s</title>
</head>

<body>
""" % title

qin = open(queryfile, "r")
query = " ".join(qin.readlines())
qin.close()

cr.execute(query) #no parameters assumed

# THE FOLLOWING BLOCK OF CODE COULD EASILY BE MADE A
# FUNCTION IN SOME HTML REPORTING MODULE AS IT IS NOT
# DEPENDENT UPON THE ACTUAL QUERY TO KNOW WHAT TO PRODUCE

print '<h3 align="center">%s</h3>' % title
print '<table border="1">\n<tr>',
for fld in cr.description:
print "<th>%s</th>" % fld[0],
print "</tr>"

for rec in cr:
print "<tr>",
for fld in rec:
print "<td>%s</td>" % fld,
print "</tr>"
print "</table>"

# BACK TO ORIGINAL CONTENT

print '<br><h4 align="center">This was a sample</h4>'
print "</body>\n</html>"

cr.close()
cn.close()
-=-=-=-=-=- USES SAME SQL FILE
-=-=-=-=-=-=- script2.html (excluded middle)

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

<html>
<head>
<title>This is demonstrably nonsense</title>
</head>

<body>

<h3 align="center">This is demonstrably nonsense</h3>
<table border="1">
<tr<th>name</th<th>URL</th<th>description</th<th>occurs</th>
</tr>
<tr<td>A Doemain of Our Own</td<td>http://www.doemain.com/</td>
<td>The new family on the block.</td<td>Typically Friday, though some
Monday and Wednesday updates</td</tr>
<tr<td>Aford</td<td>http://www.afordturtle.com/</td<td>A turtle, a
snake, and a robin; and their life in the forest.</td<td>Daily</td>
</tr>

<tr<td>Virtual Comix</td<td>http://comix.keenspace.com/</td>
<td>He's the insane rabbit next door.</td<td>No updates since March of
2003</td</tr>
<tr<td>West Corner of the Park</td>
<td>http://www.graphxpress.com/</td<td>Inspired by FurryMUCK, and done
by Jim Groat, the creator of <i>Red Shetland</i>.</td<td>Usually late
Sunday, unless a Furry Convention happens</td</tr>
<tr<td>Whatever USA</td<td>http://whateverusa.keenspace.com/</td>
<td>A precocious porcupine pup and his pals.</td<td>Sporadic</td>
</tr>
<tr<td>Wild Angels</td<td>http://www.ottercomics.com/angels/</td>
<td>It'll put the fur of God in you.</td<td>Irregular (between two and
seven times each month)</td</tr>
<tr<td>Wyldfire</td>
<td>http://www.morgankeithstudios.com/projects_wyldfire.html</td>
<td>What if the cat stuck up a tree <i>is</ithe fireman?</td>
<td>Archives available</td</tr>
</table>
<br><h4 align="center">This was a sample</h4>
</body>
</html>
--
Wulfraed Dennis Lee Bieber KD6MOG
wl*****@ix.netcom.com wu******@bestiaria.com
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: we******@bestiaria.com)
HTTP://www.bestiaria.com/
Sep 23 '06 #2
Dennis,
I was able to execute the Mysql command line code you created in
"script1.py". I was not able to test the "preferred" method used in
script2.py. I will do this later this week when I have more time to
download and install the required MySQLdb module. In any event, I
would like to know how to search and replace a given string in the sql
to a user supplied value before execution.

For example, lets say that your sql script looks like this:
-=-=-=-=-=-=-=-=-=- script1.sql
select name, URL, description, occurs
from comics
where URL like '%MyParam_1%' # Note that "MyParam_1" represents
a "catch" string.
order by name;
-=-=-=-=-=-=-=-=-=-
Using the script2.py process, how could I safely convert the above
string, "MyParam_1" to a user supplied argument (ie. "comix") before
the SQL is executed?

This capability would allow each sql script to be extendable. It would
allow me to embed a URL with specific parameters in report OR in Wiki
pages, emails when I need to.
Thanks!
Pat

Dennis Lee Bieber wrote:
On 23 Sep 2006 06:04:16 -0700, "pm******@gmail.com" <pm******@gmail.com>
declaimed the following in comp.lang.python:

Answering bottom up...

Could this script be easily converted to Python? How would you execute
the Msql command line and direct the output to a variable for display
in the cgi script? Would it be possible to easily enhance this script
by allowing the user to pass in an SQL query parameter to the sql
script? I attempted this in Perl by substituting the string "p_1" in
the where clause of the sql code but I could not substitute this string
with the value in the cgi code (ie. $query =~ s/p_1/value_variable/;).
Perhaps it would be easier in Python?

Also, would the user supplied parameter be a security issue?

Unless you duplicate the type of checking a db-api module does for
parameterized queries -- indubitably...

I don't do PERL, but from what I see, they are essentially doing the
equivalent of a popen() call.

-=-=-=-=-=-=-=-=- script1.py
import os

USERID = "BestiariaCP" #this account is a read-only, no password
DBNAME = "bestiaria" #used for CherryTemplate page generation

# lacking a CGI interface, I'm just going to put in a few assignments
query = "script1.sql"
title = "This is demonstrably nonsense"

cmdline = "mysql -H -u %s %s < %s" % (USERID, DBNAME, query)

# I don't know what header and start_html() do, so just a dummy here
print """
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

<html>
<head>
<title>%s</title>
</head>

<body>
""" % title

rpipe = os.popen(cmdline, "r")
results = rpipe.readlines()
rpipe.close()

print '<h3 align="center">%s</h3>' % title
print "".join(results)
print '<br><h4 align="center">This was a sample</h4>'
print "</body>\n</html>"
-=-=-=-=-=-=-=-=-=- script1.sql
select name, URL, description, occurs
from comics
order by name;
-=-=-=-=-=-=-=-=-=- script1.html (excluded middle)

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

<html>
<head>
<title>This is demonstrably nonsense</title>
</head>

<body>

<h3 align="center">This is demonstrably nonsense</h3>
<TABLE
BORDER=1><TR><TH>name</TH><TH>URL</TH><TH>description</TH><TH>occurs</TH></TR><TR><TD>A
Doemain of Our Own</TD><TD>http://www.doemain.com/</TD><TD>The new
family on the block.</TD><TD>Typically Friday, though some Monday and
Wednesday
updates</TD></TR><TR><TD>Aford</TD><TD>http://www.afordturtle.com/</TD><TD>A
turtle, a snake, and a robin; and their life in the
forest.</TD><TD>Daily</TD></TR>

<TR><TD>Virtual Comix</TD><TD>http://comix.keenspace.com/</TD><TD>He's
the insane rabbit next door.</TD><TD>No updates since March of
2003</TD></TR><TR><TD>West Corner of the
Park</TD><TD>http://www.graphxpress.com/</TD><TD>Inspired by FurryMUCK,
and done by Jim Groat, the creator of <i>Red
Shetland</i>.</TD><TD>Usually late Sunday, unless a Furry Convention
happens</TD></TR><TR><TD>Whatever
USA</TD><TD>http://whateverusa.keenspace.com/</TD><TD>A precocious
porcupine pup and his pals.</TD><TD>Sporadic</TD></TR><TR><TD>Wild
Angels</TD><TD>http://www.ottercomics.com/angels/</TD><TD>It'll put the
fur of God in you.</TD><TD>Irregular (between two and seven times each
month)</TD></TR><TR><TD>Wyldfire</TD><TD>http://www.morgankeithstudios.com/projects_wyldfire.html</TD><TD>What
if the cat stuck up a tree <i>is</ithe fireman?</TD><TD>Archives
available</TD></TR></TABLE>
<br><h4 align="center">This was a sample</h4>
</body>
</html>

Seems like a lot of hassle to go through when a Python function can
generate similar results without losing the safety of parameterized
queries (I do hope the SQL files the CGI is specifying were pre-defined,
and not something the user uploads <G>)
-=-=-=-=-=-=-=- script2.py
import MySQLdb

USERID = "BestiariaCP" #this account is a read-only, no password
DBNAME = "bestiaria" #used for CherryTemplate page generation

# lacking a CGI interface, I'm just going to put in a few assignments
queryfile = "script1.sql"
title = "This is demonstrably nonsense"

cn = MySQLdb.connect(host="localhost", user=USERID, db=DBNAME)
cr = cn.cursor()

# I don't know what header and start_html() do, so just a dummy here
print """
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

<html>
<head>
<title>%s</title>
</head>

<body>
""" % title

qin = open(queryfile, "r")
query = " ".join(qin.readlines())
qin.close()

cr.execute(query) #no parameters assumed

# THE FOLLOWING BLOCK OF CODE COULD EASILY BE MADE A
# FUNCTION IN SOME HTML REPORTING MODULE AS IT IS NOT
# DEPENDENT UPON THE ACTUAL QUERY TO KNOW WHAT TO PRODUCE

print '<h3 align="center">%s</h3>' % title
print '<table border="1">\n<tr>',
for fld in cr.description:
print "<th>%s</th>" % fld[0],
print "</tr>"

for rec in cr:
print "<tr>",
for fld in rec:
print "<td>%s</td>" % fld,
print "</tr>"
print "</table>"

# BACK TO ORIGINAL CONTENT

print '<br><h4 align="center">This was a sample</h4>'
print "</body>\n</html>"

cr.close()
cn.close()
-=-=-=-=-=- USES SAME SQL FILE
-=-=-=-=-=-=- script2.html (excluded middle)

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">

<html>
<head>
<title>This is demonstrably nonsense</title>
</head>

<body>

<h3 align="center">This is demonstrably nonsense</h3>
<table border="1">
<tr<th>name</th<th>URL</th<th>description</th<th>occurs</th>
</tr>
<tr<td>A Doemain of Our Own</td<td>http://www.doemain.com/</td>
<td>The new family on the block.</td<td>Typically Friday, though some
Monday and Wednesday updates</td</tr>
<tr<td>Aford</td<td>http://www.afordturtle.com/</td<td>A turtle, a
snake, and a robin; and their life in the forest.</td<td>Daily</td>
</tr>

<tr<td>Virtual Comix</td<td>http://comix.keenspace.com/</td>
<td>He's the insane rabbit next door.</td<td>No updates since March of
2003</td</tr>
<tr<td>West Corner of the Park</td>
<td>http://www.graphxpress.com/</td<td>Inspired by FurryMUCK, and done
by Jim Groat, the creator of <i>Red Shetland</i>.</td<td>Usually late
Sunday, unless a Furry Convention happens</td</tr>
<tr<td>Whatever USA</td<td>http://whateverusa.keenspace.com/</td>
<td>A precocious porcupine pup and his pals.</td<td>Sporadic</td>
</tr>
<tr<td>Wild Angels</td<td>http://www.ottercomics.com/angels/</td>
<td>It'll put the fur of God in you.</td<td>Irregular (between two and
seven times each month)</td</tr>
<tr<td>Wyldfire</td>
<td>http://www.morgankeithstudios.com/projects_wyldfire.html</td>
<td>What if the cat stuck up a tree <i>is</ithe fireman?</td>
<td>Archives available</td</tr>
</table>
<br><h4 align="center">This was a sample</h4>
</body>
</html>
--
Wulfraed Dennis Lee Bieber KD6MOG
wl*****@ix.netcom.com wu******@bestiaria.com
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: we******@bestiaria.com)
HTTP://www.bestiaria.com/
Sep 24 '06 #3
Thanks again Dennis,
This should do what I want with additional flexibility... I will
develop the code later this week. During this excersize, I have come
to really appreciate Python over Perl. I love the Python command line
interpreter that allowed me to pretest any code very quickly. What
Python learning resource (book web link or both) do you recommend?
Pat

Dennis Lee Bieber wrote:
On 24 Sep 2006 14:05:29 -0700, "pm******@gmail.com" <pm******@gmail.com>
declaimed the following in comp.lang.python:

download and install the required MySQLdb module. In any event, I
would like to know how to search and replace a given string in the sql
to a user supplied value before execution.
I would not even want to try to code that. I will mention that the
default mode for MySQLdb is to use %s as the substitution field, and
since both scripts used the same SQL file...
For example, lets say that your sql script looks like this:
-=-=-=-=-=-=-=-=-=- script1.sql
select name, URL, description, occurs
from comics
where URL like '%MyParam_1%' # Note that "MyParam_1" represents
a "catch" string.
order by name;
-=-=-=-=-=-=-=-=-=-
select name, URL, description, occurs
from comics
where URL like %s
order by name;

Then, on the cursor.execute() call:
cr.execute(query) #no parameters assumed

cr.execute(query, ("%" + MyParam_1 + "%",))

{MySQLdb /can/ work with a singleton for convenience, but the DB-API
emphasizes a tuple be passed, hence the (... ,) to create a tuple}

I'm presuming you do not require the "user" to enter the % wildcards
on search terms.

MySQLdb can also be fed with a dictionary (last time I looked, at
least) so something like:

select name, URL, description, occurs
from comics
where URL like %(MyParam_1)s or URL like %(MyParam_2)s
order by name;

and

params = {}
params["MyParam_1"] = "%" + somevariable + "%"
params["MyParam_2"] = "%" + anotherstring + "%"
cr.execute(query, params)

may be possible (I've never used this mode, so no confirmation).
--
Wulfraed Dennis Lee Bieber KD6MOG
wl*****@ix.netcom.com wu******@bestiaria.com
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: we******@bestiaria.com)
HTTP://www.bestiaria.com/
Sep 25 '06 #4

pm******@gmail.com wrote:
Thanks again Dennis,
This should do what I want with additional flexibility... I will
develop the code later this week. During this excersize, I have come
to really appreciate Python over Perl. I love the Python command line
interpreter that allowed me to pretest any code very quickly. What
Python learning resource (book web link or both) do you recommend?
I have 3 O'Reilly books that are indispensable: "Python Pocket
Reference", "Python Cookbook", and "Learning Python"

Sep 25 '06 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: bezeee | last post by:
At my work we are in the process of building a tool to test an XML based API. Basically, XML in and XML out over http. Currently, there are two engines that do all of the schema validations, xml...
0
by: Terry Hancock | last post by:
Hi all, Suppose you have found a really nice module that does what you want (or will do, it still being in development), and you don't want to waste a lot of duplicated effort, but this module...
4
by: Xah Lee | last post by:
# -*- coding: utf-8 -*- # Python # some venture into standard modules import os # print all names exported by the module print dir(os)
1
by: Xah Lee | last post by:
suppose you want to do find & replace of string of all files in a directory. here's the code: ©# -*- coding: utf-8 -*- ©# Python © ©import os,sys © ©mydir= '/Users/t/web'
5
by: Mothra | last post by:
Hi All, I am the current author of the Astro-Sunrise perl module http://search.cpan.org/~rkhill/Astro-Sunrise-0.91/Sunrise.pm and was wondering if it would be worth while to convert it to...
7
by: ChiTownBob | last post by:
I'm trying to convert our documentation over to the Wikipedia format (MediaWiki), and about ready to give up on the Perl-based method I found (because Perl just sucks, as all good Python hackers...
4
by: eight02645999 | last post by:
hi i need help with converting a piece of perl code to python ...... my $start = '\'; my $file = '\'; my $end = '\'; ..... while(<FILE>) #open a file {
4
by: eight02645999 | last post by:
hi i have some regular exp code in perl that i want to convert to python. if $line =~ m#<(tag1)>(.*)</\1># { $variable = $2; }
4
by: Samuel Hardman | last post by:
Hello, I am trying to write a perl script to parse a string into an array. The string has the fields separated by tabs. So what I want to do is read each field into a variable so I can process...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.