mod_python exception catching, other repetitious per-page actions

[Karl A. Krueger]

I'm in the middle of refactoring a small mod_python Web application,
which uses the Publisher handler. This application is currently a
single main Python file (which loads several other files as modules)
with several function entry points for the different pages in the
application.

(The application is a tool to allow staff members here to request
firewall port openings for their servers. Thus there is a page (and
function) for "view my servers", "add a server", "add a service to a
server", and so forth. Each of these is a function in the published
file.)

One thing I would like to do is factor all of the top-level database
connecting, top-level exception handling, and error-page generation into
a single place. Whenever an exception occurs not trapped locally --
including an error connecting to the database -- I want to generate an
HTML error page instead of throwing the stacktrace at the user.

Currently this means that every function is one big try/except block
where the except block calls a TypesetErrorPag e() function. Also
repeated are the function calls to set up the database backend
connection and to set up some per-user globals:

def mainpage(apache _request, some_cgi_arg):
try:
_connect_to_dat abase()
# other repetitious stuff here
_do_user_specif ic_stuff(apache _request)
#
# specific code here
#
except Exception, err:
return _TypesetErrorPa ge(err)

def addserverpage(a pache_request, some_cgi_arg, another_arg):
try:
_connect_to_dat abase()
# other repetitious stuff here
_do_user_specif ic_stuff(apache _request)
#
# specific code here
#
except Exception, err:
return _TypesetErrorPa ge(err)

I'd like to factor this repetition out. However there does not seem to
be a straightforward way to do it -- the connect_to_data base() could be
done at the module level (i.e. outside of any function), but this might
have problems if the Apache instance persists long enough between
requests for the database to time out.

One thing I have considered is to handle _all_ the pages through a
single function, which would look up the specific pages by name:

PAGES = { "mainpage" : mainpagefunc, "addserverp age" : addserverfunc }

def page(apache_req uest, which_page, *args):
try:
connect_to_data base()
# other repetitious stuff here
do_user_specifi c_stuff(apache_ request)
return PAGES[which_page](req, *args) # call the real page
except Exception, err:
return TypesetErrorPag e(err)

.... but this has the problem that it sacrifices the argument name
matching which is a major useful feature of mod_python Publisher. I'm
not sure if keyword args are exactly the Right Thing either. It also
would make the URLs even longer than they already are, though I can
kludge that with Apache URL rewriting rules. (Bleah.)

I feel like I want a Lisp macro, but that's a topic I shouldn't even
invoke. Forget I said that. :)

Any thoughts?

--
Karl A. Krueger <kk******@examp le.edu>
Woods Hole Oceanographic Institution
Email address is spamtrapped. s/example/whoi/
"Outlook not so good." -- Magic 8-Ball Software Reviews

[Jeff Epler]

You could use a wrapper for each function, which at least keeps you
from needing the which_page argument..

def page_wrapper(ca llable):
def _wrapper(apache _request, *args):
try:
_connect_to_dat abase()
_do_user_specif ic_stuff(apache _request)
return callable(*args)
except Exception, err:
return _TypesetErrorPa ge(err)
return _wrapper

def mainpage(apache _request, some_cgi_arg):
# specific code ehre
mainpage = page_wrapper(ma inpage)
page_wrapper() could do extra magic with exec if you want it to have the
same signature as callable.. something like
def page_wrapper(ca llable):
d = {}
name = callable.__name __
argspec = getargspecstr(c allable) # based on inspect.getargs pec()
callspec = ... # argspec with default values removed
exec """def %s(%s): ... return callable(%s) ...""" \
% (name, argspec, callspec) in d
return d[name]
.... barf, eh?

Jeff
PS one of the currently discussed PEPs may let you write
def mainpage(apache _request, some_cgi_arg)[page_wrapper]: ...
instead of writing mainpage = page_wrapper(ma inpage) after the function
body.

[Jacek Trzmiel]

"Karl A. Krueger" wrote:

I'm in the middle of refactoring a small mod_python Web application,
which uses the Publisher handler.
I have never used Publisher handler myself. Yes, I've read about it,
but I've decided to not use it after seeing that I will have to
explicitly tell which modules/functions I DON'T want external user to
be able to run - completely broken idea from security POV.
One thing I would like to do is factor all of the top-level database
connecting, top-level exception handling, and error-page generation into
a single place. Whenever an exception occurs not trapped locally --
including an error connecting to the database -- I want to generate an
HTML error page instead of throwing the stacktrace at the user. [...] One thing I have considered is to handle _all_ the pages through a
single function, which would look up the specific pages by name:
If you do this then you may as well drop Publisher handler completely.
One handle() function will work for you.
... but this has the problem that it sacrifices the argument name
matching which is a major useful feature of mod_python Publisher.
Not so big, as you need to validate arguments format anyway. With calls
like this:
pos = self.GetFsInt( 'posId' )
lastDate = self.GetFsDate( 'lastDate' )
you can get arguments from FieldStorage, ensure that arguments are
present and convert them to usable format (e.g. string=>datetim e.date).
Some examples:

def RaiseMangled( self ):
raise apache.SERVER_R ETURN, apache.HTTP_BAD _REQUEST

def RaiseNotFound( self ):
raise apache.SERVER_R ETURN, apache.HTTP_NOT _FOUND

def GetFsString( self, key, default=None ):
if(default != None):
if self.fs.has_key (key):
value = self.fs[key]
else:
value = default
else:
value = self.fs[key]
if not isinstance( value, str ):
self.RaiseMangl ed()
return value

def GetFsInt( self, key ):
try:
return int( self.GetFsStrin g(key,'') )
except ValueError:
self.RaiseMangl ed()

def GetFsDate( self, key ):
try:
return Url2Date( self.GetFsStrin g(key,'') )
except ValueError:
self.RaiseMangl ed()

I've used one method in class to handle db connections, dispatch
requests and handle errors:

def handler( req ):
h = Handler()
return h.Handle( req )

class Handler:
[...]
def Handle(req):
try:
db = DbAccess.Connec tDb( dbName, host, user )
self.fs = util.FieldStora ge( self.req, True )
[...]
self.req.add_co mmon_vars()
env = req.subprocess_ env
self.script = env['SCRIPT_NAME']
self.cmd = self.script[ rfind(self.scri pt,'/')+1: ]

if self.cmd == '':
self.HandleMenu ( db )
elif self.cmd == 'PosDailyFeed':
self.HandlePosD ailyFeed( db )
elif self.cmd == 'PosDailySubmit ':
self.HandlePosD ailySubmit( db )
[...]
else:
self.RaiseNotFo und()
[...]
except:
[...]

# handler for urls like this:
# /PosDailyFeed?po sId=3&date=2004-03-22
def HandlePosDailyF eed( self, db ):
pos = self.GetFsInt( 'posId' )
date = self.GetFsDate( 'date' )
pb = self.CreatePage Builder( db )
valueDict = db.GetPosWorker Data( date, pos )
BuildDailyFeedP age( pb, db, date, pos, valueDict,
String2Html(db. GetPosName(pos) ) )
self.SendPage( pb )

I'm not sure if keyword args are exactly the Right Thing either. It also
would make the URLs even longer than they already are, though I can
kludge that with Apache URL rewriting rules. (Bleah.)

If you use straight handler() function instead of Publisher handler then
you have complete freedom how your URLs will look like.

Best regards,
Jacek.

[Jeff Epler]

One more idea -- I don't know mod_python, but is the Publisher class
something you could subclass, to add your own behavior there instead of
in each page function?

Jeff

[Karl A. Krueger]

Jeff Epler <je****@unpytho nic.net> wrote:

page_wrapper() could do extra magic with exec if you want it to have the
same signature as callable.. something like
def page_wrapper(ca llable):
d = {}
name = callable.__name __
argspec = getargspecstr(c allable) # based on inspect.getargs pec()
callspec = ... # argspec with default values removed
exec """def %s(%s): ... return callable(%s) ...""" \
% (name, argspec, callspec) in d
return d[name]
... barf, eh?

Barf? I -did- say I kind of wanted a Lisp macro; exec is about as close
as Python can get, I suppose.

Anyway, this has given me useful thoughts. Of course, I may just end up
back with keyword arguments. :-/

(But probably -not- reimplementing my whole application in Lisp. I'm
not that good in Lisp yet.)

--
Karl A. Krueger <kk******@examp le.edu>
Woods Hole Oceanographic Institution
Email address is spamtrapped. s/example/whoi/
"Outlook not so good." -- Magic 8-Ball Software Reviews

[Karl A. Krueger]

Jacek Trzmiel <sc***@hot.pl > wrote:

"Karl A. Krueger" wrote:

I'm in the middle of refactoring a small mod_python Web application,
which uses the Publisher handler.

I have never used Publisher handler myself. Yes, I've read about it,
but I've decided to not use it after seeing that I will have to
explicitly tell which modules/functions I DON'T want external user to
be able to run - completely broken idea from security POV.

Actually, Publisher never exposes modules imported into yours. It
traverses your module for names and types before exposing anything -- it
won't expose any object whose name begins with an underscore, or any
module.

(Oddly enough, it does expose exceptions, but just their string
representation. )

If anyone reading this is concerned about Publisher security, these
audit functions might help:

def AuditModuleForP ublisher(module ):
import types
for name in module.__dict__ :
object = module.__dict__[name]
if name.startswith ("_"):
# name starts with an underscore, not exposed
pass
elif type(object) == types.ModuleTyp e:
# object is a module, not exposed
pass
elif type(object) == types.FunctionT ype:
print ":: Exposed function:", name
elif isinstance(obje ct, str) or isinstance(obje ct, unicode):
if "passw" in name or "PASSW" in name:
print "!! Your %s password is %s." % (name, object)
else:
print ":: Exposed string:", name
else:
print ":: Exposed variable:", name

A nicer one:

def SymbolsExposedT oPublisher(modu le):
return [ sym for sym in module.__dict__
if (not sym.startswith( "_"))
and (type(module.__ dict__[sym]) != type(module)) ]

By enumerating the list of functions and variables you *intend* to
expose, a unit test should not be too hard to derive from this.

One thing I have considered is to handle _all_ the pages through a
single function, which would look up the specific pages by name:

If you do this then you may as well drop Publisher handler completely.
One handle() function will work for you.

I agree. However, since I want the functionality of Publisher's
argument name mapping, that is not what I want to do.

--
Karl A. Krueger <kk******@examp le.edu>
Woods Hole Oceanographic Institution
Email address is spamtrapped. s/example/whoi/
"Outlook not so good." -- Magic 8-Ball Software Reviews

[Dave Benjamin]

In article <ma************ *************** *********@pytho n.org>, Jeff Epler wrote:

One more idea -- I don't know mod_python, but is the Publisher class
something you could subclass, to add your own behavior there instead of
in each page function?

I suppose I should be embarrassed to say this, but I just made a copy and
modified it to suit my own needs. I don't think its design is particularly
extensible, but it's a small script that's easy to understand.

--
..:[ dave benjamin: ramen/[sp00] -:- spoomusic.com -:- ramenfest.com ]:.
: please talk to your son or daughter about parametric polymorphism. :