473,287 Members | 1,588 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

.py to sqlite translator [1 of 2]

Disclaimer(s): the author is nobody's pythonista. This could probably
be done more elegantly.
The driver for the effort is to get PyMacs to work with new-style
classes.
This rendering stage stands alone, and might be used for other
purposes.
A subsequent post will show using the resulting file to produce (I
think valid) .el trampoline
signatures for PyMacs.
If nothing else, it shows some python internals in an interesting way.
Tested against version 2.5.1
Maybe "lumberjack.py" would be a better name, since "It cuts down
trees, goes real slow, and uses disk galore. Wishes it'd been
webfoot[1], just like its dear author".
Cheers,
Chris

[1] Author was born in Oregon.

#A sample file:
class sample( object ):
"""fairly trivial sample class for demonstration purposes.
"""
def __init__( self
, some_string ):
self.hold_it = some_string

def show( self ):
print self.hold_it

#Invocation:
# ./pysqlrender.py -f sample.py -o output

#Script:
#!/usr/bin/python

"""Script to dump the parse tree of an input file to a SQLite
database.
"""

from optparse import OptionParser
import os
import parser
import pprint
import re
import sqlite3
import symbol
import token
import types

from types import ListType \
, TupleType

target_table = """CREATE TABLE tbl_parse_tree (
parse_tree_id INTEGER PRIMARY KEY
AUTOINCREMENT
, parse_tree_symbol_id
, parse_tree_indent
, parse_tree_value );"""

target_insert = """INSERT INTO tbl_parse_tree (
parse_tree_symbol_id
, parse_tree_indent
, parse_tree_value )
VALUES (%s, %s, '%s' );"""

symbol_table = """CREATE TABLE tlp_parse_tree_symbol (
parse_tree_symbol_id INTEGER PRIMARY KEY
, parse_tree_symbol_val );"""
symbol_insert = """INSERT INTO tlp_parse_tree_symbol (
parse_tree_symbol_id
, parse_tree_symbol_val )
VALUES ( %s, '%s' );"""

class symbol_manager( object ):
""" Class to merge symbols and tokens for ease of use.
"""
def __init__( self
, c ):
for k in symbol.sym_name:
sql = symbol_insert % ( k, symbol.sym_name[k] )
try:
c.execute( sql )
except sqlite3.IntegrityError:
pass
for k in token.tok_name:
sql = symbol_insert % ( k, token.tok_name[k] )
try:
c.execute( sql )
except sqlite3.IntegrityError:
pass

def get_symbol( self
, key ):
ret = -1
if symbol.sym_name.has_key(key): ret = symbol.sym_name[key]
elif token.tok_name.has_key(key) : ret = token.tok_name[ key]
return ret

def recurse_it( self, tester ):
"""Check to see if dump_tup should recurse
"""
if self.get_symbol(tester) 0:
return True
return False

class stocker( object ):
"""Remembers the depth of the tree and effects the INSERTs
into the output file.
"""
def __init__( self ):
self.cur_indent = 0

def do_symbol( self
, c
, symbol_value
, val = "" ):
"""Stuff something from the parse tree into the database
table.
"""
if symbol_value==5: self.cur_indent += 1
elif symbol_value==6: self.cur_indent -= 1

try:
sql = target_insert \
% ( symbol_value
, self.cur_indent
, re.sub( "'", "`", str(val) ))
c.execute( sql )
except AttributeError:
print "connection bad in lexer"
except sqlite3.OperationalError:
print "suckage at indent of %s for %s" \
% (self.cur_indent, sql)

def dump_tup( tup
, sym
, c
, stok ):
"""Recursive function to descend TUP and analyze its elements.
tup parse tree of a file, rendered as a tuple
sym dictionary rendered from symbol module
c live database cursor
stok output object effect token storage
"""
for node in tup:
typ = type( node )
r = getattr( typ
, "__repr__"
, None )

if (issubclass(typ, tuple) and r is tuple.__repr__):

if token.tok_name.has_key( node[0] ):
stok.do_symbol( c
, node[0]
, node[1] )
elif sym.recurse_it( node[0] ):
stok.do_symbol( c
, node[0]
, '__py__' ) #If you say node[1] here,
# the sqlite file is fat
# and instructive
for node2 in node[1:]:
dump_tup( node2
, sym
, c
, stok )
else:
stok.do_symbol( c
, node[0]
, node[1] )
dump_tup( node[1]
, sym
, c
, stok )
else:
stok.do_symbol( c
, 0
, node )
def convert_python_source_tree_to_table( file_name
, target_name ):
"""Retrieve information from the parse tree of a source file.
Create an output database file in sqlite.
Make a table in there, and then procede to stuff the flattened
input parse tree into it.

file_name Name of the file to read Python source code from.
target_name Name for the sqlite database
"""
x = open( file_name ).readlines()
y = []
[y.append( line.replace("\r\n","") ) for line in x]

ast = parser.suite( "\n".join(y) )
conn = sqlite3.connect( target_name )
conn.isolation_level = None
c = conn.cursor()
c.execute( target_table )
c.execute( symbol_table )
sym = symbol_manager( c )
stok = stocker()

#pprint.pprint( ast.totuple() )
dump_tup( ast.totuple()
, sym
, c
, stok )

def main():
usage = "usage: %prog [options] arg"
parser = OptionParser(usage)
parser.add_option("-f", "--file", dest="filename"
, action="store", type="string"
, help ="read python source from FILENAME")
#TODO: test for existence of output file, eject if exists
parser.add_option("-o", "--output",dest="output"
, action="store", type="string"
, help ="name of sqlite output file")
(options, args) = parser.parse_args()

convert_python_source_tree_to_table( options.filename
, options.output )

if __name__ == "__main__":
main()

Oct 26 '07 #1
0 1414

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: David Fowler | last post by:
I'm new to getting in touch with other PHP users/PHP team, so please excuse me if this post is at all inappropriate or in the wrong place. In the current release of PHP (5.1.4) and in the CVS for...
12
by: John Salerno | last post by:
I've been looking around and reading, and I have a few more questions about SQLite in particular, as it relates to Python. 1. What is the current module to use for sqlite? sqlite3? or is that not...
4
by: Jim Carlock | last post by:
I added the following lines to PHP.INI. extension=php_pdo.dll extension=php_pdo_sqlite.dll extension=php_sqlite.dll specifically in that order. I noticed the extensions getting loaded are...
10
by: Luigi | last post by:
Hello all! I'm a newbie in PHP. I have written a short script that tries to update a SQLite database with the user data. It is pretty simple, something like this: <?php $sqlite =...
9
by: Gilles Ganault | last post by:
Hello I was looking for a lighter web server than Apache, and installed Lighttpd on CentOS through yum. It works fine, but I now need to use SQLite from a PHP script. I seem to understand that...
8
by: Gilles Ganault | last post by:
Hello I need to compile PHP5 with SQLite statically and without PDO. I don't need DB-independence, so it's fine using sqlite_*() functions instead of PDO. 1. I've downloaded, compiled, and...
3
by: Daniel Fetchinson | last post by:
Does Python 2.5.2's embedded SQLite support full text searching? Sqlite itself is not distributed with python. Only a python db api compliant wrapper is part of the python stdlib and as such it...
0
by: Joe Goldthwaite | last post by:
Thanks Guilherme. That helped. I guess I was thinking that pysqlite would automatically come with some version of sqlite. The fact that it doesn't is what was causing me to get the strange...
20
by: timotoole | last post by:
Hi all, On a (sun) webserver that I use, there is python 2.5.1 installed. I'd like to use sqlite3 with this, however sqlite3 is not installed on the webserver. If I were able to compile sqlite...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.