473,403 Members | 2,338 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,403 software developers and data experts.

Code: Rolling a Container Into a String

I want to convert a dict into string form, then back again. After
discovering that eval is insecure, I wrote some code to roll a Python
object, dict, tuple, or list into a string. I've posted it below. Does
anyone know an easier way to accomplish this? Essentially, I want to
avoid doing an 'eval' on a string to get it back into dict form... but
still allow nested structures. (My current code doesn't handle nested
structures.)

I conked out before writing the 'unroll' code. I'm going to go watch
some boob tube with my husband, instead, and leave that code for
another day. If you know of a better way, or feel like writing the
'unroll' code and posting it, by all means, do so! :-D I'll check
Google newsgroups before I tackle the job, to see if some kind soul
took pity on me.

--Kamilche

import types

SimpleTypes = [types.BooleanType, types.FloatType, types.IntType, \
types.LongType, types.NoneType, types.StringType]

_dictdelim1 = "{"
_dictdelim2 = "}"
_listdelim1 = "["
_listdelim2 = "]"
_tupledelim1 = "("
_tupledelim2 = ")"

def roll(item):
"Return a string representation of an object, dict, tuple, or
list."
return _roll2(item, [], {})
def unroll(s):
"Unrolls a string back into a dict, tuple, or list."
if type(s) != types.StringType:
raise Exception("You may only pass strings to this function!")
err = "Error occurred when parsing " + s + "!"
state = 0
container = None
for c in s:
if c == _dictdelim1:
lookfor = _dictdelim2
elif c == _listdelim1:
lookfor = _listdelim2
elif c == _tupledelim1:
lookfor = _tupledelim2
else:
raise Exception(err)
state = 1

def _quoted(s):
' Return a stringized value'
if type(s) != types.StringType:
return str(s)
else:
l = []
s = s.replace("'", "\'")
l.append("'")
l.append(s)
l.append("'")
return ''.join(l)

def _roll2(d, lst, r):
' Function that does the work.'
# Start of _roll2
t = type(d)
if t == types.DictType:
theid = id(d)
if theid in r:
raise Exception("Recursion detected! Stopping now.")
r[theid] = theid
cnt = 0
lst.append(_dictdelim1)
for key in d.keys():
if key[0] != '_':
lst.append(_quoted(key))
lst.append(': ')
t = type(d[key])
if t in SimpleTypes:
lst.append(_quoted(d[key]))
else:
_roll2(d[key], lst, r)
lst.append(", ")
cnt += 1
if cnt > 0:
del lst[-1]
lst.append(_dictdelim2)
elif t in (types.ListType, types.TupleType):
theid = id(d)
if theid in r:
raise Exception("Recursion detected! Stopping now.")
r[theid] = theid
cnt = 0
if t == types.ListType:
lst.append(_listdelim1)
else:
lst.append(_tupledelim1)
for item in d:
if type(item) in SimpleTypes:
lst.append(_quoted(item))
else:
_roll2(item, lst, r)
lst.append(", ")
cnt += 1
if cnt > 0:
del lst[-1]
if t == types.ListType:
lst.append(_listdelim2)
else:
lst.append(_tupledelim2)
elif hasattr(d, '__dict__'):
_roll2(d.__dict__, lst, r)
else:
raise Exception("Unhandled type " + str(t) + \
"! You may only pass dicts, tuples, lists, and
" + \
"objects with a __dict__ to this function!")
return ''.join(lst)


class simple:
pass

def main():
l = ['List1', 'List2', 'List3']
d = {'Dict1': 'd1', 'Dict2': 'd2', 'list': l}
t = ('Tuple1', d, 'Tuple2')

print "It handles dicts, lists, and tuples."
print roll(t), "\n"

o = simple()
o.name = 'the name'
o.password = 'the password'
o.list = ['ol1', 'ol2']
o.dict = {'od1': None, 'od2': 2}
o.tuple = ('tuple1', 'tuple2')
o.float = 1.5
o.long = 12345678901234567890
o.bool = True
o.int = 1

print "It handles objects."
print roll(o), "\n"

print "It won't roll attributes whose name starts with '_'"
o._recursion = o.tuple
print roll(o), "\n"

print "It will raise an exception if it detects recursion."
print "This next one will cause recursion."
o.recursion = o.tuple
print roll(o), "\n"

print "You can't roll simple types."
print roll('a'), "\n"

if __name__ == "__main__":
main()
Jul 18 '05 #1
7 1670

"Kamilche" <kl*******@home.com> wrote in message
news:88**************************@posting.google.c om...
I want to convert a dict into string form, then back again. After
discovering that eval is insecure,
With arbitrary code from an arbitrary source, yes.
If you *know* that you are eval-ing your own safe strings, then no problem.
I wrote some code to roll a Python
object, dict, tuple, or list into a string.


repr(object) already does that for you. Why duplicate the work?

You only need custom a eval function, which might check that string is safe
(no function calls, no list comps) and then eval, or which might do parsing
and construction itself.

Terry J. Reedy
Jul 18 '05 #2
Terry Reedy wrote:
"Kamilche" <kl*******@home.com> wrote in message
news:88**************************@posting.google.c om...
I want to convert a dict into string form, then back again. After
discovering that eval is insecure,

With arbitrary code from an arbitrary source, yes.
If you *know* that you are eval-ing your own safe strings, then no problem.

I wrote some code to roll a Python
object, dict, tuple, or list into a string.

repr(object) already does that for you. Why duplicate the work?

You only need custom a eval function, which might check that string is safe
(no function calls, no list comps) and then eval, or which might do parsing
and construction itself.

Terry J. Reedy

Or use the pprint module which does nice pretty-printing

David
Jul 18 '05 #3
"Kamilche" <kl*******@home.com> wrote in message
news:88**************************@posting.google.c om...
I want to convert a dict into string form, then back again. After
discovering that eval is insecure, I wrote some code to roll a Python
object, dict, tuple, or list into a string. I've posted it below. Does
anyone know an easier way to accomplish this? Essentially, I want to
avoid doing an 'eval' on a string to get it back into dict form... but
still allow nested structures. (My current code doesn't handle nested
structures.)

unroll is just repr().

Here is a pyparsing routine to "roll up" your data structures, a mere 60
lines or so. It's fairly tolerant of some odd cases, and fully handles
nested data. (Extension to include remaining items, such as boolean data
and scientific notation, is left as an exercise for the reader.) I hope
this is fairly easy to follow - I've dropped in a few comments.

For those of you who've been living in a cave, you can download pyparsing at
http://pyparsing.sourceforge.net.

-- Paul
from pyparsing import Word, ZeroOrMore, OneOrMore, Suppress, Forward, \
quotedString,nums,Combine,Optional,delimitedList,G roup

# create a dictionary of ugly data, complete with nested lists, tuples
# and dictionaries, even imaginary numbers!
d1 = {}
d1['a'] = [1,2,3,[4,5,6]]
d1['b'] = (7,8,(9,10),'a',"",'')
d1['c'] = { 'aa' : 1, 'bb' : "lskdj'slkdjf", 'cc':1.232, 'dd':(('z',),) }
d1[('d','e')] = 5+10j

print repr(d1)

testdata = repr(d1)
"""
looks like this:
{'a': [1, 2, 3, [4, 5, 6]], ('d', 'e'): (5+10j), 'c': {'aa': 1, 'cc': 1.232,
'dd': (('z',),), 'bb': "lskdj'slkdjf"}, 'b': (7, 8, (9, 10), 'a', '', '')}
"""

#define low-level data elements
intNum = Word( nums+"+-", nums )
realNum = Combine(intNum + "." + Optional(Word(nums)))
number = realNum | intNum
imagNum = Combine( "(" + number + "+" + number + "j" + ")" )

item = Forward() # set up for recursive grammar definition
tupleDef = Suppress("(") + ( delimitedList( item ) ^
( item + Suppress(",") ) ) + Suppress(")")
listDef = Suppress("[") + delimitedList( item ) + Suppress("]")
keyDef = tupleDef | quotedString | imagNum | number
keyVal = Group( keyDef + Suppress(":") + item )
dictDef = Suppress("{") + delimitedList( keyVal ) + Suppress("}")

item << ( quotedString | number | imagNum |
tupleDef | listDef | dictDef )

# define low-level conversion routines
intNum.setParseAction( lambda s,loc,toks: int(toks[0]) )
realNum.setParseAction( lambda s,loc,toks: float(toks[0]) )
imagNum.setParseAction( lambda s,loc,toks: eval(toks[0]) ) # no built-in to
convert imaginaries?

# strip leading and trailing character from parsed quoted string
quotedString.setParseAction( lambda s,loc,toks: toks[0][1:-1] )

# define list-to-list/tuple/dict routines
evalTuple = lambda s,loc,toks: [ tuple(toks) ]
evalList = lambda s,loc,toks: [ toks.asList() ]
evalDict = lambda s,loc,toks: [ dict([tuple(kv) for kv in toks]) ]

tupleDef.setParseAction( evalTuple )
listDef.setParseAction( evalList )
dictDef.setParseAction( evalDict )

# first element of returned tokens list is the reconstructed list/tuple/dict
results = item.parseString( testdata )[0]
print results

if repr(results) == repr(d1):
print "Eureka!"
else:
print "Compare results for mismatch"
print repr(results)
print repr(d1)
Jul 18 '05 #4

Use YAML

import yaml

then from your code:

yaml.dump( whatever ) =>

then yaml.loadstring(str)...

It handles objects.
{'name': 'the name', 'tuple': ('tuple1', 'tuple2'), 'int': 1, 'float':
1.5, 'list': ['ol1', 'ol2'], 'long': 12345678901234567890,
'dict': {'od1': None, 'od2': 2}, 'bool': True, 'password': 'the password'}

--- !!__main__.simple # instanciates class for you
bool: True
dict:
od1: ~
od2: 2
float: 1.5
int: 1
list:
- ol1
- ol2
long: 12345678901234567890
name: the name
password: the password
tuple:
- tuple1
- tuple2

It won't roll attributes whose name starts with '_'
{'name': 'the name', 'tuple': ('tuple1', 'tuple2'), 'int': 1, 'float':
1.5, 'list': ['ol1', 'ol2'], 'long': 12345678901234567890,
'dict': {'od1': None, 'od2': 2}, 'bool': True, 'password': 'the password'}

--- !!__main__.simple
_recursion: # yaml does not handle recursion (I have an old version)
- tuple1
- tuple2
bool: True
dict:
od1: ~
od2: 2
float: 1.5
int: 1
list:
- ol1
- ol2
long: 12345678901234567890
name: the name
password: the password
tuple:
- tuple1
- tuple2

It will raise an exception if it detects recursion.
This next one will cause recursion.
--- !!__main__.simple
_recursion:
- tuple1
- tuple2
bool: True
dict:
od1: ~
od2: 2
float: 1.5
int: 1
list:
- ol1
- ol2
long: 12345678901234567890
name: the name
password: the password
recursion:
- tuple1
- tuple2
tuple:
- tuple1
- tuple2
Jul 18 '05 #5
> I want to convert a dict into string form, then back again.

Since you probably don't want to do this just for the joy of
converting something to string an back ...

Maybe the pickle (and cPickle) module does what you are looking for.

Oliver
Jul 18 '05 #6
Pierre-Frédéric Caillaud <pe****@free.fr> wrote in message news:<opr941cav01v4ijd@musicbox>...
Use YAML

It looked interesting, so I downloaded it... and was confronted with
dozens of files, and the need to compile before use... when I was
looking for a simple cross-platform 2 function solution that didn't
take any DLL's. Dang.

Well, it's a new day, maybe I'll be inspired.
Jul 18 '05 #7

did you download syck or the pure python yaml parser ?
on Linux the pure python module is just a matter of typing "emerge sync"
but I don't know about Syck...

On 25 Jun 2004 13:36:31 -0700, Kamilche <kl*******@home.com> wrote:
Pierre-Frédéric Caillaud <pe****@free.fr> wrote in message
news:<opr941cav01v4ijd@musicbox>...
Use YAML

It looked interesting, so I downloaded it... and was confronted with
dozens of files, and the need to compile before use... when I was
looking for a simple cross-platform 2 function solution that didn't
take any DLL's. Dang.

Well, it's a new day, maybe I'll be inspired.


--
Using Opera's revolutionary e-mail client: http://www.opera.com/m2/
Jul 18 '05 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Maitre Bart | last post by:
What I want to perform is calling a member function of container 1 (CRunner), using as argument the value of a container 2 (CNames). The purpose is to transfer values from C2 into C1 using a...
6
by: Dan McCollick | last post by:
Hi All, I have a very small screen scrape application, that has a small problem. when I run the app and I have fiddler(an http tool to view what is being sent by the requests/responses,...
6
by: cpnet | last post by:
I've authored a custom web component (a non-ui component kinda like a DataSet) and so far it's working. When my web component is added to the web form in the designer from the toolbox, the...
6
by: Paolo Pignatelli | last post by:
I have an aspx code behind page that goes something like this in the HTML view: <asp:HyperLink id=HyperLink1 runat="server" NavigateUrl='<%#"mailto:" &amp;...
4
by: Larry Grady | last post by:
Anyone up for a challenge? I've been struggling with this for a few days and was hoping someone could help me. Pouring through all the messageboards I just can't find the solution. We have a...
1
by: Neo Geshel | last post by:
I am having conflicting results with two pieces of identical code. One is an insert, the other is an update to a db. The first code, which works, is this: Sub Add3_Click(sender As Object, e As...
2
by: mikepolitowski | last post by:
Hi folks, I am have been trying to solve this problem for quite some time now and would appreciate any advice. I have been trying to call a code-behind function that is defined in my aspx.cs...
2
by: MrCrool | last post by:
Hi I need to be able to handle the following ASP programming in pure C# code: <asp:TemplateColumn HeaderText="Customer Information"> <ItemTemplate> <table border="0"> <tr> <td...
3
by: peterhall | last post by:
In VBA an Access module has a find method - works perfectly to find a string inside a module. i'm working in A97 (legacy) systems (large ones) and want to write code that searches all modules so that...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.