473,893 Members | 1,621 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Alternatives for pickle?

I'm writing a little game, a gridler application, where you
can turn pixmaps into puzzle's and try to solve them. I already
have the data structure for such a puzzle worked out, one of
the problems is writing it to a file and reading it back in.

I first went to the pickle module but there I read this.

| Warning: The pickle module is not intended to be secure against
| erroneous or maliciously constructed data. Never unpickle data
| received from an untrusted or unauthenticated source.

But since this is for a game and people should be able to
exchange puzzles, it seems a heavy requirement to ask of
the users to check a puzzle file for security hazards.
I also thought about writing out a string that, when read
back in and fed to eval would recreate the structure. But
that seems to be just as insecure if not more so.

So how do you serialize data in python, when you want
a somewhat secure mechanisme. Preferably where a user
can make a puzzle file by hand in a text editor.

--
Antoon Pardon
Jul 18 '05 #1
7 3321
Antoon Pardon wrote:
So how do you serialize data in python, when you want
a somewhat secure mechanisme. Preferably where a user
can make a puzzle file by hand in a text editor.


There's a YAML module for Python, although if I recall
correctly this module also suffers from security issues.

You could try Gnosis utils xml pickling or the xml marshaler
from the pyxml package. They're slow, but safe.

--Irmen

Jul 18 '05 #2
Antoon Pardon wrote:
I'm writing a little game, a gridler application, where you
can turn pixmaps into puzzle's and try to solve them. I already
have the data structure for such a puzzle worked out, one of
the problems is writing it to a file and reading it back in.

I first went to the pickle module but there I read this.

| Warning: The pickle module is not intended to be secure against
| erroneous or maliciously constructed data. Never unpickle data
| received from an untrusted or unauthenticated source.

But since this is for a game and people should be able to
exchange puzzles, it seems a heavy requirement to ask of
the users to check a puzzle file for security hazards.
I also thought about writing out a string that, when read
back in and fed to eval would recreate the structure. But
that seems to be just as insecure if not more so.
Indeed. Don't do that.
So how do you serialize data in python, when you want
a somewhat secure mechanisme. Preferably where a user
can make a puzzle file by hand in a text editor.


I think this is a case where you need to come up with your own file
format and parse it yourself. Pickle and other such mechanisms have
security problems because they are so general. They will create objects
that you don't want.

You can always jump on the XML bandwagon if that is convenient for you.
Python has XML modules in the standard library. Depending on the
complexity of the structure, it might even be convenient to edit by hand
in a text editor.

--
Robert Kern
rk***@ucsd.edu

"In the fields of hell where the grass grows high
Are the graves of dreams allowed to die."
-- Richard Harter
Jul 18 '05 #3
Antoon Pardon wrote:
I also thought about writing out a string that, when read
back in and fed to eval would recreate the structure. But
that seems to be just as insecure if not more so.


As I understand it, this is *exactly* what pickle already does.
So how do you serialize data in python, when you want
a somewhat secure mechanisme. Preferably where a user
can make a puzzle file by hand in a text editor.


I'd agree with the earlier comment -- define your own file format, and
write code to parse that format and instantiate the necessary objects.
If it's hard to define something that's both effective for your
purposes, and hard to hand-code in a text editor, then consider writing
a puzzle-editor app that will allow GUI creation of puzzles which can be
saved in your custom file format.

Jeff Shannon
Technician/Programmer
Credit International
Jul 18 '05 #4
Antoon Pardon <ap*****@forel. vub.ac.be> wrote in message news:<sl******* *************@r cpc42.vub.ac.be >...
I'm writing a little game, a gridler application, where you
can turn pixmaps into puzzle's and try to solve them. I already
have the data structure for such a puzzle worked out, one of
the problems is writing it to a file and reading it back in.

I first went to the pickle module but there I read this.

| Warning: The pickle module is not intended to be secure against
| erroneous or maliciously constructed data. Never unpickle data
| received from an untrusted or unauthenticated source.

Hmmm..... I wonder how easy it is to craft a malicious pickle that
will automatically run code objects just because they are unpickled.
My guess is that it's quite difficult - I've never heard of it *ever*
being done. Someone would have to be *very* malicious to work out how
to do it on the off chance of planting a back door into someone's
machine through a program like yours. No offence intended, but if they
were going to go to all that effort I expect they might aim for
something with a wider audience.

I would expect it to be 'safe enough', but that might not be safe
enough for you !

Creating your own data format is probably the way forward - and
probably not that difficult either.

Regards,

Fuzzy

http://www.voidspace.org.uk/atlantib...thonutils.html
But since this is for a game and people should be able to
exchange puzzles, it seems a heavy requirement to ask of
the users to check a puzzle file for security hazards.
I also thought about writing out a string that, when read
back in and fed to eval would recreate the structure. But
that seems to be just as insecure if not more so.

So how do you serialize data in python, when you want
a somewhat secure mechanisme. Preferably where a user
can make a puzzle file by hand in a text editor.

Jul 18 '05 #5
Antoon Pardon wrote:
| Warning: The pickle module is not intended to be secure against
| erroneous or maliciously constructed data. Never unpickle data
| received from an untrusted or unauthenticated source.

But since this is for a game and people should be able to
exchange puzzles, it seems a heavy requirement to ask of
the users to check a puzzle file for security hazards.


http://twistedmatrix.com/products/spread#jelly

I haven't used it myself, though. In fact you might be able to use
twisted in other ways as well.

Shalabh

--
http://www.qlime.org

Jul 18 '05 #6
Antoon Pardon <ap*****@forel. vub.ac.be> writes:
So how do you serialize data in python, when you want
a somewhat secure mechanisme. Preferably where a user
can make a puzzle file by hand in a text editor.


There are a lot of different serialization formats in the Python
library but the general ones are not secure and the secure ones are
not general. You may have to concoct an ad-hoc format just for your
puzzles.
Jul 18 '05 #7
On 11 Oct 2004 08:26:12 GMT, Antoon Pardon <ap*****@forel. vub.ac.be> wrote:
I'm writing a little game, a gridler application, where you
can turn pixmaps into puzzle's and try to solve them. I already
have the data structure for such a puzzle worked out, one of
the problems is writing it to a file and reading it back in.

I first went to the pickle module but there I read this.

| Warning: The pickle module is not intended to be secure against
| erroneous or maliciously constructed data. Never unpickle data
| received from an untrusted or unauthenticated source.

But since this is for a game and people should be able to
exchange puzzles, it seems a heavy requirement to ask of
the users to check a puzzle file for security hazards.
I also thought about writing out a string that, when read
back in and fed to eval would recreate the structure. But
that seems to be just as insecure if not more so.

So how do you serialize data in python, when you want
a somewhat secure mechanisme. Preferably where a user
can make a puzzle file by hand in a text editor.

I would consider saving and retrieving your puzzle info in a simple csv format.
You can invent your own very simple interpreter based on lines of the
form
cmd, whatever...

the csv module has methods and options to control delimiters and quoting etc,
but e.g., by default:
import csv
lines = """ ... this, can, be, a, command, format
... cmd, easy to edit with editor, note what happened to spaces
... cmd, arg, "quoted arg", etc
... do, something, else
... push, something on a stack
... set, something,to,a, value
... call, afunction, arg1, arg2, etc
... etc
... """ rdr = csv.reader(line s.splitlines())
for row in rdr: print row ...
[]
['this', ' can', ' be', ' a', ' command', ' format']
['cmd', ' easy to edit with editor', ' note what happened to spaces']
['cmd', ' arg', ' "quoted arg"', ' etc']
['do', ' something', ' else']
['push', ' something on a stack']
['set', ' something', 'to', 'a', 'value']
['call', ' afunction', ' arg1', ' arg2', ' etc']
['etc']

Anything that will iterate by lines should be ok to pass to csv.reader
so you can pass an open file, e.g., file('mypuzzles etup.txt')

Obviously, you can interpret row[0] as operation and do what you like, e.g.,

rdr = csv.reader("""\ ... abs, 123
... abs, -123
... sum, 1,2,3
... sum, 100,200,5
... xxx, what?
... """.splitlines( ))
for row in rdr:

... cmd = getattr(__built ins__, row[0], None)
... if cmd is None: print 're-edit your info:',row
... else:
... args = map(int, row[1:])
... if len(args)==1: args=args[0]
... print row,'=>', cmd(args)
...
['abs', ' 123'] => 123
['abs', ' -123'] => 123
['sum', ' 1', '2', '3'] => 6
['sum', ' 100', '200', '5'] => 305
re-edit your info: ['xxx', ' what?']

The csv module also has stuff to control delimiters and a writer method, etc.
See help(csv) interactively, after importing csv.

Note that no one can introduce an xxx do do anything weird, and if you
validate all the command formats, and don't do int conversions etc without
try/except, etc, you should be able to reject anything not safe, and write
informative error messages. You don't necessarily have to abort on any and
all errors, but you can have options for that if you want to get fancy.

HTH

Regards,
Bengt Richter
Jul 18 '05 #8

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
4025
by: Michael Hohn | last post by:
Hi, under python 2.2, the pickle/unpickle sequence incorrectly restores a larger data structure I have. Under Python 2.3, these structures now give an explicit exception from Pickle.memoize(): assert id(obj) not in self.memo I'm shrinking the offending data structure down to find the problem
0
1791
by: Mike P. | last post by:
Hi all, I'm working on a simulation (can be considered a game) in Python where I want to be able to dump the simulation state to a file and be able to load it up later. I have used the standard Python pickle module and it works fine pickling/unpickling from files. However, I want to be able to use a third party tool like an XML editor (or other custom tool) to setup the initial state of the simulation, so I have been playing around...
6
12364
by: Jim Lewis | last post by:
Pickling an instance of a class, gives "can't pickle instancemethod objects". What does this mean? How do I find the class method creating the problem?
10
4457
by: crystalattice | last post by:
I'm creating an RPG for experience and practice. I've finished a character creation module and I'm trying to figure out how to get the file I/O to work. I've read through the python newsgroup and it appears that shelve probably isn't the best option for various reasons. This lead me to try messing w/ pickle, but I can't figure out how to use it with classes. I've found many examples of using pickle w/ non-OOP code but nothing that...
5
93245
by: Chris | last post by:
Why can pickle serialize references to functions, but not methods? Pickling a function serializes the function name, but pickling a staticmethod, classmethod, or instancemethod generates an error. In these cases, pickle knows the instance or class, and the method, so what's the problem? Pickle doesn't serialize code objects, so why can't it serialize the name as it does for functions? Is this one of those features that's feasible, but...
3
6116
by: fizilla | last post by:
Hello all! I have the following weird problem and since I am new to Python I somehow cannot figure out an elegant solution. The problem reduces to the following question: How to pickle a collections.defaultdict object that has set the default_factory property? For Example (from the IDLE console): >>> words = collections.defaultdict(lambda: 1) >>> f = file("temp","w")
2
3547
by: Michele Simionato | last post by:
Can somebody explain what's happening with the following script? $ echo example.py import pickle class Example(object): def __init__(self, obj, registry): self._obj = obj self._registry = registry
2
4526
by: Nagu | last post by:
I am trying to save a dictionary of size 65000X50 to a local file and I get the memory error problem. How do I go about resolving this? Is there way to partition the pickle object and combine later if this is a problem due to limited resources (memory) on the machine (it is 32 bit machine Win XP, with 4GB RAM). Here is the detail description of the error:
1
6357
by: IceMan85 | last post by:
Hi to all, I have spent the whole morning trying, with no success to pickle an object that I have created. The error that I get is : Can't pickle 'SRE_Match' object: <_sre.SRE_Match object at 0x2a969c0ad0> the complete stack is the following : Traceback (most recent call last): File "manager.py", line 305, in ? commandLineExec (log, parser) File "manager.py", line 229, in commandLineExec
0
10839
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
10469
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9645
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
7173
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5858
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
6066
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4684
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4280
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3289
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.