By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
464,735 Members | 1,248 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 464,735 IT Pros & Developers. It's quick & easy.

problem doing unpickle in an exec statement

P: n/a
Howdy,

In my app I need to exec user text that defines a function. I want this
function to unpickle an object. Pickle breaks because it is looking for
the object definition that isn't in the calling namespace.

I have mocked up a simple example that shows the problem. Run this
first code (from create_pickle.py) to create the pickle.

create_pickle.py: (run this first)

#############################################
import cPickle

# the pickle file name
file_name = 'd:\\temp\\test1.pickle'

# define a class
class Tree(object):
pass
def main():
# instantiate
t = Tree()

# create the sweet pickle
fp = open(file_name, 'wb')
cPickle.dump(t, fp)
fp.close()

# try to unpickle directly
fp = open(file_name, 'rb')
result = cPickle.load(fp)
fp.close()
print "unpickling directly works just fine, result = ", result

if __name__=='__main__':
main()
#############################################

run this second:

exec_pickle.py
#############################################
# this file shows a problem with sweet pickle in an exec statement

# the pickle file name
file_name = 'd:\\temp\\test1.pickle'

# code to be turned into a function
code_text = '''
def include():
print "this works!"
'''

# a function for creating functions
def create_fun(code_text):
clean_dict = {}
exec code_text in clean_dict
return clean_dict['include']

# include_fun is a bona fide function
include_fun = create_fun(code_text)

# this works
include_fun()
# now try to load the pickle in an exec statement
code_text = '''
def include(file_name):
print "processing file_name: ", file_name
import cPickle
fp = open(file_name, "rb")
result = cPickle.load(fp)
fp.close()
print "result = ", result
'''

# create the new include_fun
include_fun = create_fun(code_text)

# run it
include_fun(file_name)

#############################################

Can anyone enlighten me what I need to do to exec_pickle.py
to get this to work?

thanks,
Danny

Jul 23 '08 #1
Share this Question
Share on Google+
2 Replies

P: n/a
On Jul 23, 6:01*pm, Danny Shevitz <shev...@lanl.govwrote:
Howdy,

In my app I need to exec user text that defines a function. I want this
function to unpickle an object. Pickle breaks because it is looking for
the object definition that isn't in the calling namespace.

I have mocked up a simple example that shows the problem. Run this
first code (from create_pickle.py) to create the pickle.

create_pickle.py: (run this first)

#############################################
import cPickle

# the pickle file name
file_name = 'd:\\temp\\test1.pickle'

# define a class
class Tree(object):
* * * * pass

def main():
* * * * # instantiate *
* * * * t = Tree()

* * * * # create the sweet pickle
* * * * fp = open(file_name, 'wb')
* * * * cPickle.dump(t, fp)
* * * * fp.close()

* * * * # try to unpickle directly
* * * * fp = open(file_name, 'rb')
* * * * result = cPickle.load(fp)
* * * * fp.close()
* * * * print "unpickling directly works just fine, result = ",result

if __name__=='__main__':
* * * * main()
#############################################

run this second:

exec_pickle.py
#############################################
# this file shows a problem with sweet pickle in an exec statement

# the pickle file name
file_name = 'd:\\temp\\test1.pickle'

# code to be turned into a function
code_text = '''
def include():
* print "this works!"
'''

# a function for creating functions
def create_fun(code_text):
* * * * clean_dict = {}
* * * * exec code_text in clean_dict
* * * * return clean_dict['include']

# include_fun is a bona fide function
include_fun = create_fun(code_text)

# this works
include_fun()

# now try to load the pickle in an exec statement
code_text = '''
def include(file_name):
* print "processing file_name: ", file_name
* import cPickle
* fp = open(file_name, "rb")
* result = cPickle.load(fp)
* fp.close()
* print "result = ", result
'''

# create the new include_fun
include_fun = create_fun(code_text)

# run it
include_fun(file_name)

#############################################

Can anyone enlighten me what I need to do to exec_pickle.py
to get this to work?

thanks,
Danny
Hi,

It works if you paste

# define a class
class Tree(object):
pass

into exec_pickle.py. There are inherent dilemmas in pickling
instances of non-primitive types, such as definition location, and
change in definition across versions. How does 'class Tree' fit in to
your design module?
Jul 24 '08 #2

P: n/a
Danny Shevitz wrote:
Howdy,

In my app I need to exec user text that defines a function. I want this
function to unpickle an object. Pickle breaks because it is looking for
the object definition that isn't in the calling namespace.

I have mocked up a simple example that shows the problem. Run this
first code (from create_pickle.py) to create the pickle.

create_pickle.py: (run this first)

#############################################
import cPickle

# the pickle file name
file_name = 'd:\\temp\\test1.pickle'

# define a class
class Tree(object):
pass
def main():
# instantiate
t = Tree()

# create the sweet pickle
fp = open(file_name, 'wb')
cPickle.dump(t, fp)
fp.close()

# try to unpickle directly
fp = open(file_name, 'rb')
result = cPickle.load(fp)
fp.close()
print "unpickling directly works just fine, result = ", result

if __name__=='__main__':
main()
#############################################

run this second:

exec_pickle.py
#############################################
# this file shows a problem with sweet pickle in an exec statement

# the pickle file name
file_name = 'd:\\temp\\test1.pickle'

# code to be turned into a function
code_text = '''
def include():
print "this works!"
'''

# a function for creating functions
def create_fun(code_text):
clean_dict = {}
exec code_text in clean_dict
return clean_dict['include']

# include_fun is a bona fide function
include_fun = create_fun(code_text)

# this works
include_fun()
# now try to load the pickle in an exec statement
code_text = '''
def include(file_name):
print "processing file_name: ", file_name
import cPickle
fp = open(file_name, "rb")
result = cPickle.load(fp)
fp.close()
print "result = ", result
'''

# create the new include_fun
include_fun = create_fun(code_text)

# run it
include_fun(file_name)

#############################################

Can anyone enlighten me what I need to do to exec_pickle.py
to get this to work?
Pickling saves the name of the module and the class (and of course the
instance data). Because you put the class in your main script the module
name is __main__, and when you unpickle later pickle imports __main__ which
unfortunately is now a different script that doesn't contain the/a Tree
class. To solve that problem put the Tree class in a separate module, say
tree.py, that can be imported by both create_pickle.py and exec_pickle.py.
You only need an explicit

from tree import Tree

in create_pickle.

Peter
Jul 24 '08 #3

This discussion thread is closed

Replies have been disabled for this discussion.