Function Serialization

[Mike Tammerman]

I want to serialize a function using its pointer. For example

s = """ .... def square(number):
.... return number**2
.... """ functions = {}
exec s in functions
f = functions['square']
f

<function square at 0xb7b5d10c>

Then,
1. Serialize f,
2. Store it into a file a db.

One day later,
3. Retrieve from file or db,
4. Unserialize it
5. Use as it is a normal function, maybe I would set it to an object
with setattr

Any help will be very useful.

Mike

[Michele Simionato]

The simple way is to use shelve:

$ ipython
Python 2.4.1c2 (#2, Mar 19 2005, 01:04:19)
Type "copyright", "credits" or "license" for more information.

IPython 0.6.5 -- An enhanced Interactive Python.
? -> Introduction to IPython's features.
%magic -> Information about IPython's 'magic' % functions.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: import shelve

In [2]: sh=shelve.open("x.shelve")

In [3]: def f(): pass
...:

In [4]: sh["f"]=f

In [5]: sh.close()

In [6]: sh=shelve.open("x.shelve")

In [7]: sh["f"]
Out[7]: <function f at 0xb7d5f614>

You may want to experiment a bit.

Michele Simionato

[Mike Tammerman]

Michele, thank you for the suggestion, it works as you mention.

But in my case the function is inside a string and I should exec the
string to get the function. Something like this

fields = {}
s = 'def f(): pass'
exec s in fields
f = fields['f']
f <function f at 0xb7a00bfc> import shelve
d = shelve.open('test.db')
d['f'] = f

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/lib/python2.4/shelve.py", line 129, in __setitem__
p.dump(value)
TypeError: expected string or Unicode object, NoneType found

I am not a pro python developer, so I couldn't understand why it throws
exception. I am a bit confused.

Sincerely,

Mike

[Lonnie Princehouse]

By default, shelve uses pickle to serialize objects. Pickle doesn't
actually serialize functions... it just saves the function's name and
name of its defining module, and upon loading attempts to find that
function again. This is pretty much /never/ going to work the way you
want it to if you're using exec.

If you really want to save a function, you're going to have to save its
source code and exec it upon reloading. I think the best way to
achieve this would be to make your own wrapper class which defines a
__call__ method and implements custom pickling routines (see the pickle
documentation for how to do this). You could probably even make it
into a decorator, but I wouldn't know about that since I'm stuck with
Python 2.3.

It would look something like this

def f(): pass
f = SerializableFunction(f)

where SerializableFunction is defined as:

class SerializableFunction(object):
def __init__(self, function):
self.function = function

def __call__(self, *args, **keywords):
return self.function(*args, **keywords)

# custom pickle routines
def __getnewargs__(self):
...
def __getstate__(self):
# extract function source
...
def __setstate__(self):
...

You can use the inspect module to get function source code, although
AFAIK it's impossible to get source from functions defined in the
interactive interpreter.