self modifying code

[Robin Becker]

When young I was warned repeatedly by more knowledgeable folk that self
modifying code was dangerous.

Is the following idiom dangerous or unpythonic?

def func(a):
global func, data
data = somethingcomple xandcostly()
def func(a):
return simple(data,a)
return func(a)

It could be replaced by

data = somethingcomple xandcostly()
def func(a):
return simple(data,a)

but this always calculates data.
--
Robin Becker

[nikie]

Robin Becker schrieb:

When young I was warned repeatedly by more knowledgeable folk that self
modifying code was dangerous.

Is the following idiom dangerous or unpythonic?

def func(a):
global func, data
data = somethingcomple xandcostly()
def func(a):
return simple(data,a)
return func(a)
It took me quite a while to figure out how it works, so, yes, I'd say
it's unpythonic ;-). It's not really dangerous, but it can get nasty if
someone tries to rename the function, or put it into a class.

But that's probably not the kind of "self-modifying code" you've been
warned against anyway: I've only ever seen self-modifying code in
assembly language or in lisp, the idea is that you really change the
code (i.e. single opcodes in the function that's currently running), so
you can e.g. make an infinite loop, and eventually overwrite the loop
statement to do something else so the loop ends. I'm not sure if you
can do the same thing in Python, maybe by changing the bytecode of a
running function.
It could be replaced by

data = somethingcomple xandcostly()
def func(a):
return simple(data,a)

but this always calculates data.

You could of course initialize data with None and calculate it only on
demand. Or you could use:
http://www.phyast.pitt.edu/~micheles...n.html#memoize
This has the advantage of encapsulating the memoization logic so it can
be tested (and understood) separately from your code.

[Peter Otten]

Robin Becker wrote:

When young I was warned repeatedly by more knowledgeable folk that self
modifying code was dangerous.

Is the following idiom dangerous or unpythonic?

def func(a):
global func, data
data = somethingcomple xandcostly()
def func(a):
return simple(data,a)
return func(a)

It could be replaced by

data = somethingcomple xandcostly()
def func(a):
return simple(data,a)

but this always calculates data.

Consider

data = None
def func(a):
global data
if data is None:
data = costly()
return simple(data, a)

if you want lazy evaluation. Not only is it easier to understand,
it also works with

from lazymodule import func

at the cost of just one object identity test whereas your func()
implementation will do the heavy-lifting every time func() is called in the
client (unless func() has by chance been invoked as lazymodule.func ()
before the import).

Peter

[Robin Becker]

Peter Otten wrote:
.....

def func(a):
global func, data
data = somethingcomple xandcostly()
def func(a):
return simple(data,a)
return func(a)

.........
at the cost of just one object identity test whereas your func()
implementation will do the heavy-lifting every time func() is called in the
client (unless func() has by chance been invoked as lazymodule.func ()
before the import).

in the example code the heavy lifting, costly(), is done only once as
the function that does it is overwritten. As pointed out it won't work
as simply in a class. Memoisation could improve the performance of the
normal case form of the function ie

def func(a):
return simple(data,a)

but I guess that would depend on whether simple(data,a) is relatively
expensive compared to the costs the memo lookup.
--
Robin Becker

[John J. Lee]

Robin Becker <ro***@NOSPAMre portlab.com> writes:

When young I was warned repeatedly by more knowledgeable folk that self
modifying code was dangerous.

Is the following idiom dangerous or unpythonic?

def func(a):
global func, data
data = somethingcomple xandcostly()
def func(a):
return simple(data,a)
return func(a)

1. I don't think most people would call that "self-modifying code". I
won't try defining that term precisely because I know you'll just
pick holes in my definition ;-)

2. The use of global func is just plain weird :-)

3. Peter Otten's version is OK, but how about this, using a closure
instead of globals (UNTESTED)

def make_func():
namespace = object()
namespace.data = None
def func(a):
if namespace.data is None:
namespace.data = somethingcomple xandcostly()
return simple(namespac e.data, a)
return func
func = make_func()
John

[Robin Becker]

John J. Lee wrote:

1. I don't think most people would call that "self-modifying code". I
won't try defining that term precisely because I know you'll just
pick holes in my definition ;-)

Don't really disagree about the rewriting code, but the function does
re-define itself.

2. The use of global func is just plain weird :-)

3. Peter Otten's version is OK, but how about this, using a closure
instead of globals (UNTESTED)

def make_func():
namespace = object()
namespace.data = None
def func(a):
if namespace.data is None:
namespace.data = somethingcomple xandcostly()
return simple(namespac e.data, a)
return func
func = make_func()

........
the inner function is almost precisely what I started with, except I
used the global namespace. However, it keeps the test in side the
function which costs about 1%.
--
Robin Becker

[Ben C]

On 2006-04-29, Robin Becker <ro***@NOSPAMre portlab.com> wrote:

When young I was warned repeatedly by more knowledgeable folk that self
modifying code was dangerous.

Is the following idiom dangerous or unpythonic?

def func(a):
global func, data
data = somethingcomple xandcostly()
def func(a):
return simple(data,a)
return func(a)
It looks quite clever (a bit too clever ... :)
It could be replaced by

data = somethingcomple xandcostly()
def func(a):
return simple(data,a)

but this always calculates data.

Why not just:

data = None
def func(a):
global data

if not data:
data = somethingcomple xandcostly()

return simple(data, a)

Or nicer to use a "singleton" perhaps than a global, perhaps something
like this:

class Func(object):
exists = False

def __init__(self):
assert not Func.exists
Func.exists = True

self.data = None

def simple(self, a):
assert self.data is not None
# ... do something with self.data presumably
return something

def __call__(self, a):
if self.data is None:
self.data = somethingcomple xandcostly()
return self.simple(a)

func = Func()

func(a)

[Steven Bethard]

John J. Lee wrote:

Robin Becker <ro***@NOSPAMre portlab.com> writes:

When young I was warned repeatedly by more knowledgeable folk that self
modifying code was dangerous.

Is the following idiom dangerous or unpythonic?

def func(a):
global func, data
data = somethingcomple xandcostly()
def func(a):
return simple(data,a)
return func(a)

1. I don't think most people would call that "self-modifying code". I
won't try defining that term precisely because I know you'll just
pick holes in my definition ;-)

2. The use of global func is just plain weird :-)

3. Peter Otten's version is OK, but how about this, using a closure
instead of globals (UNTESTED)

def make_func():
namespace = object()
namespace.data = None
def func(a):
if namespace.data is None:
namespace.data = somethingcomple xandcostly()
return simple(namespac e.data, a)
return func
func = make_func()

Unfortunately, this doesn't work because you can add attributes to plain
object instances:

namespace = object()
namespace.data = None Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
AttributeError: 'object' object has no attribute 'data'

Maybe you want something like:
def make_func(): .... def func(a):
.... if func.data is None:
.... func.data = somethingcomple xandcostly()
.... return simple(func.dat a, a)
.... func.data = None
.... return func
.... func = make_func()
def somethingcomple xandcostly(): .... print 'executing somethingcomple xandcostly'
.... return 42
.... def simple(data, a): .... return data, a
.... func(1) executing somethingcomple xandcostly
(42, 1) func(2)

(42, 2)

STeVe

[Robin Becker]

Ben C wrote:
........

Why not just:

data = None
def func(a):
global data

if not data:
data = somethingcomple xandcostly()

return simple(data, a)

well in the original instance the above reduced to something like

data=None
def func(arg):
global data
if data:
data = ......
return ''.join(map(dat a.__getitem__,a rg))

so the actual function is pretty low cost, but the extra cost of the
test is actually not very significant, but if the actual function had
been cheaper eg

def func(arg):
global data
if data is None:
data = ....
return data+arg

then the test is a few percent of the total cost; why keep it?

All the other more complex solutions involving namespaces, singletons
etc seem to add even more overhead.
Or nicer to use a "singleton" perhaps than a global, perhaps something
like this:

class Func(object):
exists = False

def __init__(self):
assert not Func.exists
Func.exists = True

self.data = None

def simple(self, a):
assert self.data is not None
# ... do something with self.data presumably
return something

def __call__(self, a):
if self.data is None:
self.data = somethingcomple xandcostly()
return self.simple(a)

func = Func()

func(a)

--
Robin Becker

[taleinat]

First of all, the test can be optimized by adding a boolean flag which
indicates if the data has been initialized or not, and then just
testing a boolean value instead of doing an "is" comparison, at the
cost of an extra global variable. But this is still ugly (actually
uglier IMO).
I think this is a more Pythonic way to do it.
This class implements a function which initializes itself upon the
first call:

class InitializingFun ction(object):
def __init__(self, init):
def initializer(*ar gs, **kw):
self.func = init()
return self(*args, **kw)
self.func = initializer
def __call__(self, *args, **kw):
return self.func(*args , **kw)

Now you can write your function almost exactly like you did before:

def init():
data = somethingcomple xandcostly()
def foo(a):
return simple(data, a)
return foo
func = InitializingFun ction(init)

What have we gained from this? Two major advantages:
* no ugly 'global' statement
* no reliance on the function's name

And now you can easily create such functions forever using this class
to abstract away the ugly implementation ;)
Notice that since Function Decorators were introduced in Python2.4, you
can use InitializingFun ction as a Decorator to achieve the same effect,
this time even without the need for a temporary name for a function:

@InitializingFu nction
def func():
data = somethingcomple xandcostly()
def foo(a):
return simple(data, a)
return foo
And finally I must note that no matter which way you turn this around,
it will still be hard to read!