user-supplied locals dict for function execution?

[Lonnie Princehouse]

Can anyone think of a way to substitute a user-supplied dictionary as
the local dict for a function call?

e.g.

def f():
x = 5

d = {}

exec_function_in_dictionary( f, d ) # ???

print d["x"] # 5
#--------------

Right now, the way I'm doing this is to parse the function body out of
its source code and call exec on the body. This has problems ---
notably that it only works on functions for which source code can be
found, also that the parsing is tricky (but I can fix that).

[bruno at modulix]

Lonnie Princehouse wrote:

Can anyone think of a way to substitute a user-supplied dictionary as
the local dict for a function call?

e.g.

def f():
x = 5

d = {}

exec_function_in_dictionary( f, d ) # ???

print d["x"] # 5

def f(**kw):
kw['x'] = 5

d = {}
f(**d)
print d['x']

.... But I suppose this is not what you're looking for !-)
#--------------

Right now, the way I'm doing this is to parse the function body out of
its source code and call exec on the body. This has problems ---
notably that it only works on functions for which source code can be
found, also that the parsing is tricky (but I can fix that).

What's your use case exactly ?

Well, I suppose you could either go for bytecode hacks on the function's
code object or have a look at the new AST module, but that's actually
way too black magic for me...
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"

[Lonnie Princehouse]

> What's your use case exactly ?

I'm trying to use a function to implicitly update a dictionary. The
whole point is to avoid the normal dictionary semantics, so kw['x'] = 5
unfortunately won't do.

I think bytecode hacks may be the way to go

[Michael Spencer]

Lonnie Princehouse wrote:

What's your use case exactly ?

I'm trying to use a function to implicitly update a dictionary. The
whole point is to avoid the normal dictionary semantics, so kw['x'] = 5
unfortunately won't do.

I think bytecode hacks may be the way to go

I once messed around with something like that:

#----------- def_hacks.py

"""Silly operations on bytecode. Just for fun"""

from dis import HAVE_ARGUMENT, opmap
from types import CodeType as code

def gendis(co):
"""Yield atomic operations from bytecode"""
coiter = iter(co)
for char in coiter:
op = ord(char)
if op >= HAVE_ARGUMENT:
yield [op, ord(coiter.next()), ord(coiter.next())]
else:
yield [op]

def thunk(func):
"""Function decorator -> code object.
Hacks func.func_code so that it uses a real local dictionary rather than
optimized locals. Returns a code object that can be exec'd in a
context. This amounts to `exec function_body_source in context`, but
does not requre accesses to the source"""

out = []
c= func.func_code
codestring = c.co_code

# These may not be all the necessary hacks!
replace_map = {
opmap["STORE_FAST"]: opmap["STORE_NAME"],
opmap["LOAD_FAST"]: opmap["LOAD_NAME"],
opmap["DELETE_FAST"]: opmap["DELETE_NAME"],
}

names_list = list(c.co_names)

# optimized locals are indexed in co_varnames
# non-locals are indexed in co_names
# so when we switch operations, we have to change the
# index variables too

name_map = dict((ix, names_list.index(name))
for ix, name in enumerate(c.co_varnames)
if name in names_list)

for atom in gendis(codestring):
opcode = atom[0]
if opcode in replace_map:
atom[0] = replace_map[opcode]
varindex = atom[1] + 256 * atom[2]
atom[1:] = reversed(divmod(name_map[varindex], 256))
out.append("".join(chr(byte) for byte in atom))

codestring = "".join(out)
# Make a new code object, using most of the properties of the original
# but with new codestring, no arguments, and with flags adjusted
return code(0, #c.co_argcount
c.co_nlocals, c.co_stacksize, c.co_flags-3,
codestring, c.co_consts, c.co_names, c.co_varnames,
c.co_filename, c.co_name, c.co_firstlineno, c.co_lnotab,
c.co_freevars, c.co_cellvars)
@thunk
def simple_test():
a = 4
b = 2
return c*a+b

Then usage is:

d = {"__builtins__":None, "c":10}
eval(simple_test, d) 42 d {'__builtins__': None, 'a': 4, 'c': 10, 'b': 2}

Use at your peril!

Michael

[bruno at modulix]

Lonnie Princehouse wrote:

What's your use case exactly ?

I'm trying to use a function to implicitly update a dictionary.

I think this was pretty obvious. What I wonder is *why* you want/think
you need to do such a thing -I don't mean there can't be no good reason
to do so, but this has to be a pretty uncommon use case, and some of the
gurus here may point you to (possibly other and possibly better)
solutions if you give a bit more context.

And BTW, please, lauch your python shell and type 'import this', then
read carefully.

The
whole point is to avoid the normal dictionary semantics,

Yes, but *why ?*
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"

[Steve Holden]

bruno at modulix wrote:

Lonnie Princehouse wrote:

What's your use case exactly ?

I'm trying to use a function to implicitly update a dictionary.

I think this was pretty obvious. What I wonder is *why* you want/think
you need to do such a thing -I don't mean there can't be no good reason
to do so, but this has to be a pretty uncommon use case, and some of the
gurus here may point you to (possibly other and possibly better)
solutions if you give a bit more context.

And BTW, please, lauch your python shell and type 'import this', then
read carefully.

The
whole point is to avoid the normal dictionary semantics,

Yes, but *why ?*

Well, one way would be:

def f(d, ...):
...
d.update(locals())

Seems to work, though it does create a self-reference in the dictionary
because the dictionary is passed as an argument.

def f(d, a, b, c=23.0): ... print "a:", a, "b:", b, "c:", c
... d.update(locals())
... myd = {'a': "this is a", 'x': "this is x"}
f(myd, 1, 2, 3) a: 1 b: 2 c: 3 myd {'a': 1, 'c': 3, 'b': 2, 'd': {...}, 'x': 'this is x'}

I can't think of a way to wrap this up as a decorator, though.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd www.holdenweb.com
Love me, love my blog holdenweb.blogspot.com

[Lonnie Princehouse]

Occaisionally, the first two lines of The Zen of Python conflict with
one another.

An API I'm working on involves a custom namespace implementation using
dictionaries, and I want a pretty syntax for initializing the custom
namespaces. The fact that these namespaces are implemented as
dictionaries is an implementation detail, and I don't want the users to
access them directly. I find the "implicit update" syntax to be much
cleaner:

@namespace # indicates function should be executed in namespace
def initialize_namespace():
x = 5

# versus the alternative

__namespace__ = {
'x' : 5,
}
Of course, "x = 5" is a trivial example. The contents of these
namespaces will probably contain on average dozens of items; they are
analagous to a module's global dictionary. Most people probably prefer
to specify a module's globals with regular statements:

x = 5

.... instead of explicit dictionary assignment:

__dict__['x'] = 5

[Lonnie Princehouse]

Very clever!
This looks like exactly what I wanted. Thanks =)

[Bruno Desthuilliers]

Lonnie Princehouse a écrit :

Occaisionally, the first two lines of The Zen of Python conflict with
one another.
"""
Beautiful is better than ugly.
Explicit is better than implicit.
"""

Err... I see no contradiction nor conflict here.
An API I'm working on involves a custom namespace implementation using
dictionaries, and I want a pretty syntax for initializing the custom
namespaces. The fact that these namespaces are implemented as
dictionaries is an implementation detail, and I don't want the users to
access them directly.

Ok. That's effectively not a very common use case. And I agree than it's
far better to abstract implementation details. (And FWIW, I still have
no clean solution - but I'm not a guru).

[Lonnie Princehouse]

> Beautiful is better than ugly.

Explicit is better than implicit.

Err... I see no contradiction nor conflict here.

What to do when explicit is ugly and implicit is beautiful? Aye,
there's the rub. ;-)

[bruno at modulix]

Lonnie Princehouse wrote:

Beautiful is better than ugly.
Explicit is better than implicit.

Err... I see no contradiction nor conflict here.

What to do when explicit is ugly and implicit is beautiful?

Depends on your understanding of explicit/implicit.

Side effects on globals or automatic type conversion à la PHP are
"implicit". Passing args and getting results back or requiring explicit
conversion are "explicit".

Abstracting implementation details is a somewhat different topic IMHO.

May I had :
"""
Simple is better than complex.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
"""

and of course :
"""
Namespaces are one honking great idea -- let's do more of those!
"""

You know, the whole is greater than the sum of the parts and all that
kind of things !-)

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"

[Steve Holden]

Lonnie Princehouse wrote:

Beautiful is better than ugly.
Explicit is better than implicit.

Err... I see no contradiction nor conflict here.

What to do when explicit is ugly and implicit is beautiful? Aye,
there's the rub. ;-)

Realise that sometimes explicitly ugly can be implicitly beautiful, and
you will become enlightened.

could-that-be-the-sound-of-one-hand-clapping-ly y'rs - steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd www.holdenweb.com
Love me, love my blog holdenweb.blogspot.com

[Ziga Seilnacht]

Lonnie Princehouse wrote:

Occaisionally, the first two lines of The Zen of Python conflict with
one another.

An API I'm working on involves a custom namespace implementation using
dictionaries, and I want a pretty syntax for initializing the custom
namespaces. The fact that these namespaces are implemented as
dictionaries is an implementation detail, and I don't want the users to
access them directly. I find the "implicit update" syntax to be much
cleaner:

This can be easier achieved with a custom metaclass:

class MetaNamespace(type): .... def __new__(metaclass, name, bases, dict):
.... try:
.... Namespace
.... except NameError:
.... return type.__new__(metaclass, name, bases, dict)
.... dict.pop('__module__', None)
.... return dict
.... class Namespace(object): .... __metaclass__ = MetaNamespace
....

Now whenever you want to create your dictionary you simply
declare a class that inherits from Namespace:
class MyNamespace(Namespace): .... x = 5
.... y = 'spam'
.... z = 'eggs'
.... print sorted(MyNamespace.items())

[('x', 5), ('y', 'spam'), ('z', 'eggs')]
Ziga

[Alex Martelli]

Lonnie Princehouse <fi**************@gmail.com> wrote:
...

@namespace # indicates function should be executed in namespace
def initialize_namespace():
x = 5

# versus the alternative

__namespace__ = {
'x' : 5,
}

Hm, what about:

ns = namespace(x=5)

and perhaps later

ns.update(y=6)

if you need to add a name to ns? You can start with 'namespace=dict'
(in whatever ubernamespace you'd define your @namespace decorator above)
and change it later in any way you wish -- and it's vastly less
blackmagical, too.
Alex