Functions and code objects

[Fuzzyman]

Hello all,

I'm trying to extract the code object from a function, and exec it
without explicitly passing parameters.

The code object 'knows' it expects to receive paramaters. It's
'arg_count' attribute is readonly.

How can I set the arg_count to 0, or pass parameters to the code object
when I exec it ?

>>def f(x):

.... print x
....

>>c = f.func_code
type(c)

<type 'code'>

>>exec c

Traceback (most recent call last):
File "<input>", line 1, in ?
TypeError: f() takes exactly 1 argument (0 given)

>>c.co_argcount

1

>>c.co_argcount = 0

Traceback (most recent call last):
File "<input>", line 1, in ?
TypeError: readonly attribute

>>>

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

[Fuzzyman]

Fuzzyman wrote:

Hello all,

I'm trying to extract the code object from a function, and exec it
without explicitly passing parameters.

The code object 'knows' it expects to receive paramaters. It's
'arg_count' attribute is readonly.

How can I set the arg_count to 0, or pass parameters to the code object
when I exec it ?

Ok, so now I'm getting somewhere, without really knowing what I'm
doing. Using the CodeType I can create a new code object with identical
attributes, except an 'argcount' of 0.

It doesn't quite work, so I probably need to set some of the attributes
*differently*.

The code I use is :

>>def f(x):

... print x
...

>>c = f.func_code
CodeType = type(c)
a = CodeType(0, c.co_nlocals, c.co_stacksize, c.co_flags, c.co_code,

... c.co_consts, c.co_names, c.co_varnames, c.co_filename,
c.co_name,
... c.co_firstlineno, c.co_lnotab, c.co_freevars, c.co_cellvars)

>>a

<code object f at 01C707A0, file "<input>", line 1>

>>exec a

Traceback (most recent call last):
File "<input>", line 1, in ?
File "<input>", line 2, in f
UnboundLocalError: local variable 'x' referenced before assignment

(the first argument, 0, becomes the 'arg_count' attribute)

So the code object is still creating a local scope, and knows that x is
a local variable. ('co_nlocals' and 'co_varnames' ?).

I'd like to construct the code object so that it takes the parameters
from the enclosing scope (the context I pass into exec I guess),
without clobbering any local variables that may be defined in the code
object.

Anyone got any clues ?
The attributes mean (from http://pyref.infogami.com/type-code ) :

* co_name gives the function name
* co_argcount is the number of positional arguments (including
arguments with default values)
* co_nlocals is the number of local variables used by the function
(including arguments)
* co_varnames is a tuple containing the names of the local
variables (starting with the argument names)
* co_cellvars is a tuple containing the names of local variables
that are referenced by nested functions
* co_freevars is a tuple containing the names of free variables
* co_code is a string representing the sequence of bytecode
instructions
* co_consts is a tuple containing the literals used by the bytecode
* co_names is a tuple containing the names used by the bytecode
* co_filename is the filename from which the code was compiled
* co_firstlineno is the first line number of the function
* co_lnotab is a string encoding the mapping from byte code offsets
to line numbers (for details see the source code of the interpreter)
* co_stacksize is the required stack size (including local
variables)
* co_flags is an integer encoding a number of flags for the
interpreter.

In case anyone wonders, this is purely an experiment in creating
'annoymous code blocks'. :-)

What is a 'free variable' ?

'cell_vars' also looks interesting, but probably better not to mess
with it :-) (I guess it is only relevant for functions defined in the
code block)

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

[Fuzzyman]

Fuzzyman wrote:

Fuzzyman wrote:

Hello all,

I'm trying to extract the code object from a function, and exec it
without explicitly passing parameters.

The code object 'knows' it expects to receive paramaters. It's
'arg_count' attribute is readonly.

How can I set the arg_count to 0, or pass parameters to the code object
when I exec it ?

Ok, so now I'm getting somewhere, without really knowing what I'm
doing. Using the CodeType I can create a new code object with identical
attributes, except an 'argcount' of 0.

It doesn't quite work, so I probably need to set some of the attributes
*differently*.

The code I use is :

>def f(x):

... print x
...

>c = f.func_code
CodeType = type(c)
a = CodeType(0, c.co_nlocals, c.co_stacksize, c.co_flags, c.co_code,

... c.co_consts, c.co_names, c.co_varnames, c.co_filename,
c.co_name,
... c.co_firstlineno, c.co_lnotab, c.co_freevars, c.co_cellvars)

>a

<code object f at 01C707A0, file "<input>", line 1>

>exec a

Traceback (most recent call last):
File "<input>", line 1, in ?
File "<input>", line 2, in f
UnboundLocalError: local variable 'x' referenced before assignment

(the first argument, 0, becomes the 'arg_count' attribute)

So the code object is still creating a local scope, and knows that x is
a local variable. ('co_nlocals' and 'co_varnames' ?).

I'd like to construct the code object so that it takes the parameters
from the enclosing scope (the context I pass into exec I guess),
without clobbering any local variables that may be defined in the code
object.

Anyone got any clues ?

*Damn* I've extracted the code object and told it that it has no
arguments. Executing the code object results in the function object !

The code object is obviously the code object for the function
definition. *sigh*

I was hoping I could get to the code object for the *body* of the
function. Looks like that won't be possible without dis-assembling the
bytecode or other tricks even more hackish than what I've already done.

For the record, the code I was using was :

x = 3
def f(x):
print x

CodeType = type(f.func_code)

def convert_function(f):
code = f.func_code
nlocals = max(code.co_nlocals - code.co_argcount, 0)
newCode = CodeType(0, nlocals, code.co_stacksize, code.co_flags,
code.co_code, code.co_consts, code.co_names,
code.co_varnames, code.co_filename,
code.co_name,
code.co_firstlineno, code.co_lnotab,
code.co_freevars,
code.co_cellvars)
return newCode

print convert_function(f)
exec convert_function(f)

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

[Simon Forman]

Fuzzyman wrote:

Fuzzyman wrote:

Fuzzyman wrote:

Hello all,
>
I'm trying to extract the code object from a function, and exec it
without explicitly passing parameters.
>
The code object 'knows' it expects to receive paramaters. It's
'arg_count' attribute is readonly.
>
How can I set the arg_count to 0, or pass parameters to the code object
when I exec it ?
>

Ok, so now I'm getting somewhere, without really knowing what I'm
doing. Using the CodeType I can create a new code object with identical
attributes, except an 'argcount' of 0.

It doesn't quite work, so I probably need to set some of the attributes
*differently*.

The code I use is :

>>def f(x):

... print x
...

>>c = f.func_code
>>CodeType = type(c)
>>a = CodeType(0, c.co_nlocals, c.co_stacksize, c.co_flags, c.co_code,

... c.co_consts, c.co_names, c.co_varnames, c.co_filename,
c.co_name,
... c.co_firstlineno, c.co_lnotab, c.co_freevars, c.co_cellvars)

>>a

<code object f at 01C707A0, file "<input>", line 1>

>>exec a

Traceback (most recent call last):
File "<input>", line 1, in ?
File "<input>", line 2, in f
UnboundLocalError: local variable 'x' referenced before assignment

(the first argument, 0, becomes the 'arg_count' attribute)

So the code object is still creating a local scope, and knows that x is
a local variable. ('co_nlocals' and 'co_varnames' ?).

I'd like to construct the code object so that it takes the parameters
from the enclosing scope (the context I pass into exec I guess),
without clobbering any local variables that may be defined in the code
object.

Anyone got any clues ?

*Damn* I've extracted the code object and told it that it has no
arguments. Executing the code object results in the function object !

The code object is obviously the code object for the function
definition. *sigh*

I was hoping I could get to the code object for the *body* of the
function. Looks like that won't be possible without dis-assembling the
bytecode or other tricks even more hackish than what I've already done.

For the record, the code I was using was :

x = 3
def f(x):
print x

CodeType = type(f.func_code)

def convert_function(f):
code = f.func_code
nlocals = max(code.co_nlocals - code.co_argcount, 0)
newCode = CodeType(0, nlocals, code.co_stacksize, code.co_flags,
code.co_code, code.co_consts, code.co_names,
code.co_varnames, code.co_filename,
code.co_name,
code.co_firstlineno, code.co_lnotab,
code.co_freevars,
code.co_cellvars)
return newCode

print convert_function(f)
exec convert_function(f)

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

Out of curiosity, why are you doing this?

Peace,
~Simon

[Duncan Booth]

Fuzzyman wrote:

>I'd like to construct the code object so that it takes the parameters
from the enclosing scope (the context I pass into exec I guess),
without clobbering any local variables that may be defined in the code
object.

Anyone got any clues ?

Does this do what you wanted? Instead of messing about with the code object
just work out which values from the namespace the function actually
expects.

>>def callfromnamespace(fn, namespace):

names = fn.func_code.co_varnames[:fn.func_code.co_argcount]
fn(**dict((name, namespace[name])
for name in names if name in namespace))

>>def f(x, y=99):

z = 2
print x, y, z

>>x = 42
callfromnamespace(f, globals())

42 99 2

>>y = 37
callfromnamespace(f, globals())

42 37 2

>>def testme():

x = 3
callfromnamespace(f, vars())
y = 9
callfromnamespace(f, vars())

>>testme()

3 99 2
3 9 2

[Fuzzyman]

Simon Forman wrote:

Fuzzyman wrote:

[snip..]

I was hoping I could get to the code object for the *body* of the
function. Looks like that won't be possible without dis-assembling the
bytecode or other tricks even more hackish than what I've already done.

For the record, the code I was using was :

x = 3
def f(x):
print x

CodeType = type(f.func_code)

def convert_function(f):
code = f.func_code
nlocals = max(code.co_nlocals - code.co_argcount, 0)
newCode = CodeType(0, nlocals, code.co_stacksize, code.co_flags,
code.co_code, code.co_consts, code.co_names,
code.co_varnames, code.co_filename,
code.co_name,
code.co_firstlineno, code.co_lnotab,
code.co_freevars,
code.co_cellvars)
return newCode

print convert_function(f)
exec convert_function(f)

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

Out of curiosity, why are you doing this?

In Ruby anonymous code blocks (which take parameters and can presumably
return values) are executed within the scope in which they are used,
rather than the scope that they are defined.

I wanted to see how close to that I could get in Python. Obviously code
could be compiled from a string, but I thought extracting it from the
body of a function was closer.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

Peace,
~Simon

[Fuzzyman]

Duncan Booth wrote:

Fuzzyman wrote:

[snip..]

>
Does this do what you wanted? Instead of messing about with the code object
just work out which values from the namespace the function actually
expects.

>def callfromnamespace(fn, namespace):

names = fn.func_code.co_varnames[:fn.func_code.co_argcount]
fn(**dict((name, namespace[name])
for name in names if name in namespace))

>def f(x, y=99):

z = 2
print x, y, z

>x = 42
callfromnamespace(f, globals())

42 99 2

>y = 37
callfromnamespace(f, globals())

42 37 2

>def testme():

x = 3
callfromnamespace(f, vars())
y = 9
callfromnamespace(f, vars())

>testme()

3 99 2
3 9 2

Hmmm... it may do, thanks. :-)

I'll play around, and this may become a blog entry.

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml