dynamically generated runtime methods & reflection

[Jay Loden]

Hi all,

First, apologies if anyone gets this twice, but it took me quite a
while to figure out that Python.org is evidently rejecting all mail
from my mail server because I don't have reverse DNS configured.
Anyway:

I'm not even sure how to phrase this question properly or the right
terminology on this so bear with me. What I'm looking to find out is

a) is there a functionality in Python where I can call a method I have
not defined ahead of time, and based on the method name, change the
functionality of the method at runtime?

b) if not, what is the "Pythonic" approach to the problem outlined
below? Any recommendations on how to approach the problem differently
are welcome.

I've googled and read my Python reference pretty extensively and I've
found some hints but nothing that really answered my questions, so
here I am :-) I did figure out that you can overload __getattr__ in
a clas to define a new method at runtime, but I got stuck when I
couldn't figure out how to have a method know what name it was
originally called with. That's the basic question, see below for the
context I'm asking the question in and *why* I want to do the above
:-)

-----

The software product I work on has a socket-based API that can be
accessed directly via telnet & manually typing commands, or by a set
of Perl modules that wrap the socket functionality. I am looking to
write a set of Python modules that match the same functionality.
Basically you can telnet to a port and type something like

item.list "param1=value1", "param2=value2"

And it will return the results (if any) to you in a given format along
with a response code, e.g. 200 OK or 400 ERROR. The Perl modules just
wrap this so that instead of the above, you can do something like
this:

$MySocketServer = new SocketServer(Host=>'127.0.0.'1, Port=>'9999');

if (! $MySocketServer->ListItem(itemName=>$item_name)) {
print "failed to retrieve list";
print "reason: " . $MySocketServer->GetErrorMsg();
exit;
}
The ListItem() method handles the work of communicating across the
socket, waiting for the response, and determine the success/failure
based on the return code. The part where it gets interesting is that
our Perl modules don't actually have a ListItem() method. Instead,
there is a generalized "_api_func()" method that looks at the name of
the method you called (using Autoload I believe, for any Perlheads out
there), and bases the action taken on the name.

In the example above, for instance, it sees that you called
ListItem(), so it transforms that to item.list and makes the
parameters of the method the parameters that are passed to item.list
via the socket server. Then it takes the response back from the socket
to determine the function return code. If there are any results such
as a list of items, they are then able to be accessed through a
reference to a hash of the results, and there is a GetResultRef
accessor function. In essence, our Perl modules are an incredibly
lightweight, simplistic wrapper for the socket-based API, and almost
all of the work is offloaded to the API implementation server-side.

I welcome any suggestions, feedback, sample code, documentation,
whitepapers etc. The only thing I don't want to have to do is go and
define a separate method for every single one of the possible
permutations of socket commands if I can avoid it!

-----

Thanks in advance,

-Jay

[Josiah Carlson]

Jay Loden wrote:

Hi all,

First, apologies if anyone gets this twice, but it took me quite a
while to figure out that Python.org is evidently rejecting all mail
from my mail server because I don't have reverse DNS configured.
Anyway:

I'm not even sure how to phrase this question properly or the right
terminology on this so bear with me. What I'm looking to find out is

a) is there a functionality in Python where I can call a method I have
not defined ahead of time, and based on the method name, change the
functionality of the method at runtime?

Yes. Implement a __getattr__ method on your class (which you mention).

b) if not, what is the "Pythonic" approach to the problem outlined
below? Any recommendations on how to approach the problem differently
are welcome.

I've googled and read my Python reference pretty extensively and I've
found some hints but nothing that really answered my questions, so
here I am :-) I did figure out that you can overload __getattr__ in
a clas to define a new method at runtime, but I got stuck when I
couldn't figure out how to have a method know what name it was
originally called with. That's the basic question, see below for the
context I'm asking the question in and *why* I want to do the above
:-)

Ahh, so you want to pass the method name to the method that you are
returning to be called. No problem.

>>import functools

class foo:

.... def __getattr__(self, name):
.... return functools.partial(self.ActualMethod, name)
....
.... def ActualMethod(self, name, *args, **kwargs):
.... #handle *args and **kwargs based on name!
.... print name, args, kwargs
....

>>foo().bar('hello', world=1)

bar ('hello',) {'world': 1}

>>>

- Josiah

[Jay Loden]

Josiah Carlson wrote:

>
Ahh, so you want to pass the method name to the method that you are
returning to be called. No problem.

>>import functools
>>>
>>class foo:

... def __getattr__(self, name):
... return functools.partial(self.ActualMethod, name)
...
... def ActualMethod(self, name, *args, **kwargs):
... #handle *args and **kwargs based on name!
... print name, args, kwargs
...

>>foo().bar('hello', world=1)

bar ('hello',) {'world': 1}

>>>

Thanks, this is exactly what I was looking for! For some reason functools didn't even show up at all during Google searches...must have just had the wrong search terms.

-Jay

[Josiah Carlson]

Jay Loden wrote:

Josiah Carlson wrote:

>Ahh, so you want to pass the method name to the method that you are
returning to be called. No problem.

> >>import functools

class foo:

... def __getattr__(self, name):
... return functools.partial(self.ActualMethod, name)
...
... def ActualMethod(self, name, *args, **kwargs):
... #handle *args and **kwargs based on name!
... print name, args, kwargs
...

> >>foo().bar('hello', world=1)

bar ('hello',) {'world': 1}

> >>>

Thanks, this is exactly what I was looking for! For some reason functools didn't even show up at all during Google searches...must have just had the wrong search terms.

Well, the particular operation is typically called 'currying a
function', and unless you know what to look for, it isn't very easy to
make happen.

On the other hand, it is also relatively easy to implement by hand if
necessary.

- Josiah

[Josiah Carlson]

Josiah Carlson wrote:

Well, the particular operation is typically called 'currying a
function', and unless you know what to look for, it isn't very easy to
make happen.

Replace "make happen" to "discover in the standard library".

- Josiah

[Bruno Desthuilliers]

Josiah Carlson a écrit :
(snip)

Well, the particular operation is typically called 'currying a
function',

<pedantic>
it's not 'currying' but 'partial application'.

Currying is somehow the reverse of partial : it's a way of building a
multiple-args function from single-args functions.
</pedantic>

[Alex Martelli]

Bruno Desthuilliers <br********************@wtf.websiteburo.oops.com >
wrote:

Josiah Carlson a écrit :
(snip)

Well, the particular operation is typically called 'currying a
function',

<pedantic>
it's not 'currying' but 'partial application'.

Currying is somehow the reverse of partial : it's a way of building a
multiple-args function from single-args functions.
</pedantic>

Wikipedia says "currying or Schönfinkelisation[1] is the technique of
transforming a function that takes multiple arguments into a function
that takes a single argument" -- and FWIW I agree with Wikipedia in this
case; the reverse (going from single-arg to multiple-args) would be
"uncurrying", though I don't think I've ever used that term myself.

functools.partial's name may be more precise (because it can, e.g., go
from a function taking 3 arguments to one taking 2 -- not just from N
down to 1) but your 'pedantic' remark seems pedantically wrong:-).
Alex

[Bruno Desthuilliers]

Alex Martelli a écrit :

Bruno Desthuilliers <br********************@wtf.websiteburo.oops.com >
wrote:

>Josiah Carlson a écrit :
(snip)

>>Well, the particular operation is typically called 'currying a
function',

<pedantic>
it's not 'currying' but 'partial application'.

Currying is somehow the reverse of partial : it's a way of building a
multiple-args function from single-args functions.
</pedantic>

Wikipedia says

"currying or Schönfinkelisation[1] "

cf below...

"is the technique of
transforming a function that takes multiple arguments into a function
that takes a single argument"

The definition commonly agreed upon (in the FP world at least) is that
currying is the process that "build" ("emulate", whatever...)
multiple-args functions in a context that only supports single-arg
functions (ie: ML, Haskell), so that (using Python syntax):

f(x, y, z)

would really be in fact (and under the hood):

f(x)(y)(z)

where f(x) returns a function closing over(x) and taking y, itself
returning a function closing over x and y and taking z...

Talking about this:
"""
currying (...) reduces multiple-argument
functions to single-argument functions only (Schoenfinkel,
1924)
"""
http://srfi.schemers.org/srfi-26/mai.../msg00015.html
So while *very closely* related to partial application, it's not exactly
the same thing.

FWIW, you can also have a look here:
http://www.python.org/dev/peps/pep-0309/#motivation

-- and FWIW I agree with Wikipedia in this
case;

I don't - and I'm not the only one:

http://lambda-the-ultimate.org/node/2266
"""
I had mistakenly learned that curry was a form of generalized partial
application from the paper : Function Currying in Scheme by Jeffrey A.
Meunier
and the Wikipedia entry (I should have known better), however I was
mildly reprimanded for making this novice mistake in a recent paper
submission to ICFP
"""

the reverse (going from single-arg to multiple-args)

Note that I didn't say "going from", but "building". The context is a
functional language where there's *no* such thing as "multiple args"
functions.

Re-reading, I can agree that my choice of words may have been a bit
poor, specially wrt/ the word "reverse". What I meant here was that
partial application is used in the context of multiple-args function to
'build' a function taking n-x arguments from a function taking n
arguments, while currying is used in the context of single-arg functions
to "emulate" multiple-args functions.

would be
"uncurrying", though I don't think I've ever used that term myself.

functools.partial's name may be more precise (because it can, e.g., go
from a function taking 3 arguments to one taking 2 -- not just from N
down to 1)

but your 'pedantic' remark seems pedantically wrong:-).

I certainly won't pretend knowing more about CS that you do, but given
the definition of currying in pep-0309 - which exactly matches the
definition I first learned when toying with Haskell -, I maintain my
pedantic remark until proven wrong - which BTW should not be overly
difficult if it happened to be the case !-)

As a last word, the original version of pep-0309 was named "curry", and
has been corrected since:

"""
It isn't function currying, but partial application. Hence the name is
now proposed to be partial().
"""
http://www.python.org/dev/peps/pep-0...and-python-dev