decorators as generalized pre-binding hooks

[Bengt Richter]

;-)
We have

@deco
def foo(): pass
as sugar (unless there's an uncaught exception in the decorator) for
def foo(): pass
foo = deco(foo)

The binding of a class name is similar, and class decorators would seem natural, i.e.,

@cdeco
class Foo: pass
for
class Foo: pass
Foo = cdeco(Foo)

What is happening is we are intercepting the binding of some object
and letting the decorator do something to the object before the binding occurs.

So why not

@deco
foo = lambda:pass
equivalent to
foo = deco(lambda:pass)

and from there,
@deco
<left-hand-side> = <right-hand-side>
being equivalent to
<left-hand-side> = deco(<right-hand-side>)

e.g.,
@range_check(1,5)
a = 42
for
a = range_check(1,5)(42)

or
@default_value(42)
b = c.e['f']('g')
for
b = default_value(42)(c.e['f']('g'))

Hm, binding-intercept-decoration could be sugar for catching exceptions too,
and passing them to the decorator, e.g., the above could be sugar for

try:
b = default_value(42)(c.e['f']('g'))
except Exception, e:
b = default_value(__exception__=e) # so decorator can check
# and either return a value or just re-raise with raise [Note 1]

This might be useful for plain old function decorators too, if you wanted the decorator
to define the policy for substituting something if e.g. a default argument evaluation
throws and exception. Thus

@deco
def foo(x=a/b): pass # e.g., what if b==0?
as
try:
def foo(x=a/b): pass # e.g., what if b==0?
foo = deco(foo)
except Exception, e:
if not deco.func_code.co_flags&0x08: raise #avoid mysterious unexpected keyword TypeError
foo = deco(__exception__=e)

[Note 1:]
Interestingly raise doesn't seem to have to be in the same frame or down-stack, so a decorator
called as above could re-raise:

def deco(**kw): ... print kw
... raise
... try: 1/0

... except Exception, e: deco(__exception__=e)
...
{'__exception__': <exceptions.ZeroDivisionError instance at 0x02EF190C>}
Traceback (most recent call last):
File "<stdin>", line 2, in ?
File "<stdin>", line 1, in ?
ZeroDivisionError: integer division or modulo by zero
orthogonal-musing-ly ;-)

Regards,
Bengt Richter

[Ron Adam]

Bengt Richter wrote:

;-)
We have
Have we?

Looks like not a lot of interested takers so far.

But I'll bite. ;-)
<clip intro>
So why not

@deco
foo = lambda:pass
equivalent to
foo = deco(lambda:pass)

and from there,
@deco
<left-hand-side> = <right-hand-side>
being equivalent to
<left-hand-side> = deco(<right-hand-side>)

e.g.,
@range_check(1,5)
a = 42
for
a = range_check(1,5)(42)

or
@default_value(42)
b = c.e['f']('g')
for
b = default_value(42)(c.e['f']('g'))
So far they are fairly equivalent. So there's not really any advantage
over the equivalent inline function. But I think I see what you are
going towards. Decorators currently must be used when a function is
defined. This option attempts to makes them more dynamic so that they
can be used where and when they are needed.

How about if you make it optional too?

if keep_log:
@log_of_interesting_values
b = get_value(c,d):

Or...

@@keeplog log_of_interesting_values # if keeplog decorate.
b = get_value(c,d)

Just a thought.
Hm, binding-intercept-decoration could be sugar for catching exceptions too,
and passing them to the decorator, e.g., the above could be sugar for

try:
b = default_value(42)(c.e['f']('g'))
except Exception, e:
b = default_value(__exception__=e) # so decorator can check
# and either return a value or just re-raise with raise [Note 1]
I'm not sure I follow this one.. Well I do a little. Looks like it might
be going the direction of with statements, but only applied to a single
expression instead of a block or suite.

This might be useful for plain old function decorators too, if you wanted the decorator
to define the policy for substituting something if e.g. a default argument evaluation
throws and exception. Thus

@deco
def foo(x=a/b): pass # e.g., what if b==0?
as
try:
def foo(x=a/b): pass # e.g., what if b==0?
foo = deco(foo)
except Exception, e:
if not deco.func_code.co_flags&0x08: raise #avoid mysterious unexpected keyword TypeError
foo = deco(__exception__=e)
Wouldn't this one work now?
[Note 1:]
Interestingly raise doesn't seem to have to be in the same frame or down-stack, so a decorator
called as above could re-raise:

>>> def deco(**kw): ... print kw
... raise
... >>> try: 1/0

... except Exception, e: deco(__exception__=e)
...
{'__exception__': <exceptions.ZeroDivisionError instance at 0x02EF190C>}
Traceback (most recent call last):
File "<stdin>", line 2, in ?
File "<stdin>", line 1, in ?
ZeroDivisionError: integer division or modulo by zero

Interestin.

When it comes to decorators, and now the with statements, I can't help
but feel that there's some sort of underlying concept that would work
better. It has to do with generalizing flow control in a dynamic way
relative to an associated block.

One thought is to be able to use a place holder in an expression list to
tell a statement when to do the following block of code.

I'll use 'do' here... since it's currently unused and use @@@ as the
place holder.

(These are *NOT* thought out that far yet, I know... just trying to
show a direction that might be possible.)
do f=open(filename); @@@; f.close(): # do the block where @@@ is.
for line in f:
print line,
print
And an alternate version similar to a "with" statement.

try f=open(filename); @@@; finally f.close():
for line if f:
print line,
print

Maybe the exception could be held until after the try line is complete?
The place holder idea might be useful for decorating as well. But I'm
not sure how the function name and arguemtns would get passed to it.

do deco(@@@):
def foo():
pass

or maybe it would need to be...

do f=@@@, deco(f):
def foo()
pass

As I said, it still needs some thinking.. ;-)
Maybe leaving off the colon would use the following line without
indenting as per your example?

do deco(@@@)
b = 42

It doesn't quite work this way I think. Possibly having a couple of
different types of place holder symbols which alter the behavior might work?

do deco($$$) # $$$ intercept name binding operation?
b = 42
Well, it may need to be a bit (or a lot) of changing. But the idea of
controlling flow with a place holder symbol might be a way to generalize
some of the concepts that have been floating around into one tool.

I like the place holders because I think they make the code much more
explicit and they are more flexible because you can put them where you
need them.

orthogonal-musing-ly ;-)
"Orthogonal is an unusual computer language in which your program flow
can go sideways. In actuality in can go in just about any direction you
could want."

http://www.muppetlabs.com/~breadbox/orth/
;-)

Cheers,
Ron

Regards,
Bengt Richter

[Terry Reedy]

"Bengt Richter" <bo**@oz.net> wrote in message
news:42****************@news.oz.net...

The binding of a class name is similar, and class decorators
would seem natural, i.e.,

@cdeco
class Foo: pass
for
class Foo: pass
Foo = cdeco(Foo)

This possibility was discussed on the py-dev list about a year or so ago.
The twice monthly summaries should include this topic. As I remember, the
reason for limiting to functions included:

1. classes have metaclasses, functions don't have metafunctions. No one
gave an example for classes not handled at least as well with a metaclass.

2. certain applications require long-function_names_like_this, for which
triple typing is substantially annoying and error-prone.

Terry J. Reedy

[Bengt Richter]

On Sun, 10 Jul 2005 05:35:01 GMT, Ron Adam <rr*@ronadam.com> wrote:

Bengt Richter wrote:

;-)
We have
Have we?

Looks like not a lot of interested takers so far.

But I'll bite. ;-)
<clip intro>

So why not

@deco
foo = lambda:pass
equivalent to
foo = deco(lambda:pass)

and from there,
@deco
<left-hand-side> = <right-hand-side>
being equivalent to
<left-hand-side> = deco(<right-hand-side>)

e.g.,
@range_check(1,5)
a = 42
for
a = range_check(1,5)(42)

or
@default_value(42)
b = c.e['f']('g')
for
b = default_value(42)(c.e['f']('g'))

So far they are fairly equivalent. So there's not really any advantage
over the equivalent inline function. But I think I see what you are
going towards. Decorators currently must be used when a function is
defined. This option attempts to makes them more dynamic so that they
can be used where and when they are needed.

IMO part of the decorator benefit is clearer code, and also IMO the @range_check
and @default_value decorators succeed in that. The code generated would presumably
be the same, unless the exception capture discussed further down were implemented.

How about if you make it optional too?

if keep_log:
@log_of_interesting_values
b = get_value(c,d):
I'd probably pass the condition to the decorator for a less ragged look
@log_of_interesting_values(keep_log)
b = get_value(c,d)Or...

@@keeplog log_of_interesting_values # if keeplog decorate.
b = get_value(c,d)

Just a thought. Yes, but that is too easy to do another way. Plus I want to reserve '@@' for an AST-time
decoration idea I have ;-)

Hm, binding-intercept-decoration could be sugar for catching exceptions too,
and passing them to the decorator, e.g., the above could be sugar for

try:
b = default_value(42)(c.e['f']('g'))
except Exception, e:
b = default_value(__exception__=e) # so decorator can check
# and either return a value or just re-raise with raise [Note 1]
I'm not sure I follow this one.. Well I do a little. Looks like it might
be going the direction of with statements, but only applied to a single
expression instead of a block or suite.

Yes, it's just expanding on the idea of intercepting the value of the object being bound even
if the creation of that object resulted in an exception, and allowing the decorator to handle
it either way.

This might be useful for plain old function decorators too, if you wanted the decorator
to define the policy for substituting something if e.g. a default argument evaluation
throws and exception. Thus

@deco
def foo(x=a/b): pass # e.g., what if b==0?
as
try:
def foo(x=a/b): pass # e.g., what if b==0?
foo = deco(foo)
except Exception, e:
if not deco.func_code.co_flags&0x08: raise #avoid mysterious unexpected keyword TypeError
foo = deco(__exception__=e)
Wouldn't this one work now?

I think all the second-versions would work now, but a plain decorator won't get control if the
def fails, so the def name won't get bound:

def deco(f): print f; return f # noop ... @deco ... def foo(x=1/2): pass
...
<function foo at 0x02EF409C> foo <function foo at 0x02EF409C> del foo
foo Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'foo' is not defined @deco ... def foo(x=1/0): pass
...
Traceback (most recent call last):
File "<stdin>", line 1, in ?
ZeroDivisionError: integer division or modulo by zero foo

Traceback (most recent call last):
File "<stdin>", line 1, in ?
NameError: name 'foo' is not defined

[Note 1:]
Interestingly raise doesn't seem to have to be in the same frame or down-stack, so a decorator
called as above could re-raise:

>>> def deco(**kw): ... print kw
... raise
...

>>> try: 1/0

... except Exception, e: deco(__exception__=e)
...
{'__exception__': <exceptions.ZeroDivisionError instance at 0x02EF190C>}
Traceback (most recent call last):
File "<stdin>", line 2, in ?
File "<stdin>", line 1, in ?
ZeroDivisionError: integer division or modulo by zero

Interestin.

Yes, it would seem to give one the option of catching an exception and
doing some arbitarily (so long as not new-exception-raising) recursive
or deep calls and bailing out from some leaf point with a raise.

When it comes to decorators, and now the with statements, I can't help
but feel that there's some sort of underlying concept that would work
better. It has to do with generalizing flow control in a dynamic way
relative to an associated block.

One thought is to be able to use a place holder in an expression list to
tell a statement when to do the following block of code. it depends on scope and control aspects of what you mean by "block".
But I doubt if we have much chance of introducing something is one more
bf in the storm of "with" ideas that have already been discussed.
I'll use 'do' here... since it's currently unused and use @@@ as the
place holder.

(These are *NOT* thought out that far yet, I know... just trying to
show a direction that might be possible.)
They strike me as a kind of macro idea where the only substitution argument
is the block suite that follows, which IMO is a severe limitation on both
the macro idea and the use of blocks ;-)

[...]
I like the place holders because I think they make the code much more
explicit and they are more flexible because you can put them where you
need them. Yes, but if you want to go that way, I'd want to have named place holders
and be able to refer to arbitrary things that make sense in the context.

orthogonal-musing-ly ;-)

"Orthogonal is an unusual computer language in which your program flow
can go sideways. In actuality in can go in just about any direction you
could want."

http://www.muppetlabs.com/~breadbox/orth/

Interesting. And an implementation from our own Jeff Epler?
Regards,
Bengt Richter

[George Sakkis]

"Terry Reedy" <tj*****@udel.edu> wrote:

"Bengt Richter" <bo**@oz.net> wrote in message
news:42****************@news.oz.net...

The binding of a class name is similar, and class decorators
would seem natural, i.e.,

@cdeco
class Foo: pass
for
class Foo: pass
Foo = cdeco(Foo)
This possibility was discussed on the py-dev list about a year or so ago.
The twice monthly summaries should include this topic. As I remember, the
reason for limiting to functions included:

1. classes have metaclasses, functions don't have metafunctions. No one
gave an example for classes not handled at least as well with a metaclass.

Would something like the following count ?

@abstractclass
class AbstractFrame(object):

@abstractclass
@innerclass
class AbstractPanel(object):
pass

For one thing, it's more readable than the respective __metaclass__
declarations. Moreover, stacking two or more decorators is
syntactically straightforward, while for metaclasses you have to write
boilerplate code for making a subclass of the components, e.g.:
class AbstractInnerClass(AbstractClass, InnerClass): pass
Fortunately metaclasses are not that commonly used to cause
combinatorial explosion of such boilerplate classes.
2. certain applications require long-function_names_like_this, for which
triple typing is substantially annoying and error-prone.

I'm not sure what you mean here; where is the 'triple typing' ? And how
is this an argument against class decorators ?

George

[Terry Reedy]

"George Sakkis" <gs*****@rutgers.edu> wrote in message
news:11**********************@g43g2000cwa.googlegr oups.com...

"Terry Reedy" <tj*****@udel.edu> wrote:

This possibility was discussed on the py-dev list about a year or so
ago.
The twice monthly summaries should include this topic. As I remember,
the
reason for limiting to functions included:

1. classes have metaclasses, functions don't have metafunctions. No one
gave an example for classes not handled at least as well with a
metaclass.
Would something like the following count ?

[snip]
Not qualified to comment.

2. certain applications require long-function_names_like_this, for which
triple typing is substantially annoying and error-prone.

I'm not sure what you mean here; where is the 'triple typing' ?

def long-function_names_like_this(arg1, b, xx , sklfjsl, uuuu):
'body of fuction here'
pass
long-function_names_like_this = \
some_decorator(long-function_names_like_this)

And for the example I am thinking of (integrating Python with Objective-C,
I believe), the above name is apparently not an exaggeration.
And how is this an argument against class decorators ?

It is an argument for function decos that does not apply to classes. The
does not seem to be a similar need for long, convoluted, class names. (And
there is the metaclass option for classes that there is not for functions.)
So, in relative terms, it is an argument that function decos are relatively
more needed than class decos. Given that the decision to add them all was
a fairly closely balanced one, relative differences can tip the balance one
way one time and the other way the other.

Guido did not permanently rule out class decos, that I know of, but he
noted that it would be easier to add them later if really needed than to
remove them later is not really needed.

Terry J. Reedy

[Ron Adam]

Bengt Richter wrote:

On Sun, 10 Jul 2005 05:35:01 GMT, Ron Adam <rr*@ronadam.com> wrote:

So far they are fairly equivalent. So there's not really any advantage
over the equivalent inline function. But I think I see what you are
going towards. Decorators currently must be used when a function is
defined. This option attempts to makes them more dynamic so that they
can be used where and when they are needed.

IMO part of the decorator benefit is clearer code, and also IMO the
@range_check and @default_value decorators succeed in that. The code
generated would presumably be the same, unless the exception capture
discussed further down were implemented.

If you take the decorator at face value, it's clear. (In a sort of
because I said so way.) But if you look in the decorator, it may be
quite unclear. Ie.. it sort of sweeps the dirt under the rug. (IMO)
The thing is, defining a decorator can be fairly complex compared to a
regular function depending on what one is trying to do.

How about if you make it optional too?

@@keeplog log_of_interesting_values # if keeplog decorate.
b = get_value(c,d)

Just a thought.

Yes, but that is too easy to do another way. Plus I want to reserve
'@@' for an AST-time decoration idea I have ;-)

The @@ could be whatever, but a single @ could probably be used just as
well.

How about any line that begins with an @ is preparsed as sugar. And
then create a simple sugar language to go along with it?

But that would be compile time macros wouldn't it. ;-)

When it comes to decorators, and now the with statements, I can't help
but feel that there's some sort of underlying concept that would work
better. It has to do with generalizing flow control in a dynamic way
relative to an associated block.

One thought is to be able to use a place holder in an expression list to
tell a statement when to do the following block of code.

it depends on scope and control aspects of what you mean by "block".

By block I meant the indented following suite. No special scope rules
that don't already currently exist in any 'if', 'while', or 'for' suite.

But I doubt if we have much chance of introducing something is one
more bf in the storm of "with" ideas that have already been
discussed.
I'd like to think until 2.5 is released that there's still a chance that
something better could come along. But it would have to be pretty darn
good I expect.

They strike me as a kind of macro idea where the only substitution argument
is the block suite that follows, which IMO is a severe limitation on both
the macro idea and the use of blocks ;-)
I'm not sure it's macro or not. Maybe it's a flow control parser
statement?

Does that sound any better than macro? ;-)

I like the place holders because I think they make the code much more
explicit and they are more flexible because you can put them where you
need them.

Yes, but if you want to go that way, I'd want to have named place holders
and be able to refer to arbitrary things that make sense in the context.

From what I've seen so far, there's a lot of resistance to real run
time macro's. So I don't expect them any time soon.

The mechanism I suggested doesn't store code or name it. So it's not a
macro, it's closer to a while that conditionally runs the body, but in
this case the condition is when instead of if. It's a different concept
that I think can compliment the language without being too complex.
Named macros make it even more useful.

Here I used 'this' as the keyword to indicate when the suite is to be
done. So it's a do-this-suite statement.
do f = opening(filename); try this; finally f.close():
suite
Now using "Sugar" language! ;-)

# Create sugar
@with_opened = "opening(%s); try this; finally f.close()"
do f = $with_opened%('filename'): # $ indicates sugar
suite

I used Pythons % operator as it already exists and works fine in this
situation. Easy to implement as well.

Hmm.. not sure how to apply this to a decorator. Lets see... Working it
out...

# function to use
def check_range(x):
if x in range(10,25):
return
raise RangeError # or what ever is suitable

# Make the decorator with sugar
@checkrange = "%s %s check_range(%s)"

Parses on spaces as default?

$checkrange% # Use the following line here
x = 24 # since nothing given after the %
Which will results in...

x = check_range(24)
There should be a way to specify an additional argument I think.

The exact rules would need to be worked out. It also might be a good
way to test sugar ideas before they become part of the language.

orthogonal-musing-ly ;-)

"Orthogonal is an unusual computer language in which your program flow
can go sideways. In actuality in can go in just about any direction you
could want."

http://www.muppetlabs.com/~breadbox/orth/

Interesting. And an implementation from our own Jeff Epler?

I didn't see that. LOL

Cheers,
Ron

Regards,
Bengt Richter

[Kay Schluehr]

George Sakkis schrieb:

1. classes have metaclasses, functions don't have metafunctions. No one
gave an example for classes not handled at least as well with a metaclass.

Would something like the following count ?

@abstractclass
class AbstractFrame(object):

@abstractclass
@innerclass
class AbstractPanel(object):
pass

For one thing, it's more readable than the respective __metaclass__
declarations. Moreover, stacking two or more decorators is
syntactically straightforward, while for metaclasses you have to write
boilerplate code for making a subclass of the components, e.g.:
class AbstractInnerClass(AbstractClass, InnerClass): pass
Fortunately metaclasses are not that commonly used to cause
combinatorial explosion of such boilerplate classes.

I think it would be a good idea to pronounce the similarity between
function decorators and metaclasses. Metaclasses were once introduced
as an arcane art of fuzzy bearded hackers or supersmart 'enterprise
architects' that plan at least products of Zope size but not as a tool
for the simple programmer on the street. But maybe they should be and
there should also be librarys of them representing orthogonal
customizations ready for plug in.

Since metaclasses follow an "adapt", "customize" or "decorate"
semantics it is a good idea not to use inheritance to stack them which
implies that one metaclass refines an other.

+1 for metaclasses as class decorators.

Kay

[Christopher Subich]

Kay Schluehr wrote:

I think it would be a good idea to pronounce the similarity between
function decorators and metaclasses. Metaclasses were once introduced
as an arcane art of fuzzy bearded hackers or supersmart 'enterprise
architects' that plan at least products of Zope size but not as a tool
for the simple programmer on the street. But maybe they should be and
there should also be librarys of them representing orthogonal
customizations ready for plug in.

In which case, I point out the need for better, more accessible
documentation. The Python 2.4 tutorial, for example, doesn't mention
them at all as far as I've noticed.

[Kay Schluehr]

Christopher Subich schrieb:

Kay Schluehr wrote:

I think it would be a good idea to pronounce the similarity between
function decorators and metaclasses. Metaclasses were once introduced
as an arcane art of fuzzy bearded hackers or supersmart 'enterprise
architects' that plan at least products of Zope size but not as a tool
for the simple programmer on the street. But maybe they should be and
there should also be librarys of them representing orthogonal
customizations ready for plug in.

In which case, I point out the need for better, more accessible
documentation. The Python 2.4 tutorial, for example, doesn't mention
them at all as far as I've noticed.

That's true also for properties and decorators - the latter may be
excused because they are new in Python 2.4. There are a few good
documents out there explaining advanced stuff e.g. mro and descriptors.
Maybe those texts should be delivered with the doc and linked from the
tutorial as "further reading"? But maybe the whole document structure
could be redesigned providing a Wikipedia style with seperated but
linked articles, example code etc. ?

Kay