Some language proposals.

[Antoon Pardon]

I'm rather new at this, so I don't know how I should
introduce this, nor whether these ideas are worth much
but here goes.

What I would like to change is access to variables on
an intermediate scope. Given the following example

def fun_1()

a = some_value

def fun_2()

# a is never the one above.
# on the left side of a statement.
The solution that I heard proposed was to use
a mutable variable. Something like
def fun_1():

a = [some_value]

def fun_2();

a[0] = new_value

And of course if you had a number of such variables
you could group then in an object
def fun_1():

class var_1:
pass

var_1.a = some_value
var_1.b = a_value_too

def fun_2():

var_1.a = new_value
var_1.b = next_value
Now my idea would be to look at the local variables in
a function like member variables of an object so that
you could write
def fun_1():

a = some_value
b = a_value_too

def fun_2():

fun_1.a = new_value
fun_1.b = next_value
This could maybe even extended further is seeing functions
like some kind of module which would allow something like
the following
def fun_1():

a = some_value
b = a_value_too

def fun_2():

from fun_1 import a, b

a = new_value
b = next_value
What do people think about this?
As far as I know the proposal doesn't break existing
code and seems in the spirit of python.
On a side note, what would people think about the idea of
a from ... import ... statement usable on any object? So
that we could do:

class A:
pass

A.a = ...
A.b = ...

from A import a, b

# from here a and be refer to A.a and A.b

--
Antoon Pardon

[Jeff Epler]

On Tue, Feb 24, 2004 at 03:53:42PM +0000, Antoon Pardon wrote:

Now my idea would be to look at the local variables in
a function like member variables of an object so that
you could write
def fun_1():

a = some_value
b = a_value_too

def fun_2():

fun_1.a = new_value
fun_1.b = next_value
This is not compatible with existing code. 'fun_1.a = new_value'
already has a meaning (it creates or modifies an attribute on the
function object), so it can't be used to modify the locals() of a
particular invocation of fun_1.
def fun_1():

a = some_value
b = a_value_too

def fun_2():

from fun_1 import a, b

a = new_value
b = next_value
What do people think about this?
As far as I know the proposal doesn't break existing
code and seems in the spirit of python.
This is not in the sprit of Python.

This already has a meaning:
from m import a
a = 3
.... it almost is the same as
import m as _m
a = m.a
del _m
a = 3
.... which basically just sets a to 3 (it doesn't modify the module m)
On a side note, what would people think about the idea of
a from ... import ... statement usable on any object? So
that we could do:

No, that won't do. Consider the following Python program:
sys = 64738
from sys import exit
exit()
The module name in 'import' and 'from ... import' doesn't refer to
objects in the current module's namespace, but in the namespace of
modules.

Anyway, because 'from .. import' is not magic, but just assignment,
there's no reason to prefer your proposed
from A import a, b
to
a, b = A.a, A.b
... for instance, it can't make 'a = 3' change A. It has a small
advantage that you don't need to type 'a' once on the left and once on
the right of '=', but that's about it.

Jeff

[Anton Vredegoor]

Antoon Pardon <ap*****@forel.vub.ac.be> wrote:

I'm rather new at this, so I don't know how I should
introduce this, nor whether these ideas are worth much
but here goes.
I'm rather new at criticizing new language proposals so forgive me for
not fighting new ideas ferociously enough :-) But seriously, I think
it's better to keep an eye open to new proposals. Even if things have
been discussed before, the fact that the same ideas come up again and
again should mean something in itself. Also a lot of proposals come in
the company of new ideas that have not yet been evaluated.
Now my idea would be to look at the local variables in
a function like member variables of an object so that
you could write
def fun_1():

a = some_value
b = a_value_too

def fun_2():

fun_1.a = new_value
fun_1.b = next_value
This could maybe even extended further is seeing functions
like some kind of module which would allow something like
the following
def fun_1():

a = some_value
b = a_value_too

def fun_2():

from fun_1 import a, b

a = new_value
b = next_value
What do people think about this?
As far as I know the proposal doesn't break existing
code and seems in the spirit of python.
I don't see much merit in this, since functions are designed to go out
of scope once they are executed. Having sub-scopes change names higher
up is considered a no-no in Python, although new developments like
"properties" are already nibbling little chunks away from this
concept.

However, in the case of *generator functions* this situation changes
dramatically, so you might score some points there. The road to take
would be something along the lines of "practicality beats purity".
On a side note, what would people think about the idea of
a from ... import ... statement usable on any object? So
that we could do:

class A:
pass

A.a = ...
A.b = ...

from A import a, b

# from here a and be refer to A.a and A.b

This raises the question of why not to use a class to begin with even
for your earlier proposals. Anyway, this can be accomplished easily
with:

a,b = A.a,A.b

If you still want to go through with this, have a look at this link:

http://us.st5.yimg.com/store4.yimg.c...s_1780_2263455

Anton

[Terry Reedy]

"Antoon Pardon" <ap*****@forel.vub.ac.be> wrote in message
news:sl********************@trout.vub.ac.be...

What I would like to change is access to variables on
an intermediate scope.

As I understand, you want write (rebind) access in addition to read
access -- without having to wrap the variable in a mutable, which one can
do at present. Some proposals to do this have been discussed on the PyDev
list. They should be listed in Brett Cannon's summaries, which are
archived on Python.org.

Bottom line so far: given that write access is only clumsy (sometimes,
through wrapping) rather than impossible, no proposal struck enough people
as worth the bother of a syntax change. Nested functions are much rarer
than unnested. Needing to rebind an outer local, rather than merely read
or mutate, is rarer still.

Terry J. Reedy

[Jacek Generowicz]

Antoon Pardon <ap*****@forel.vub.ac.be> writes:

The solution that I heard proposed was to use
a mutable variable. Something like
def fun_1():

a = [some_value]

def fun_2();

a[0] = new_value

Could somebody point me to an explanation of why closures are broken
in this way in the first place, please ?

[Jacek Generowicz]

"Terry Reedy" <tj*****@udel.edu> writes:

Nested functions are much rarer than unnested. Needing to rebind an
outer local, rather than merely read or mutate, is rarer still.

Chicken and Egg.

Sapir-Whorf.

I sumbit to you that read-only closures are rare in Python because
they are a recent addition to the language.

I sumbit to you that read-write closures are so much rarer still,
because they require an ugly hack to work.

[Antoon Pardon]

Op 2004-02-24, Anton Vredegoor schreef <an***@vredegoor.doge.nl>:

Antoon Pardon <ap*****@forel.vub.ac.be> wrote:

[ ... ]

What do people think about this?
As far as I know the proposal doesn't break existing
code and seems in the spirit of python.

I don't see much merit in this, since functions are designed to go out
of scope once they are executed. Having sub-scopes change names higher
up is considered a no-no in Python,

I don't understand why. It already is possible with global variables.
I don't see much difference between a function changing a higher up
variable that is global or just in between.

On a side note, what would people think about the idea of
a from ... import ... statement usable on any object? So
that we could do:

class A:
pass

A.a = ...
A.b = ...

from A import a, b

# from here a and be refer to A.a and A.b

This raises the question of why not to use a class to begin with even
for your earlier proposals. Anyway, this can be accomplished easily
with:

a,b = A.a,A.b

Well maybe my wording was off, but this doesn't accomplisch what I
want since after your statement followed by: a = new_value; A.a will
not be changed.

--
Antoon Pardon

[Paul Prescod]

Jacek Generowicz wrote:

...

Chicken and Egg.

Sapir-Whorf.

I sumbit to you that read-only closures are rare in Python because
they are a recent addition to the language.

I sumbit to you that read-write closures are so much rarer still,
because they require an ugly hack to work.

I disagree. Closures are rare in Python because Python is primarily an
OOP language.

http://www.kimbly.com/blog/000063.html

"""The venerable master Qc Na was walking with his student, Anton.
Hoping to prompt the master into a discussion, Anton said "Master, I
have heard that objects are a very good thing - is this true?" Qc Na
looked pityingly at his student and replied, "Foolish pupil - objects
are merely a poor man's closures."

Chastised, Anton took his leave from his master and returned to his
cell, intent on studying closures. He carefully read the entire
"Lambda: The Ultimate..." series of papers and its cousins, and
implemented a small Scheme interpreter with a closure-based object
system. He learned much, and looked forward to informing his master of
his progress.

On his next walk with Qc Na, Anton attempted to impress his master by
saying "Master, I have diligently studied the matter, and now understand
that objects are truly a poor man's closures." Qc Na responded by
hitting Anton with his stick, saying "When will you learn? Closures are
a poor man's object." At that moment, Anton became enlightened. """

Paul Prescod

[Paul Prescod]

Jacek Generowicz wrote:

Could somebody point me to an explanation of why closures are broken
in this way in the first place, please ?

a = 5
def b():
a = 6
print a
b()
print a

What will this print?

5
6

Okay, then.

a = 5
def b():
a = 6
def c():
a = 7
print a
print a
c()

print b()()
print a

What does this print?

7
6
5

Python uses a single syntax for declarating a variable and assigning to
it. This causes a problem for Lisp purists but is otherwise helpful.

Paul Prescod

[Jacek Generowicz]

Paul Prescod <pa**@prescod.net> writes:

Jacek Generowicz wrote:

Could somebody point me to an explanation of why closures are broken
in this way in the first place, please ?
a = 5
def b():
a = 6
def c():
a = 7
print a
print a
c()

print b()()

Methinks you forgot to return something somewhere.
Python uses a single syntax for declarating a variable and assigning
to it.
Ah, thank you, _this_ is the salient point ( ... though I'm not sure
what "declarating" means, but I'm pretty sure that Python does not do
it to variables :-).

But what happens in other languages which support closures, but have
no syntactic distinction between binding and re-binding ?
This causes a problem for Lisp purists but is otherwise helpful.

Lisp causes problems for the simple-minded, but is otherwise helpful.

(Infer as many winks/smileys as you implied in your original.)

[Jacek Generowicz]

Paul Prescod <pa**@prescod.net> writes:

I disagree. Closures are rare in Python because Python is primarily an
OOP language.

I disagree, Python is a multi-paradigm language ... I fail to see how
this has any bearing on the use of closures ... all of which is
irrelevant to my original point, which was to note out that saying
"people don't use it much" is not a very convincing argument for not
fixing something that is broken ... because the very fact that it is
broken probably contributes to people not using it much.

[Michael Hudson]

Jacek Generowicz <ja**************@cern.ch> writes:

"Terry Reedy" <tj*****@udel.edu> writes:

Nested functions are much rarer than unnested. Needing to rebind an
outer local, rather than merely read or mutate, is rarer still.
Chicken and Egg.

Sapir-Whorf.

Possibly. Also see Paul's response.
I sumbit to you that read-only closures are rare in Python because
they are a recent addition to the language.

Actually, I think uses of nested scopes are fairly common -- in test
suites! It was writing test cases that motivated us to implement them
for PyPy...

Cheers,
mwh

--
Guido (like us!) is a bit schizophrenic here: he wants to be a
benevolent dictator, but also wants to treat people like
grownups. This probably worked better before Python got a large
American audience <0.9 wink>. -- Tim Peters, 10 Feb 2000

[Michael Hudson]

Jacek Generowicz <ja**************@cern.ch> writes:

Paul Prescod <pa**@prescod.net> writes:

I disagree. Closures are rare in Python because Python is primarily an
OOP language.
I disagree, Python is a multi-paradigm language ... I fail to see how
this has any bearing on the use of closures ...

Really? Paul is saying "if you are in a situation in Python where you
want to use a read-write closure, you'll probably be happier if you
use an object-based solution instead".

Now you may *disagree* with this point, but I find it hard to believe
you don't *see* it.

I agree with him, FWIW. Using closures to fake objects sucks (in
scheme or CL as much as in Python).

Can you post an example of using a read-write closure that you think
wouldn't be better off as an object (invent syntax as necessary...).
No-one was very convincing at this last time it went around on
python-dev.
all of which is irrelevant to my original point, which was to note
out that saying "people don't use it much" is not a very convincing
argument for not fixing something that is broken ...
OK, how about the argument above?
because the very fact that it is broken probably contributes to
people not using it much.

Well, true as this may be, I hypothesise that it wouldn't be used much
if it wasn't "broken" and if it was used most of the time you'd wish
it wasn't. Not a particularly verifiable statement, I'll grant, but
things like this is what Guido is for :-)

Cheers,
mwh

--
Structure is _nothing_ if it is all you got. Skeletons _spook_
people if they try to walk around on their own. I really wonder
why XML does not. -- Erik Naggum, comp.lang.lisp

[Jacek Generowicz]

Michael Hudson <mw*@python.net> writes:

Jacek Generowicz <ja**************@cern.ch> writes:

"Terry Reedy" <tj*****@udel.edu> writes:

Nested functions are much rarer than unnested. Needing to rebind an
outer local, rather than merely read or mutate, is rarer still.

I sumbit to you that read-only closures are rare in Python because
they are a recent addition to the language.

Actually, I think uses of nested scopes are fairly common

While I don't pretend to have any idea how much the "typical" Python
programmer uses nested scopes, I am tempted to agree with you that
their use is quite common. However, I did not want to start a debate
(with Terry or anyone else) about the _absolute_ frequency of nested
functions, and so decided to phrase my statement in terms of an
agremment on the _relative_ infrequency of nested functions as
compared to unnested ones.

:-)

[Jacek Generowicz]

Michael Hudson <mw*@python.net> writes:

Jacek Generowicz <ja**************@cern.ch> writes:

Paul Prescod <pa**@prescod.net> writes:

I disagree. Closures are rare in Python because Python is primarily an
OOP language.
I disagree, Python is a multi-paradigm language ... I fail to see how
this has any bearing on the use of closures ...

Really? Paul is saying "if you are in a situation in Python where you
want to use a read-write closure, you'll probably be happier if you
use an object-based solution instead".

Now you may *disagree* with this point, but I find it hard to believe
you don't *see* it.

Aaah, OK, _if_ you think that closures are _only_ a way of faking
objects, _then_ claiming that Python is primarily OO does have a
bearing on the use of closures. Forgive me for not pursuing
this tangent any further.
Using closures to fake objects sucks (in scheme or CL as much as in
Python).
And using objects (err ... instances of classes) to fake stateful
functions sucks just as much.

When I need "objects" I use objects, and when I need closures I want
to be able to use closures.
Can you post an example of using a read-write closure that you think
wouldn't be better off as an object (invent syntax as necessary...).
No-one was very convincing at this last time it went around on
python-dev.

Sure. I have some stateful methods of classes (which I create
dynamically and stick onto the class as the information about the
existence of the method becomes available). By implementing them as
closures I can just stick them on to the class. If I were to implement
them as instances then I'd have to reimplement all the descriptors
that take care of turning functions into bound or unbound methods.

[At this point I'd love someone to step up and show me how to re-use
the existing FunctionType descriptors in my own classes.]

Another example. I make quite heavy use of a trivial memoizer. The
closure verison is much shorter, clearer and faster. (I want it to be
writeable because I sometimes want to 'prime' the cache, or do other
things to it ... so I'm giving myself access to the cache by binding
the cache as a function attribute of the closure, before returning it
from the memoizer)

In short, whenever I want a _function_ which happens to have some
internal state, I'd much rather use a closure than an instance of a class.

all of which is irrelevant to my original point, which was to note
out that saying "people don't use it much" is not a very convincing
argument for not fixing something that is broken ...

OK, how about the argument above?

My original point, although prompted in a context which discussed
closures, in independent of closures.

[Yes, I happen to wish that closures were writeable as god intended
them to be, dammit :-), but that is completely independent of my
objection to the self-fulfilling prophecy argument.]

[Anton Vredegoor]

Antoon Pardon <ap*****@forel.vub.ac.be> wrote:

I don't see much difference between a function changing a higher up
variable that is global or just in between.

If you want a function with its own namespace, that is already
possible as you showed yourself in an earlier post. On the other hand,
if a function has its own namespace what is the difference between a
function and a class? Why not use a class in the first place?

Remember that "global" is just a way for a function to tell the
interpreter that this variable belongs to the global namespace.

There is some inconsistency if one has a function inside a function
and that function has a global 'x' but the outer function doesn't have
a global 'x'. Modifications of 'x' from the deepest function do not
reach the global namespace, in fact they do not even reach the
namespace of the containing function.

Only if both functions have a global 'x' the name is visible at the
global and intermediate level.

However there is already a mechanism to solve these kind of problems:
namespaces! Why not use them?

a,b = A.a,A.b

Well maybe my wording was off, but this doesn't accomplisch what I
want since after your statement followed by: a = new_value; A.a will
not be changed.

For that to work you'd have to imbue names with new magical powers.
They would have to have some way of knowing who their fellow holders
are with whom they share their objects and inform them about their new
responsibilities.

The fellow objects would need some instructions as to how to handle
such things. Should they hold on to their previous bindings or let go
and use the new objects? Maybe some automatic message passing between
objects "behind the covers" would be necessary. Maybe it's a great
idea or maybe we would get hit over the head with a stick. I don't
know.

By the way, would it then be really, really black magic to be able to
call an object by its *real* name and force all holders to obey? One
ring to rule them all!

.... much, much, later ...

Message from your Atoned Governor.

Ok, after finally having relinquished the ring of power and freely
dropping it into the crater, I would like to make a public
announcement.

It is a really bad idea to indiscriminately rip the structure and
inner cohesion and the "whole is more than the sum of the parts"
quality of existing cooperations of objects apart in order to gain
access to a little trinket, and from now on I will desist from such
actions until there is a way to replace the lost objects with
functionality that is the same or better, according to the judgement
of said cooperations, in order not to disrupt working ecosystems.

Anton

[Paul Prescod]

Jacek Generowicz wrote:

Paul Prescod <pa**@prescod.net> writes:

I disagree. Closures are rare in Python because Python is primarily an
OOP language.
I disagree, Python is a multi-paradigm language ...

In Python functions are objects but objects are not functions. In (e.g.)
Scheme the opposite is true.
... I fail to see how
this has any bearing on the use of closures ... all of which is
irrelevant to my original point, which was to note out that saying
"people don't use it much" is not a very convincing argument for not
fixing something that is broken ... because the very fact that it is
broken probably contributes to people not using it much.

But the more important point is that people do not NEED it much. Guido
writes hundreds of lines of Python code per week. If he often ran into
situations where a mutable closure would make a big difference then he
would presumably find some way of doing it. The people who want this
seem most often to be people trying to import their coding styles from
another language.

Paul Prescod

[Paul Prescod]

While I'm thinking about it, let me refute your Sapir-Wharf claim (BTW,
its been pretty widly debunked in linguistics)

Before Python had nested scopes at all, it was VERY common to fake them
using a linguistic trick: default arguments. This idiom was very common
in Python's source base. Python didn't have a proper way to do the
nesting but people figured out how to do it anyway and did it a LOT.
(similarly people often emulate OO in C)

The "wrap your mutables" feature is by comparison _much less intrusive_
but I don't ever see it in real Python code. This suggests to me that
people don't need the feature that badly.

Jacek Generowicz wrote:

...
And using objects (err ... instances of classes) to fake stateful
functions sucks just as much.
Let's step above the words "function" and "object" for a second.
Sometimes you need to combine mutable state and code into a single unit.
Python has a first-class feature for doing that: the object. That's the
implementation of the requirement.

But you are expressing an _implementation_ as a _requirement_. "I need a
combination of state and data and it must be a function." Why?
Sure. I have some stateful methods of classes (which I create
dynamically and stick onto the class as the information about the
existence of the method becomes available).
Why are the METHODS stateful rather than keeping their state in the
class? If I had to maintain your code I would much rather that you keep
state in the same place all the other Python programmers do.

But if you must do this then use the well-known mutable closure hack.
It'll take you just a few extra characters of code. When we start seeing
dozens or hundreds of people making use of it then we'll know the
community really needs it (and isn't just trying to turn Python into
Lisp for its own sake).
... By implementing them as
closures I can just stick them on to the class. If I were to implement
them as instances then I'd have to reimplement all the descriptors
that take care of turning functions into bound or unbound methods.
Or you could wrap them in functions.

inst = Class()

someclass.method = lambda x, *args, **kwargs: inst(*args, **kwargs)

You've cleverly found one of the very few places where Python
distinguishes between "true" functions and callable objects. If one is
looking at the overall Python system, the bug is in the way functions
are bound. In general, Python very seldom distinguishes between objects
based on type (rather than behaviour) and in particular, seldom
distinguishes between functions and callable objects.

I would argue that classes should distinguish between
attributes-to-be-treated-as-methods and
attributes-to-be-treated-as-just-attributes using something other than
type. Consider the following program:

class a:
pass

def func():
pass

lst = [1, 2, 3, "a", "b", "c", object(), lambda x:x ]

for x in lst:
a.b = x
assert a.b is x, "What???? %s"% x
print "Okay", x

Frankly I think it is unpythonic that a second after assignment to a.b
the thing I get back is different than the thing I put in.

But now we're talking about a pretty subtle bug that has been in Python
for over ten years and it so seldom bites anyone that it is probably nto
worth fixing. (if I were to fix it I would add an add_method method to
classes)
Another example. I make quite heavy use of a trivial memoizer. The
closure verison is much shorter, clearer and faster.
Well it boils down to a question of who would believe it is clearer.
Lisp programmers think mutable closures are clearer. Perl programmers
think having a variety of different ways to spell "if" is clearer than
having just one. Part of the Zen of Python is try to encourage different
camps towards common spellings of idioms.

bash-2.05a$ python -c "import this" | grep way
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
In short, whenever I want a _function_ which happens to have some
internal state, I'd much rather use a closure than an instance of a class.

Barring the one ancient design misfeature above, callable instances are
almost always good replacements for functions.

all of which is irrelevant to my original point, which was to note
out that saying "people don't use it much" is not a very convincing
argument for not fixing something that is broken ...

OK, how about the argument above?

...

OK, how about the argument at the top? If people needed it they would
fake it as they do many other missing feature (in Python and other
languages). That they don't fake it despite how easy it is to do so
indicates that they don't need it.

Paul Prescod

[Michael Hudson]

Jacek Generowicz <ja**************@cern.ch> writes:

Michael Hudson <mw*@python.net> writes:

Jacek Generowicz <ja**************@cern.ch> writes:

Paul Prescod <pa**@prescod.net> writes:

> I disagree. Closures are rare in Python because Python is primarily an
> OOP language.

I disagree, Python is a multi-paradigm language ... I fail to see how
this has any bearing on the use of closures ...
Really? Paul is saying "if you are in a situation in Python where you
want to use a read-write closure, you'll probably be happier if you
use an object-based solution instead".

Now you may *disagree* with this point, but I find it hard to believe
you don't *see* it.

Aaah, OK, _if_ you think that closures are _only_ a way of faking
objects, _then_ claiming that Python is primarily OO does have a
bearing on the use of closures. Forgive me for not pursuing
this tangent any further.

OK, you really hadn't seen this argument before. You must be new here
:-) (or at least, new to this particular dispute).

Using closures to fake objects sucks (in scheme or CL as much as in
Python).

And using objects (err ... instances of classes) to fake stateful
functions sucks just as much.

Hmm. Really? I'm not sure that I often find myself wanting a
"stateful function". I think I probably usually use bound methods.
This could be the tail wagging the dog, of course.
When I need "objects" I use objects, and when I need closures I want
to be able to use closures.
So could this :-)

(trivially, you never *need* closures, just the same as you don't
*need* objects, there's a rather dull and unhelpful isomorphism
between the two concepts).

Can you post an example of using a read-write closure that you think
wouldn't be better off as an object (invent syntax as necessary...).
No-one was very convincing at this last time it went around on
python-dev.

Sure. I have some stateful methods of classes (which I create
dynamically and stick onto the class as the information about the
existence of the method becomes available). By implementing them as
closures I can just stick them on to the class. If I were to implement
them as instances then I'd have to reimplement all the descriptors
that take care of turning functions into bound or unbound methods.

Which isn't that hard... I think a code example might lend a bit more
clarity.
[At this point I'd love someone to step up and show me how to re-use
the existing FunctionType descriptors in my own classes.]
Post some code, and I'll have a go.
Another example. I make quite heavy use of a trivial memoizer. The
closure verison is much shorter, clearer and faster.
I'll grant you this: I've done the same.
(I want it to be writeable because I sometimes want to 'prime' the
cache, or do other things to it ... so I'm giving myself access to
the cache by binding the cache as a function attribute of the
closure, before returning it from the memoizer)
Um, here, unless I misunderstand you, you don't want to mutate the
closed over binding (which is what Python currently doesn't let you
do) but mutate the object the closed over binding is bound to. In
this respect, Python is no different from anything else, you need to
arrange some way to expose said object outside the closure.

What am I missing?
In short, whenever I want a _function_ which happens to have some
internal state, I'd much rather use a closure than an instance of a
class.

Maybe, once you've been assimilated into the Python Borg mindset, you
don't, actually.

There are sort of two extreme possibilities here:

1) I am so blinded by the fact that I know Python as of 2.3 FAR better
than any other language that I don't see how wonderful Python would
be if "full closures" were added.

2) Full closures just aren't that appropriate to Pythonic programming.

The truth, almost certainly, is somewhere in between these points.

all of which is irrelevant to my original point, which was to note
out that saying "people don't use it much" is not a very convincing
argument for not fixing something that is broken ...

OK, how about the argument above?

My original point, although prompted in a context which discussed
closures, in independent of closures.

Fine. Let's have a real argument instead :-)

Cheers,
mwh

--
Never meddle in the affairs of NT. It is slow to boot and quick to
crash. -- Stephen Harris
-- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html

[Aahz]

In article <ty*************@pcepsft001.cern.ch>,
Jacek Generowicz <ja**************@cern.ch> wrote:

Sure. I have some stateful methods of classes (which I create
dynamically and stick onto the class as the information about the
existence of the method becomes available). By implementing them as
closures I can just stick them on to the class. If I were to implement
them as instances then I'd have to reimplement all the descriptors
that take care of turning functions into bound or unbound methods.

Why not use callable instances?
--
Aahz (aa**@pythoncraft.com) <*> http://www.pythoncraft.com/

"Do not taunt happy fun for loops. Do not change lists you are looping over."
--Remco Gerlich, comp.lang.python

[Josiah Carlson]

Paul,

Your outputs were all incorrect.

Could somebody point me to an explanation of why closures are broken
in this way in the first place, please ?

a = 5
def b():
a = 6
print a
b()
print a

What will this print?

5
6

a = 5
def b(): .... a = 6
.... print a
.... b() 6 print a 5

Okay, then.

a = 5
def b():
a = 6
def c():
a = 7
print a
print a
c()

print b()()
print a

What does this print?

7
6
5

What you give doesn't work, here's a slight modification:
a = 5
def b(): .... a = 6
.... def c():
.... a = 7
.... print a
.... print a
.... return c
....
b()() 6
7 print a

5
I don't know what you were trying to prove, but next time, test it out.

- Josiah

[Jacek Generowicz]

aa**@pythoncraft.com (Aahz) writes:

In article <ty*************@pcepsft001.cern.ch>,
Jacek Generowicz <ja**************@cern.ch> wrote:

Sure. I have some stateful methods of classes (which I create
dynamically and stick onto the class as the information about the
existence of the method becomes available). By implementing them as
closures I can just stick them on to the class. If I were to implement
them as instances then I'd have to reimplement all the descriptors
that take care of turning functions into bound or unbound methods.

Why not use callable instances?

Errrrr ...

Reading the paragraph you quoted and trying to understand it is left
as an exercise to you. (Hint: it's the last sentence.)

[Jacek Generowicz]

Paul Prescod <pa**@prescod.net> writes:

But the more important point is that people do not NEED it much. Guido
writes hundreds of lines of Python code per week. If he often ran into
situations where a mutable closure would make a big difference then he
would presumably find some way of doing it. The people who want this
seem most often to be people trying to import their coding styles from
another language.

Methinks that your image of just how much Guido drives new additions to
the language, is a little dated.

[Jacek Generowicz]

Michael Hudson <mw*@python.net> writes:

Jacek Generowicz <ja**************@cern.ch> writes:

Aaah, OK, _if_ you think that closures are _only_ a way of faking
objects, _then_ claiming that Python is primarily OO does have a
bearing on the use of closures. Forgive me for not pursuing
this tangent any further.
OK, you really hadn't seen this argument before. You must be new here
:-) (or at least, new to this particular dispute).

Yes, I usually switch off rather early in the "Python purity vs
whatever" threads, and the "The current state of Python is programming
language Nirvana" threads. You see, I use Python because it lets me
gets certain jobs done easily and enjoyably, and don't really much
care for language patriotism. I will fiercely defend Python on its
merits (which are legion), but that does not preclude me from
sometimes noting that it would be handy if something were different
than it is right now.

My contribution to this thread started with a question: I paraphrase
"why are Python closures read only?". Paul gave me an answer which has
some merit and I am satisfied with that answer. Frankly, I have no
time for, or interest in discussing how often I should on should not be
using closures in Python. When they are convenient for me, I will use
them, and when callable instances are more convenient, I will use
those, regardless of what you and Paul might say, so there,
na-na-na-naaa-naaah !
(trivially, you never *need* closures, just the same as you don't
*need* objects, there's a rather dull and unhelpful isomorphism
between the two concepts).
Yes, and we all know about Turing equivalence too. So we don't need
Python at all.

Sure. I have some stateful methods of classes (which I create
dynamically and stick onto the class as the information about the
existence of the method becomes available). By implementing them as
closures I can just stick them on to the class. If I were to implement
them as instances then I'd have to reimplement all the descriptors
that take care of turning functions into bound or unbound methods.

Which isn't that hard...

It's not hard, but it's pointless, when there are ways of getting it
for free.
I think a code example might lend a bit more clarity.
What's unclear about it? I get the impression that you understand
exactly what the point is.

class foo: pass .... class callable: .... def __call__(self): print self
.... instance = callable()
def meth(self): print self .... foo.meth = meth
foo.instance = instance
f = foo()
f <__main__.foo instance at 0x815fa64> f.meth() <__main__.foo instance at 0x815fa64> # self == f f.instance() <__main__.callable instance at 0x815f624> # self != f f.meth <bound method foo.meth of <__main__.foo instance at 0x815fa64>> f.instance <__main__.callable instance at 0x815f624> # Hmm, it's not a method

meth behaves like a Python method, instance does not.
[At this point I'd love someone to step up and show me how to re-use
the existing FunctionType descriptors in my own classes.]

Post some code, and I'll have a go.

Find some way of making callable [from above] use
types.FunctionType.__get__. I don't see what code examples could
possibly help in specifying the problem more clearly (unless the code
is the solution, of course).

But please don't bother unless you really want to do it for your own
entertainment. My not having found a way to do it, is in no way
stopping me from achieving my goals. (Which is more than can be said
for contributing to this thread :-) There being perfectly viable
alternatives to doing it, means that I haven't invested much time in
finding an answer, so it may well be that there is as simple solution.

Another example. I make quite heavy use of a trivial memoizer. The
closure verison is much shorter, clearer and faster.

I'll grant you this: I've done the same.

That's my point. Closures are there, sometimes they are more
appropriate than instances, so use them. No point in discussing the
merits and demerits of the two alternatives and pontificating on what
is Pythonic or not.

(I want it to be writeable because I sometimes want to 'prime' the
cache, or do other things to it ... so I'm giving myself access to
the cache by binding the cache as a function attribute of the
closure, before returning it from the memoizer)

Um, here, unless I misunderstand you, you don't want to mutate the
closed over binding (which is what Python currently doesn't let you
do) but mutate the object the closed over binding is bound to. In
this respect, Python is no different from anything else, you need to
arrange some way to expose said object outside the closure.

What am I missing?

Nothing. It was an inappropriate example.

In short, whenever I want a _function_ which happens to have some
internal state, I'd much rather use a closure than an instance of a
class.

Maybe, once you've been assimilated into the Python Borg mindset, you
don't, actually.

Aaah, this thread is an attempt to assimilate me :-) Now I understand.
There are sort of two extreme possibilities here:

1) I am so blinded by the fact that I know Python as of 2.3 FAR better
than any other language that I don't see how wonderful Python would
be if "full closures" were added.

2) Full closures just aren't that appropriate to Pythonic programming.

The truth, almost certainly, is somewhere in between these points.

An excellent summary, which, in my opinion says all that remains to be
said in this thread.

And, with that, I take my leave. I bid you a good day, gentlemen.

[Jacek Generowicz]

Paul Prescod <pa**@prescod.net> writes:

You've cleverly found one of the very few places where Python
distinguishes between "true" functions and callable objects.
What bothers me more (I think, though I can't remember the exact
reasons why, right now, so maybe it's not that important), is that
Python distinguishes between "true" functions and "builtin" functions.
I would argue that classes should distinguish between
attributes-to-be-treated-as-methods and
attributes-to-be-treated-as-just-attributes using something other than
type.
Look again. They do. [Hint: descriptors]
Consider the following program:
class a:
pass

def func():
pass

lst = [1, 2, 3, "a", "b", "c", object(), lambda x:x ]

for x in lst:
a.b = x
assert a.b is x, "What???? %s"% x
print "Okay", x

Frankly I think it is unpythonic that a second after assignment to a.b
the thing I get back is different than the thing I put in.

I do wish you'd run your examples through a Python interpreter before
posting them. The little bugs you include, really interfere with
trying to understand what your point is.

I suspect that what you are trying to demonstrate is this:

for x in lst:

.... a.b = x
.... print a.b is x
....
1
1
1
1
1
1
1
0

Though why you need 3 integers, and 3 strings, and a whole lot of line
noise to do it, escapes me.

[An I've stated in my followup to Michael, I have nothing else to say
about closures in Python.]

[Paul Prescod]

Jacek Generowicz wrote:

I do wish you'd run your examples through a Python interpreter before
posting them. The little bugs you include, really interfere with
trying to understand what your point is.

Sorry. Don't know what else I can do:

bash-2.05a$ cat foo.py
class a:
pass

def func():
pass

lst = [1, 2, 3, "a", "b", "c", object(), lambda x:x ]

for x in lst:
a.b = x
assert a.b is x, "What???? %s"% x
print "Okay", x

bash-2.05a$ python foo.py
Okay 1
Okay 2
Okay 3
Okay a
Okay b
Okay c
Okay <object object at 0x116e20>
Traceback (most recent call last):
File "foo.py", line 11, in ?
assert a.b is x, "What???? %s"% x
AssertionError: What???? <function <lambda> at 0x122190>
bash-2.05a$

And this is after cutting and pasting the program from your reply back
into a file.

Given that I was trying to demonstrate an inconsistency in behaviour I
felt it helpful to show examples of logical behaviour.

Paul Prescod

[Jacek Generowicz]

Ooops, sorry, you're right, the bug was of _my_ creation this time.

[Michael Hudson]

Jacek Generowicz <ja**************@cern.ch> writes:

Michael Hudson <mw*@python.net> writes:

Jacek Generowicz <ja**************@cern.ch> writes:

Aaah, OK, _if_ you think that closures are _only_ a way of faking
objects, _then_ claiming that Python is primarily OO does have a
bearing on the use of closures. Forgive me for not pursuing
this tangent any further.
OK, you really hadn't seen this argument before. You must be new here
:-) (or at least, new to this particular dispute).

Yes, I usually switch off rather early in the "Python purity vs
whatever" threads,

Me too. I wonder why we're both still in this one?

[...]

(trivially, you never *need* closures, just the same as you don't
*need* objects, there's a rather dull and unhelpful isomorphism
between the two concepts).

Yes, and we all know about Turing equivalence too. So we don't need
Python at all.

Indeed. Break out the infinitely long tapes. But it does make "need"
a difficult word, is all.

Sure. I have some stateful methods of classes (which I create
dynamically and stick onto the class as the information about the
existence of the method becomes available). By implementing them as
closures I can just stick them on to the class. If I were to implement
them as instances then I'd have to reimplement all the descriptors
that take care of turning functions into bound or unbound methods.

Which isn't that hard...

It's not hard, but it's pointless, when there are ways of getting it
for free.

I think a code example might lend a bit more clarity.

What's unclear about it? I get the impression that you understand
exactly what the point is.

Well, now we're even on assuming too much about the other's
comprehension front :-)

class foo: pass ... class callable: ... def __call__(self): print self
... instance = callable()
def meth(self): print self ... foo.meth = meth
foo.instance = instance
f = foo()
f <__main__.foo instance at 0x815fa64> f.meth() <__main__.foo instance at 0x815fa64> # self == f f.instance() <__main__.callable instance at 0x815f624> # self != f f.meth <bound method foo.meth of <__main__.foo instance at 0x815fa64>> f.instance <__main__.callable instance at 0x815f624> # Hmm, it's not a method
meth behaves like a Python method, instance does not.

Ah, ok. To make it behave like a method, you need to make it a
descriptor, i.e. implement __get__ (and make everything in sight
new-style classes, of course).

[At this point I'd love someone to step up and show me how to re-use
the existing FunctionType descriptors in my own classes.]

Post some code, and I'll have a go.

Find some way of making callable [from above] use
types.FunctionType.__get__.

I don't think I can do *that*:
import types

class foo(object):
pass

class Callable(object):
__get__ = types.FunctionType.__get__

foo.inst = Callable()

print foo.inst

yields:

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "/usr/tmp/python-10511jfs.py", line 11, in ?
print foo.inst
TypeError: descriptor '__get__' requires a 'function' object but received a 'Callable'

but you can do something nearly equivalent:

import types

class foo(object):
pass

class Callable(object):
def __init__(self): # wonder why this is needed:
self.__name__ = 'Callable'
def __call__(self, ob):
return ob
def __get__(self, ob, cls=None):
return types.UnboundMethodType(self, ob, cls)

foo.inst = Callable()

print foo.inst
print foo().inst()

(needs 2.3, for 2.2 use new.instancemethod instead).
I don't see what code examples could possibly help in specifying the
problem more clearly (unless the code is the solution, of course).

But please don't bother unless you really want to do it for your own
entertainment.

Well, I've signed up to give a talk on new-style classes at Python-UK,
so I'd hope not to have to work too hard for this :-)

[...]

In short, whenever I want a _function_ which happens to have some
internal state, I'd much rather use a closure than an instance of a
class.

Maybe, once you've been assimilated into the Python Borg mindset, you
don't, actually.

Aaah, this thread is an attempt to assimilate me :-) Now I understand.

Damn, you noticed.

There are sort of two extreme possibilities here:

1) I am so blinded by the fact that I know Python as of 2.3 FAR better
than any other language that I don't see how wonderful Python would
be if "full closures" were added.

2) Full closures just aren't that appropriate to Pythonic programming.

The truth, almost certainly, is somewhere in between these points.

An excellent summary, which, in my opinion says all that remains to be
said in this thread.

And, with that, I take my leave. I bid you a good day, gentlemen.

And to you!

Cheers,
mwh

--
It's actually a corruption of "starling". They used to be carried.
Since they weighed a full pound (hence the name), they had to be
carried by two starlings in tandem, with a line between them.
-- Alan J Rosenthal explains "Pounds Sterling" on asr

[Aahz]

In article <ty*************@pcepsft001.cern.ch>,
Jacek Generowicz <ja**************@cern.ch> wrote:

aa**@pythoncraft.com (Aahz) writes:

In article <ty*************@pcepsft001.cern.ch>,
Jacek Generowicz <ja**************@cern.ch> wrote:

Sure. I have some stateful methods of classes (which I create
dynamically and stick onto the class as the information about the
existence of the method becomes available). By implementing them as
closures I can just stick them on to the class. If I were to implement
them as instances then I'd have to reimplement all the descriptors
that take care of turning functions into bound or unbound methods.

Why not use callable instances?

Errrrr ...

Reading the paragraph you quoted and trying to understand it is left
as an exercise to you. (Hint: it's the last sentence.)

Nope. Got nothing to do with my point:

class Callable:
def __call__(self):
print "!"

class C:
def __init__(self):
self.foo = Callable()

C().foo()

Now tell me why this doesn't do what you want.
--
Aahz (aa**@pythoncraft.com) <*> http://www.pythoncraft.com/

"Do not taunt happy fun for loops. Do not change lists you are looping over."
--Remco Gerlich, comp.lang.python

[Terry Reedy]

"Jacek Generowicz" <ja**************@cern.ch> wrote in message
news:ty*************@pcepsft001.cern.ch...

language Nirvana" threads. You see, I use Python because it lets me
gets certain jobs done easily and enjoyably, and don't really much
care for language patriotism. I will fiercely defend Python on its
merits (which are legion), but that does not preclude me from
sometimes noting that it would be handy if something were different
than it is right now.
Sounds sensible.
My contribution to this thread started with a question: I paraphrase
"why are Python closures read only?".

As I said before, because Python has a 'write locally unless directed
otherwise' rule, which is not going to change, and because no one has yet
come up with a convincingly good syntax or method to replace or extend the
global directive for directing otherwise -- where convincely good mean
enough better than mutable wrapping to be worth the bother. And because no
one has yet cared enough to effectively push the issue further.

On the aesthetics of wrapping: with a C background, the notion of rebind
vars in something else's strikes me as 'odder' that mutating a mutable. So
I do not see n = [n] as being that ugly, while understanding that someone
with a 'full-closure' language background well might. On the other hand,
the non-uniformity of write local, write global, but not in between, is
bothersome enough to enough developers that there was serious discussion of
how to do it.

Terry J. Reedy

[Jacek Generowicz]

Michael Hudson <mw*@python.net> writes:

Jacek Generowicz <ja**************@cern.ch> writes:

Michael Hudson <mw*@python.net> writes:

Jacek Generowicz <ja**************@cern.ch> writes:

Ah, ok. To make it behave like a method, you need to make it a
descriptor, i.e. implement __get__ (and make everything in sight
new-style classes, of course).
Yeeees, which is why waaaay upthread I wrote:
> If I were to implement them as instances then I'd have to
> reimplement all the descriptors that take care of turning
> functions into bound or unbound methods.
(although I did misplace the terminology a little, I realize.)
import types

class foo(object):
pass

class Callable(object):
def __init__(self): # wonder why this is needed:
self.__name__ = 'Callable'
def __call__(self, ob):
return ob
def __get__(self, ob, cls=None):
return types.UnboundMethodType(self, ob, cls)

foo.inst = Callable()

print foo.inst
print foo().inst()

(needs 2.3, for 2.2 use new.instancemethod instead).

Aha !

I was doing this stuff way back in 2.2[*], where you get
print foo.inst

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 7, in __get__
TypeError: cannot create 'instance method' instances

but it does indeed work in 2.3. Thanks for pointing that out.
Aaah, this thread is an attempt to assimilate me :-) Now I understand.

Damn, you noticed.

I'm well on the ball, I am.

Cheers,

[*] Actually, I'm still forced to use 2.2 in production for now.

[Jacek Generowicz]

aa**@pythoncraft.com (Aahz) writes:

class Callable:
def __call__(self):
print "!"

class C:
def __init__(self):
self.foo = Callable()

C().foo()

Now tell me why this doesn't do what you want.

Because

C.foo

Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: class C has no attribute 'foo'

for a start.

[Antoon Pardon]

Op 2004-02-25, Anton Vredegoor schreef <an***@vredegoor.doge.nl>:

Antoon Pardon <ap*****@forel.vub.ac.be> wrote:

I don't see much difference between a function changing a higher up
variable that is global or just in between.
If you want a function with its own namespace, that is already
possible as you showed yourself in an earlier post. On the other hand,
if a function has its own namespace what is the difference between a
function and a class? Why not use a class in the first place?

Maybe because somtimes it is more easy or natural to think
of a problems in terms of a function and sometimes in terms
of a
Remember that "global" is just a way for a function to tell the
interpreter that this variable belongs to the global namespace.

There is some inconsistency if one has a function inside a function
and that function has a global 'x' but the outer function doesn't have
a global 'x'. Modifications of 'x' from the deepest function do not
reach the global namespace, in fact they do not even reach the
namespace of the containing function.

Only if both functions have a global 'x' the name is visible at the
global and intermediate level.
Well that looks like either poor design or poor implementation to
me.
However there is already a mechanism to solve these kind of problems:
namespaces! Why not use them?
Maybe because that solution can sometimes seem forced onto the
problem. Like if all you have is carpenting tools you only
can provide carpenting solutions.

a,b = A.a,A.b

Well maybe my wording was off, but this doesn't accomplisch what I
want since after your statement followed by: a = new_value; A.a will
not be changed.

Aaargh!!!!, A dreadfull type on my part that completly changed the
meaning of what I wanteds to say. "not" should read "now"

--
Antoon Pardon

[Antoon Pardon]

Op 2004-02-25, Paul Prescod schreef <pa**@prescod.net>:

Jacek Generowicz wrote:

Paul Prescod <pa**@prescod.net> writes:

I disagree. Closures are rare in Python because Python is primarily an
OOP language.

I disagree, Python is a multi-paradigm language ...

In Python functions are objects but objects are not functions. In (e.g.)
Scheme the opposite is true.

... I fail to see how
this has any bearing on the use of closures ... all of which is
irrelevant to my original point, which was to note out that saying
"people don't use it much" is not a very convincing argument for not
fixing something that is broken ... because the very fact that it is
broken probably contributes to people not using it much.

But the more important point is that people do not NEED it much. Guido
writes hundreds of lines of Python code per week. If he often ran into
situations where a mutable closure would make a big difference then he
would presumably find some way of doing it. The people who want this
seem most often to be people trying to import their coding styles from
another language.

I think this can be turned around. People who don't want this included
are people who want python to be used in a "pythonic" style.

My idea is that there is nothing wrong with a laguage that supports
different coding styles. Personnaly I switch styles according to
how view the solution to the problems I have to solve. Sometimes
I view such a solution in a very object oriented way, sometimes
I don't. So when I envision a particular solution, that is best
implemented in a particular style, why shouldn't I use that style
just because I program in python.

--
Antoon Pardon

[Michael Hudson]

Jacek Generowicz <ja**************@cern.ch> writes:

import types

class foo(object):
pass

class Callable(object):
def __init__(self): # wonder why this is needed:
self.__name__ = 'Callable'
def __call__(self, ob):
return ob
def __get__(self, ob, cls=None):
return types.UnboundMethodType(self, ob, cls)

foo.inst = Callable()

print foo.inst
print foo().inst()

(needs 2.3, for 2.2 use new.instancemethod instead).
Aha !

Glad to be of service...
I was doing this stuff way back in 2.2[*], where you get

print foo.inst

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 7, in __get__
TypeError: cannot create 'instance method' instances

but it does indeed work in 2.3. Thanks for pointing that out.

This is one of the many little ways -- particularly in the area of
new-style classes -- in which Python 2.3 is just Python 2.2 done
right.

Cheers,
mwh

--
I also feel it essential to note, [...], that Description Logics,
non-Monotonic Logics, Default Logics and Circumscription Logics
can all collectively go suck a cow. Thank you.
-- http://advogato.org/person/Johnath/diary.html?start=4

[Josiah Carlson]

>>class Callable:

def __call__(self):
print "!"

class C:
def __init__(self):
self.foo = Callable()

C().foo()

Now tell me why this doesn't do what you want.

Because

C.foo

Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: class C has no attribute 'foo'

for a start.

Perhaps you want the below. You seem to be interested in doing things
with classes and not instances.

class Callable:
def __call__(self):
print "!"

class C:
foo = Callable()

- Josiah

[Jacek Generowicz]

Josiah Carlson <jc******@nospam.uci.edu> writes:

Perhaps you want the below. You seem to be interested in doing things
with classes and not instances.
class Callable:
def __call__(self):
print "!"

class C:
foo = Callable()

- Josiah

Perhaps you want to read <ty*************@pcepsft001.cern.ch> again:

http://www.google.com/groups?safe=im...n.ch&lr=&hl=en

Here's the relevant part:

Jacek Generowicz <ja**************@cern.ch> writes:

class foo: pass ... class callable: ... def __call__(self): print self
... instance = callable()
def meth(self): print self ... foo.meth = meth
foo.instance = instance
f = foo()
f <__main__.foo instance at 0x815fa64> f.meth() <__main__.foo instance at 0x815fa64> # self == f f.instance() <__main__.callable instance at 0x815f624> # self != f f.meth <bound method foo.meth of <__main__.foo instance at 0x815fa64>> f.instance <__main__.callable instance at 0x815f624> # Hmm, it's not a method

meth behaves like a Python method, instance does not.