Nested scopes, and augmented assignment

On 2006-07-04, Diez B. Roggisch <de***@nospam.web.dewrote:

Tim N. van der Leeuw wrote:

>Hi,

The following might be documented somewhere, but it hit me unexpectedly
and I couldn't exactly find this in the manual either.

Problem is, that I cannot use augmented assignment operators in a
nested scope, on variables from the outer scope:

<snip/>

>Is this an implementation artifact, bug, or should it really just
follow logically from the language definition?

From the docs:

"""
An augmented assignment expression like x += 1 can be rewritten as x = x + 1
to achieve a similar, but not exactly equal effect. In the augmented
version, x is only evaluated once. Also, when possible, the actual
operation is performed in-place, meaning that rather than creating a new
object and assigning that to the target, the old object is modified
instead.
"""

The first part is the important one. If you expand

x += 1

to

x = x + 1

it becomes clear under the python scoping rules that x is being treated as a
local to the inner scope.

There has been a discussion about this recently on python-dev[1] and this
NG - google for it.

Well no matter what explanation you give to it, and I understand how it
works, I keep finding it strange that something like

k = [0]
def f(i):
k[0] += i
f(2)

works but the following doesn't

k = 0
def f(i):
k += i
f(2)
Personnaly I see no reason why finding a name/identifier on the
leftside of an assignment should depend on whether the name
is the target or prefix for the target.
But maybe that is just me.

--
Antoon Pardon

Jul 5 '06 #3

Antoon Pardon wrote:
(snip)

Well no matter what explanation you give to it, and I understand how it
works,

I'm not sure of this.

I keep finding it strange that something like

k = [0]
def f(i):
k[0] += i
f(2)

works but the following doesn't

k = 0
def f(i):
k += i
f(2)
Personnaly I see no reason why finding a name/identifier on the
leftside of an assignment should depend on whether the name
is the target or prefix for the target.

It's not about "finding a name/identifier", it's about the difference
between (re)binding a name and mutating an object.
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"

Jul 5 '06 #4

Bruno Desthuilliers wrote:

It's not about "finding a name/identifier", it's about the difference
between (re)binding a name and mutating an object.

the difference between binding and performing an operation on an object
(mutating or not), in fact.

this is Python 101.

</F>

Jul 5 '06 #5

On 2006-07-05, Bruno Desthuilliers <on***@xiludom.growrote:

Antoon Pardon wrote:
(snip)
>Well no matter what explanation you give to it, and I understand how it
works,

I'm not sure of this.

Should I care about that?

>I keep finding it strange that something like

k = [0]
def f(i):
k[0] += i
f(2)

works but the following doesn't

k = 0
def f(i):
k += i
f(2)
Personnaly I see no reason why finding a name/identifier on the
leftside of an assignment should depend on whether the name
is the target or prefix for the target.

It's not about "finding a name/identifier", it's about the difference
between (re)binding a name and mutating an object.

The two don't contradict each other. Python has chosen that it won't
rebind variables that are out of the local scope. So if the lefthand
side of an assignment is a simple name it will only search in the
local scope for that name. But if the lefthand side is more complicated
if will also search the outerscopes for the name.

Python could have chosen an approach with a "nested" keyword, to allow
rebinding a name in an intermediate scope. It is not that big a deal
that it hasn't, but I keep finding the result strange and somewhat
counterintuitive.

Let me explain why:

Suppose someone is rather new of python and writes the following
code, manipulating vectors:

A = 10 * [0]
def f(B):
...
for i in xrange(10):
A[i] += B[i]
...

Then he hears about the vector and matrix modules that are around.
So he rewrites his code naively as follows:

A = NullVector(10):
def f(B):
...
A += B
...

And it won't work. IMO the principle of least surprise here would
be that it should work.

--
Antoon Pardon

Jul 5 '06 #6

Antoon Pardon wrote:

Python could have chosen an approach with a "nested" keyword

sure, and Python could also have been invented by aliens, powered by
space potatoes, and been illegal to inhale in Belgium.

have any of your "my mental model of how Python works is more important
than how it actually works" ever had a point ?

</F>

Jul 5 '06 #7

>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:

>APOn 2006-07-05, Bruno Desthuilliers <on***@xiludom.growrote:

>>Antoon Pardon wrote:
(snip)
Well no matter what explanation you give to it, and I understand how it
works,

I'm not sure of this.

>APShould I care about that?

Yes, because as long as you don't understand it, you are in for unpleasant
surprises.

>>It's not about "finding a name/identifier", it's about the difference
between (re)binding a name and mutating an object.

>APThe two don't contradict each other. Python has chosen that it won't
APrebind variables that are out of the local scope. So if the lefthand
APside of an assignment is a simple name it will only search in the
APlocal scope for that name. But if the lefthand side is more complicated
APif will also search the outerscopes for the name.

No. It will always use the same search order. But a variable that is bound
inside the function (with an asignment) and is not declared global, is in
the local namespace. A variable that is not assigned to inside the function
is not in the local namespace. An assignment to A[i] is not rebinding A, so
it doesn't count to make the variable A local.

>APPython could have chosen an approach with a "nested" keyword, to allow
APrebinding a name in an intermediate scope. It is not that big a deal
APthat it hasn't, but I keep finding the result strange and somewhat
APcounterintuitive.

Maybe it would have been nice if variables could have been declared as
nested, but I think it shows that nested variables have to be used with
care, similar to globals. Especially not allowing rebinding in intermediate
scopes is a sound principle (`Nested variables considered harmful').
If you need to modify the objects which are bound to names in intermediate
scopes, use methods and give these objects as parameters.

>APLet me explain why:

>APSuppose someone is rather new of python and writes the following
APcode, manipulating vectors:

>AP A = 10 * [0]
AP def f(B):
AP ...
AP for i in xrange(10):
AP A[i] += B[i]
AP ...

>APThen he hears about the vector and matrix modules that are around.
APSo he rewrites his code naively as follows:

>AP A = NullVector(10):
AP def f(B):
AP ...
AP A += B
AP ...

>APAnd it won't work. IMO the principle of least surprise here would
APbe that it should work.

Well, A = f(B) introduces a local variable A. Therefore also A = A+B.
And therefore also A += B.

The only way to have no surprises at all would be if local variables would
have to be declared explicitly, like 'local A'. But that would break
existing code. Or maybe the composite assignment operators should have been
exempted. Too late now!

You have the same problem with:

A = [10]
def inner():
A.append(2)

works but

A = [10]
def inner():
A += [2]

doesn't. The nasty thing in this example is that although A += [2] looks
like a rebinding syntactically, semantically there is no rebinding done.

I think the cleaner solution is (use parameters and don't rebind):
def inner(A):
A.append(2)

--
Piet van Oostrum <pi**@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email: pi**@vanoostrum.org

Jul 5 '06 #8

On 2006-07-05, Fredrik Lundh <fr*****@pythonware.comwrote:

Antoon Pardon wrote:

>Python could have chosen an approach with a "nested" keyword

sure, and Python could also have been invented by aliens, powered by
space potatoes, and been illegal to inhale in Belgium.

At one time one could have reacted the same when people suggested
python could use a ternary operator. In the mean time a ternary
operator is in the pipeline. If you don't want to discuss how
python could be different with me, that is fine, but I do no
harm discussing such things with others.

have any of your "my mental model of how Python works is more important
than how it actually works" ever had a point ?

Be free to correct me. But just suggesting that I'm wrong doesn't help
me in changing my mental model.

--
Antoon Pardon

Jul 6 '06 #9

On 2006-07-05, Piet van Oostrum <pi**@cs.uu.nlwrote:

>>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:

>>APOn 2006-07-05, Bruno Desthuilliers <on***@xiludom.growrote:
>>>Antoon Pardon wrote:
(snip)
Well no matter what explanation you give to it, and I understand how it
works,

I'm not sure of this.

>>APShould I care about that?

Yes, because as long as you don't understand it, you are in for unpleasant
surprises.

Well if someone explains what is wrong about my understanding, I
certainly care about that (although I confess to sometimes being
impatient) but someone just stating he is not sure I understand?

>>>It's not about "finding a name/identifier", it's about the difference
between (re)binding a name and mutating an object.

>>APThe two don't contradict each other. Python has chosen that it won't
APrebind variables that are out of the local scope. So if the lefthand
APside of an assignment is a simple name it will only search in the
APlocal scope for that name. But if the lefthand side is more complicated
APif will also search the outerscopes for the name.

No. It will always use the same search order.

So if I understand you correctly in code like:

c.d = a
b = a

All three names are searched for in all scopes between the local en global
one. That is what I understand with your statement that [python] always
uses the same search order.

My impression was that python will search for c and a in the total current
namespace but will not for b.

But a variable that is bound
inside the function (with an asignment) and is not declared global, is in
the local namespace.

Aren't we now talking about implementation details? Sure the compilor
can set things up so that local names are bound to the local scope and
so the same code can be used. But it seems somewhere was made the
decision that b was in the local scope without looking for that b in
the scopes higher up.

Let me explain a bit more. Suppose I'm writing a python interpreter
in python. One implemantation detail is that I have a list of active
scopes which are directories which map names to objects. At the
start of a new function the scope list is adapted and all local
variables are inserted to the new activated scope and mapped to
some "Illegal Value" object. Now I also have a SearchName function
that will start at the begin of a scope list and return the
first scope in which that name exists. The [0] element is the
local scope. Now we come to the line "b = a"

This could be then executed internally as follows:

LeftScope = SearchName("b", ScopeList)
RightScope = SearchName("a", ScopeList)
LeftScope["b"] = RightScope["a"]

But I don't have to do it this way. I already know in which scope
"b" is, the local one, which has index 0. So I could just as well
have that line exucuted as follows:

LeftScope = ScopeList[0]
RightScope = SearchName("a", ScopeList)
LeftScope["b"] = RightScope["a"]

As far as I understand both "implementations" would make for
a correct execution of the line "b = a" and because of the
second possibility, b is IMO not conceptually searched for in
the same way as a is searched for, although one could organise
things that the same code is used for both.

Of course it is possible I completely misunderstood how python
is supposed to work and the above is nonesense in which case
I would appreciate it if you correct me.

>>APPython could have chosen an approach with a "nested" keyword, to allow
APrebinding a name in an intermediate scope. It is not that big a deal
APthat it hasn't, but I keep finding the result strange and somewhat
APcounterintuitive.

Maybe it would have been nice if variables could have been declared as
nested, but I think it shows that nested variables have to be used with
care, similar to globals. Especially not allowing rebinding in intermediate
scopes is a sound principle (`Nested variables considered harmful').
If you need to modify the objects which are bound to names in intermediate
scopes, use methods and give these objects as parameters.

But shouldn't we just do programming in general with care? And if
Nested variables are harmfull, what is then the big difference
between rebinding them and mutating them that we should forbid
the first and allow the second?

I understand that python evolved and that this sometimes results
in things that in hindsight could have been done better. But
I sometimes have the impression that the defenders try to defend
those results as a design decision. With your remark above I have
to wonder if someone really thought this through at design time
and came to the conclusion that nested variables are harmfull
and thus may not be rebound but not that harmfull so mutation
is allowed and if so how he came to that conclusion.

>>APLet me explain why:

>>APSuppose someone is rather new of python and writes the following
APcode, manipulating vectors:

>>AP A = 10 * [0]
AP def f(B):
AP ...
AP for i in xrange(10):
AP A[i] += B[i]
AP ...

>>APThen he hears about the vector and matrix modules that are around.
APSo he rewrites his code naively as follows:

>>AP A = NullVector(10):
AP def f(B):
AP ...
AP A += B
AP ...

>>APAnd it won't work. IMO the principle of least surprise here would
APbe that it should work.

Well, A = f(B) introduces a local variable A. Therefore also A = A+B.
And therefore also A += B.

The only way to have no surprises at all would be if local variables would
have to be declared explicitly, like 'local A'. But that would break
existing code. Or maybe the composite assignment operators should have been
exempted. Too late now!

Let me make one thing clear. I'm not trying to get the python people to
change anything. Most I hope for is that they would think about this
behaviour for python 3000.

You have the same problem with:

A = [10]
def inner():
A.append(2)

works but

A = [10]
def inner():
A += [2]

doesn't. The nasty thing in this example is that although A += [2] looks
like a rebinding syntactically, semantically there is no rebinding done.

I think the cleaner solution is (use parameters and don't rebind):
def inner(A):
A.append(2)

For what it is worth. My proposal would be to introduce a rebinding
operator, ( just for the sake of this exchange written as := ).

A line like "b := a", wouldn't make b a local variable but would
search for the name b in all active scopes and then rebind b
there. In terms of my hypothetical interpreter something like
the above.

LeftScope = SearchName("b", ScopeList)
RightScope = SearchName("a", ScopeList)
LeftScope["b"] = RightScope["a"]

With the understanding that b wouldn't be inserted in the local
scope unless there was an assignment to be somewhere else in
the function.

The augmented assignments could then be redefined in terms of the
rebinding operator instead of the assignment.

I'm not sure this proposal would eliminate all surprises but as
far as I can see it wouldn't break existing code. But I don't think
this proposal would have a serious chance.

In any case thanks for your contribution.

--
Antoon Pardon

Jul 6 '06 #10

Antoon Pardon wrote:

On 2006-07-05, Piet van Oostrum <pi**@cs.uu.nlwrote:

>>>>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:

>>>APOn 2006-07-05, Bruno Desthuilliers <on***@xiludom.growrote:

>Antoon Pardon wrote:
>(snip)
>
>>Well no matter what explanation you give to it, and I understand how it
>>works,
>
>I'm not sure of this.

>>>APShould I care about that?

Yes, because as long as you don't understand it, you are in for unpleasant
surprises.

Well if someone explains what is wrong about my understanding, I
certainly care about that (although I confess to sometimes being
impatient) but someone just stating he is not sure I understand?

From what you wrote, I cannot decide if you really understand Python's
lookup rules and poorly express some disagreement on it, or if you just
don't understand Python lookup rules at all.

>

>>>>>It's not about "finding a name/identifier", it's about the difference
>between (re)binding a name and mutating an object.

>>>APThe two don't contradict each other. Python has chosen that it won't
APrebind variables that are out of the local scope. So if the lefthand
APside of an assignment is a simple name it will only search in the
APlocal scope for that name. But if the lefthand side is more complicated
APif will also search the outerscopes for the name.

Now it's pretty clear you *don't* understand.

In the second case, ie:

k = [0]
def f(i):
k[0] += i

'k[0]' is *not* a name. The name is 'k'. If we rewrite this snippet
without all the syntactic sugar, we get something like:

k = [0]
def f(i):
k.__setitem_(0, k.__getitem__(0) + i)

Now where do you see any rebinding here ?

>
>>No. It will always use the same search order.

So if I understand you correctly in code like:

c.d = a
b = a

All three names

which ones ?

are searched for in all scopes between the local en global
one.

In this example, we're at the top level, so the local scope is the
global scope. I assert what you meant was:

c = something
a = something_else

def somefunc():
c.d = a
b = a

(NB : following observations will refer to this code)

That is what I understand with your statement that [python] always
uses the same search order.

yes.

My impression was that python will search for c and a in the total current
namespace

what is "the total current namespace" ?

but will not for b.

b is bound in the local namespace, so there's no need to look for it in
enclosing namespaces.

>
>>But a variable that is bound
inside the function (with an asignment) and is not declared global, is in
the local namespace.

Aren't we now talking about implementation details?

Certainly not. Namespaces and names lookup rules are fundamental parts
of the Python language.

Sure the compilor
can set things up so that local names are bound to the local scope and
so the same code can be used. But it seems somewhere was made the
decision that b was in the local scope without looking for that b in
the scopes higher up.

binding creates a name in the current namespace. b is bound in the local
namespace, so b is local. period.

(snip)

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"

Jul 6 '06 #11

>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:

>APWell if someone explains what is wrong about my understanding, I
APcertainly care about that (although I confess to sometimes being
APimpatient) but someone just stating he is not sure I understand?

That is just a euphemistic way of stating `I think you do not understand
it'.

>APSo if I understand you correctly in code like:

>AP c.d = a
AP b = a

>APAll three names are searched for in all scopes between the local en global
APone. That is what I understand with your statement that [python] always
APuses the same search order.

The d is different because it is an attribute. So it is looked up in the
context of the object that is bound to c. For a, b, and c it is correct.

>APMy impression was that python will search for c and a in the total current
APnamespace but will not for b.

The assignment to b inside the function (supposing the code above is part
of the function body) tells the compiler that b is a local variable. So the
search stops in the local scope. The search order is always from local to
global. First the current function, then nested function, then the module
namespace, and finally the builtins. The first match will stop the search.
Now for local variables inside a function, including the parameters, the
compiler will usually optimize the search because it knows already that it
is a local variable. But that is an implementation detail.

>>But a variable that is bound
inside the function (with an asignment) and is not declared global, is in
the local namespace.

>APAren't we now talking about implementation details? Sure the compilor
APcan set things up so that local names are bound to the local scope and
APso the same code can be used. But it seems somewhere was made the
APdecision that b was in the local scope without looking for that b in
APthe scopes higher up.

Yes, as I (and others) have already said several times: an assignment to a
variable inside a function body (but not an assignment to an attribute or
part of an object) without a global declaration makes that variable a local
variable. That is not an implementation detail; it is part of the language definition.

>APLet me explain a bit more. Suppose I'm writing a python interpreter
APin python. One implemantation detail is that I have a list of active
APscopes which are directories which map names to objects. At the
APstart of a new function the scope list is adapted and all local
APvariables are inserted to the new activated scope and mapped to
APsome "Illegal Value" object. Now I also have a SearchName function
APthat will start at the begin of a scope list and return the
APfirst scope in which that name exists. The [0] element is the
APlocal scope. Now we come to the line "b = a"

>APThis could be then executed internally as follows:

>AP LeftScope = SearchName("b", ScopeList)
AP RightScope = SearchName("a", ScopeList)
AP LeftScope["b"] = RightScope["a"]

>APBut I don't have to do it this way. I already know in which scope
AP"b" is, the local one, which has index 0. So I could just as well
APhave that line exucuted as follows:

>AP LeftScope = ScopeList[0]
AP RightScope = SearchName("a", ScopeList)
AP LeftScope["b"] = RightScope["a"]

>APAs far as I understand both "implementations" would make for
APa correct execution of the line "b = a" and because of the
APsecond possibility, b is IMO not conceptually searched for in
APthe same way as a is searched for, although one could organise
APthings that the same code is used for both.

That is the optimization I spoke of above. But it is not the problem we
were discussing. Conceptually it is the same. It is similar to constant
folding (replacing x = 2+3 by x = 5).

>APOf course it is possible I completely misunderstood how python
APis supposed to work and the above is nonesense in which case
API would appreciate it if you correct me.

>APPython could have chosen an approach with a "nested" keyword, to allow
APrebinding a name in an intermediate scope. It is not that big a deal
APthat it hasn't, but I keep finding the result strange and somewhat
APcounterintuitive.

>>>
Maybe it would have been nice if variables could have been declared as
nested, but I think it shows that nested variables have to be used with
care, similar to globals. Especially not allowing rebinding in intermediate
scopes is a sound principle (`Nested variables considered harmful').
If you need to modify the objects which are bound to names in intermediate
scopes, use methods and give these objects as parameters.

>APBut shouldn't we just do programming in general with care? And if
APNested variables are harmfull, what is then the big difference
APbetween rebinding them and mutating them that we should forbid
APthe first and allow the second?

There is no big difference I think. Only Python doesn't have syntax for the
former. Older versions of Python didn't even have nested scopes. maybe it
was a mistake to add them. I think an important reason was the use in
lambda expressions, which Guido also regrets IIRC.

>API understand that python evolved and that this sometimes results
APin things that in hindsight could have been done better. But
API sometimes have the impression that the defenders try to defend
APthose results as a design decision. With your remark above I have
APto wonder if someone really thought this through at design time
APand came to the conclusion that nested variables are harmfull
APand thus may not be rebound but not that harmfull so mutation
APis allowed and if so how he came to that conclusion.

I don't think that was the reasoning. On the other hand I think nested
scopes were mainly added for read-only access (to the namespace that is,
not to the values) . But then Python doesn't forbid you to change mutable
objects once you have access to them.
--
Piet van Oostrum <pi**@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email: pi**@vanoostrum.org

Jul 6 '06 #12

On 2006-07-06, Piet van Oostrum <pi**@cs.uu.nlwrote:

>>APAren't we now talking about implementation details? Sure the compilor
APcan set things up so that local names are bound to the local scope and
APso the same code can be used. But it seems somewhere was made the
APdecision that b was in the local scope without looking for that b in
APthe scopes higher up.

Yes, as I (and others) have already said several times: an assignment to a
variable inside a function body (but not an assignment to an attribute or
part of an object) without a global declaration makes that variable a local
variable. That is not an implementation detail; it is part of the language definition.

You seem to think I didn't understand this. Maybe I'm not very good
at explaining what I mean, but you really haven't told me anything
here I didn't already know.

>>AP[ ... ]
APNow we come to the line "b = a"

>>APThis could be then executed internally as follows:

>>AP LeftScope = SearchName("b", ScopeList)
AP RightScope = SearchName("a", ScopeList)
AP LeftScope["b"] = RightScope["a"]

>>APBut I don't have to do it this way. I already know in which scope
AP"b" is, the local one, which has index 0. So I could just as well
APhave that line exucuted as follows:

>>AP LeftScope = ScopeList[0]
AP RightScope = SearchName("a", ScopeList)
AP LeftScope["b"] = RightScope["a"]

>>APAs far as I understand both "implementations" would make for
APa correct execution of the line "b = a" and because of the
APsecond possibility, b is IMO not conceptually searched for in
APthe same way as a is searched for, although one could organise
APthings that the same code is used for both.

That is the optimization I spoke of above. But it is not the problem we
were discussing.

Could you maybe clarify what problem we are discussing? All I wrote
was that with an assignment the search for the lefthand variable
depends on whether the lefthand side is a simple variable or
more complicated. Sure people may prefer to speak about (re)binding
vs mutating variables, but just because I didn't use the prefered terms,
starting to doubt my understanding of the language, seems a bit
premature IMO. I'm sure there are areas where my understanding of
the language is shaky, metaclasses being one of them, but understanding
how names are searched doesn't seem to be one of them.

--
Antoon Pardon

Jul 6 '06 #13

On 2006-07-06, Bruno Desthuilliers <on***@xiludom.growrote:

Antoon Pardon wrote:
>On 2006-07-05, Piet van Oostrum <pi**@cs.uu.nlwrote:

>>>>>>It's not about "finding a name/identifier", it's about the difference
>>between (re)binding a name and mutating an object.

APThe two don't contradict each other. Python has chosen that it won't
APrebind variables that are out of the local scope. So if the lefthand
APside of an assignment is a simple name it will only search in the
APlocal scope for that name. But if the lefthand side is more complicated
APif will also search the outerscopes for the name.

Now it's pretty clear you *don't* understand.

In the second case, ie:

k = [0]
def f(i):
k[0] += i

'k[0]' is *not* a name. The name is 'k'. If we rewrite this snippet
without all the syntactic sugar, we get something like:

k = [0]
def f(i):
k.__setitem_(0, k.__getitem__(0) + i)

Now where do you see any rebinding here ?

What point do you want to make? As far as I can see, I
didn't write anything that implied I expected k to
be rebound in code like

k[0] += i

So why are you trying so hard to show me this?

>>
>>>No. It will always use the same search order.

So if I understand you correctly in code like:

c.d = a
b = a

All three names

which ones ?

>are searched for in all scopes between the local en global
one.

In this example, we're at the top level, so the local scope is the
global scope. I assert what you meant was:

I'm sorry I should have been more clear. I meant it to be
a piece of function code.

c = something
a = something_else

def somefunc():
c.d = a
b = a

(NB : following observations will refer to this code)

>That is what I understand with your statement that [python] always
uses the same search order.

yes.

>My impression was that python will search for c and a in the total current
namespace

what is "the total current namespace" ?

>but will not for b.

b is bound in the local namespace, so there's no need to look for it in
enclosing namespaces.

Now could you clarify please. First you agree with the statement that python
always uses the same search order, then you state here there is no need
to look for b because it is bound to local namespace. That seems to
imply that the search order for b is different.

AFAIR my original statement was that the search for b was different than
the search for a; meaning that the search for b was limited to the local
scope and this could be determined from just viewing a line like "b = a"
within a function. The search for a (or c in a line like: "c.d = a")
is not limited to the local scope.

I may see some interpretation where you may say that the search order
for b is the same as for a and c but I still am not comfortable with
it.

>>>But a variable that is bound
inside the function (with an asignment) and is not declared global, is in
the local namespace.

Aren't we now talking about implementation details?

Certainly not. Namespaces and names lookup rules are fundamental parts
of the Python language.

I don't see the contradiction. That Namespaces and names lookup are
fundamentel parts of the Python language, doesn't mean that
the right behaviour can't be implemented in multiple ways and
doesn't contradict that a specific explanation depend on a specific
implementation instead of just on language definition.

>Sure the compilor
can set things up so that local names are bound to the local scope and
so the same code can be used. But it seems somewhere was made the
decision that b was in the local scope without looking for that b in
the scopes higher up.

binding creates a name in the current namespace. b is bound in the local
namespace, so b is local. period.

I wrote nothing that contradicts that.

--
Antoon Pardon

Jul 6 '06 #14

Terry Reedy

"Antoon Pardon" <ap*****@forel.vub.ac.bewrote in message
news:sl********************@rcpc42.vub.ac.be...

And if Nested variables are harmfull,

I don't know if anyone said that they were, but Guido obviously does not
think so, or he would not have added them. So skip that.

what is then the big difference between rebinding them and mutating them

A variable is a name. Name can be rebound (or maybe not) but they cannot
be mutated. Only objects (with mutation methods) can be mutated. In other
words, binding is a namespace action and mutation is an objectspace action.
In Python, at least, the difference is fundamental.

Or, in other other words, do not be fooled by the convenient but incorrect
abbreviated phrase 'mutate a nested variable'.

that we should forbid the first and allow the second?

Rebinding nested names is not forbidden; it has just not yet been added
(see below).

Being able to mutate a mutable object is automatic once you have a
reference to it. In other words, it you can read the value, you can mutate
it (if it is mutable).

I understand that python evolved and that this sometimes results
in things that in hindsight could have been done better.

So does Guido. That is one explicit reason he gave for not choosing any of
the nunerous proposals for the syntax and semantics of nested scope write
access. In the face of anti-consensus among the developers and no
particular personal preference, he decided, "Better to wait than roll the
dice and make the wrong, hard to reverse, choice now". (Paraphrased quote)

>I have to wonder if someone really thought this through at design time

Giving the actual history of, if anything, too many people thinking too
many different thoughts, this is almost funny.

Recently however, Guido has rejected most proposals to focus attention on
just a few variations and possibly gain a consensus. So I think there is
at least half a chance that some sort of nested scope write access will
appear in 2.6 or 3.0.

Terry Jan Reedy

Jul 7 '06 #15

>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:

>APCould you maybe clarify what problem we are discussing? All I wrote
APwas that with an assignment the search for the lefthand variable
APdepends on whether the lefthand side is a simple variable or
APmore complicated.

What do you mean with `the lefthand variable'? Especially when talking
about `complicated lefthand sides'?

>APSure people may prefer to speak about (re)binding
APvs mutating variables, but just because I didn't use the prefered terms,
APstarting to doubt my understanding of the language, seems a bit
APpremature IMO. I'm sure there are areas where my understanding of
APthe language is shaky, metaclasses being one of them, but understanding
APhow names are searched doesn't seem to be one of them.

You didn't understand it in your OP. Maybe your understanding has gained in
the meantime?
--
Piet van Oostrum <pi**@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email: pi**@vanoostrum.org

Jul 7 '06 #16

>>>>"Terry Reedy" <tj*****@udel.edu(TR) wrote:

>TR"Antoon Pardon" <ap*****@forel.vub.ac.bewrote in message
TRnews:sl********************@rcpc42.vub.ac.be. ..

>>And if Nested variables are harmfull,

>TRI don't know if anyone said that they were, but Guido obviously does not
TRthink so, or he would not have added them. So skip that.

I used that phrase (with correct spelling). I had supposed that it would
be recognised as a variation of 'Global variables considered harmful',
(William Wulf and Mary Shaw, ACM SIGPLAN Notices, 1973, 8 (2) pp. 28--34).
I think nested variables have the same troubles as global variables, so
they should be used with care.
--
Piet van Oostrum <pi**@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email: pi**@vanoostrum.org

Jul 7 '06 #17

Antoon Pardon wrote:

>have any of your "my mental model of how Python works is more important
than how it actually works" ever had a point ?

Be free to correct me. But just suggesting that I'm wrong doesn't help
me in changing my mental model.

over the years, enough people have wasted enough time on trying to get
you to understand how Python works, in various aspects. if you really
were interested in learning, you would have learned something by now,
and you wouldn't keep repeating the same old misunderstandings over and
over again.

</F>

Jul 7 '06 #18

Antoon "I'm no nincompoop, but I play one on the internet" Pardon wrote:

I don't see the contradiction. That Namespaces and names lookup are
fundamentel parts of the Python language, doesn't mean that
the right behaviour can't be implemented in multiple ways and
doesn't contradict that a specific explanation depend on a specific
implementation instead of just on language definition.

the behaviour *is* defined in the language definition, and has nothing
to do with a specific implementation. have you even read the language
reference ? do you even know that it exists ?

</F>

Jul 7 '06 #19

Piet van Oostrum wrote:

There is no big difference I think. Only Python doesn't have syntax for the
former. Older versions of Python didn't even have nested scopes.

arbitrarily nested scopes, at least. the old local/global/builtin
approach (the LGB rule) is of course a kind of nesting; the new thing is
support for "enclosing scopes" in Python 2.1/2.2 (the LEGB rule). for
some background, see:

http://www.python.org/dev/peps/pep-0227/

the section "Rebinding names in enclosing scopes" discusses the 2.X-
specific thinking; this may be revised in 3.0 (see current python-dev
discussions).

</F>

Jul 7 '06 #20

Antoon Pardon wrote:

On 2006-07-06, Bruno Desthuilliers <on***@xiludom.growrote:

>>Antoon Pardon wrote:

>>>On 2006-07-05, Piet van Oostrum <pi**@cs.uu.nlwrote:
>>>It's not about "finding a name/identifier", it's about the difference
>>>between (re)binding a name and mutating an object.

>APThe two don't contradict each other. Python has chosen that it won't
>APrebind variables that are out of the local scope. So if the lefthand
>APside of an assignment is a simple name it will only search in the
>APlocal scope for that name. But if the lefthand side is more complicated
>APif will also search the outerscopes for the name.

Now it's pretty clear you *don't* understand.

In the second case, ie:

k = [0]
def f(i):
k[0] += i

'k[0]' is *not* a name. The name is 'k'. If we rewrite this snippet
without all the syntactic sugar, we get something like:

k = [0]
def f(i):
k.__setitem_(0, k.__getitem__(0) + i)

Now where do you see any rebinding here ?

What point do you want to make? As far as I can see, I
didn't write anything that implied I expected k to
be rebound in code like

Please re-read your own writing above.

>
k[0] += i

So why are you trying so hard to show me this?

I was just expecting to be of any help, but it seems you just *refuse*
to understand.

>>>>No. It will always use the same search order.
So if I understand you correctly in code like:

c.d = a
b = a

All three names

which ones ?

>>>are searched for in all scopes between the local en global
one.

In this example, we're at the top level, so the local scope is the
global scope. I assert what you meant was:

I'm sorry I should have been more clear. I meant it to be
a piece of function code.

>>c = something
a = something_else

def somefunc():
c.d = a
b = a

(NB : following observations will refer to this code)

>>>That is what I understand with your statement that [python] always
uses the same search order.

yes.

>>>My impression was that python will search for c and a in the total current
namespace

what is "the total current namespace" ?

I still wait your explanation on this...

>>>but will not for b.

b is bound in the local namespace, so there's no need to look for it in
enclosing namespaces.

Now could you clarify please. First you agree with the statement that python
always uses the same search order,

Yes : local namespace, then enclosing namespaces.

then you state here there is no need
to look for b because it is bound to local namespace.

Yes. b being bound in the local namespace, it's found in the local
namespace, so lookup stops here. Pretty simple.

That seems to
imply that the search order for b is different.

cf above.

AFAIR my original statement was that the search for b was different than
the search for a;

And it's plain wrong, as anyone taking a few minutes reading the doc and
doing some tests would know.

meaning that the search for b was limited to the local
scope and this could be determined from just viewing a line like "b = a"
within a function. The search for a (or c in a line like: "c.d = a")
is not limited to the local scope.

Please repeat after me :
1/ binding in the local namespace makes the name local
2/ search order is local namespace first, then enclosing namespaces.

I may see some interpretation where you may say that the search order
for b is the same as for a and c

There's nothing to "interpret" here.

but I still am not comfortable with
it.

Too bad for you. But whether you are "comfortable" with reality is none
of my concern.

>

>>>>But a variable that is bound
inside the function (with an asignment) and is not declared global, is in
the local namespace.
Aren't we now talking about implementation details?

Certainly not. Namespaces and names lookup rules are fundamental parts
of the Python language.

I don't see the contradiction.

So go and get yourself some glasses.

That Namespaces and names lookup are
fundamentel parts of the Python language, doesn't mean that
the right behaviour

define "right behaviour" ?

can't be implemented in multiple ways

I don't give a damn about how it's implemented.

and
doesn't contradict that a specific explanation depend on a specific
implementation instead of just on language definition.

This is totally meaningless.

>>>Sure the compilor
can set things up so that local names are bound to the local scope and
so the same code can be used. But it seems somewhere was made the
decision that b was in the local scope without looking for that b in
the scopes higher up.

binding creates a name in the current namespace. b is bound in the local
namespace, so b is local. period.

I wrote nothing that contradicts that.

I give up. You're a crank.
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"

Jul 7 '06 #21

Piet van Oostrum wrote:
(snip)

There is no big difference I think. Only Python doesn't have syntax for the
former. Older versions of Python didn't even have nested scopes. maybe it
was a mistake to add them.

Certainly not. Nested scopes allow closures, which allow decorators and
lot of *very* useful things. Remove this from Python, and you'll see a
*lot* of experimented programmers switch to another language.
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"

Jul 7 '06 #22

Antoon Pardon wrote:

On 2006-07-06, Piet van Oostrum <pi**@cs.uu.nlwrote:

>>>APAren't we now talking about implementation details? Sure the compilor
APcan set things up so that local names are bound to the local scope and
APso the same code can be used. But it seems somewhere was made the
APdecision that b was in the local scope without looking for that b in
APthe scopes higher up.

Yes, as I (and others) have already said several times: an assignment to a
variable inside a function body (but not an assignment to an attribute or
part of an object) without a global declaration makes that variable a local
variable. That is not an implementation detail; it is part of the language definition.

You seem to think I didn't understand this.

And he's right, cf below.

(snip)

Could you maybe clarify what problem we are discussing? All I wrote
was that with an assignment the search for the lefthand variable
depends on whether the lefthand side is a simple variable or
more complicated.

You're obviously clueless. Which would not be a problem if you did not
refuse to first aknowledge the fact then take appropriate actions.

Sure people may prefer to speak about (re)binding
vs mutating variables, but just because I didn't use the prefered terms,

If you refuse to understand that there are pretty good reasons to use
the appropriate semantic, that's your problem, but then no one can help
you.

starting to doubt my understanding of the language, seems a bit
premature IMO.

I do not 'doubt', I'm 111% confident.

I'm sure there are areas where my understanding of
the language is shaky, metaclasses being one of them, but understanding
how names are searched doesn't seem to be one of them.

It is, obviously.

And you're definitively a crank.

--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"

Jul 7 '06 #23

Bruno Desthuilliers wrote:

Certainly not. Nested scopes allow closures, which allow decorators and
lot of *very* useful things.

decorators can be trivially implemented as classes, of course. it's a
bit unfortunate that many people seem to think that decorators *have* to
be implemented as nested functions, rather than arbitrary callables.

</F>

Jul 7 '06 #24

On 2006-07-07, Piet van Oostrum <pi**@cs.uu.nlwrote:

>>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:

>>APCould you maybe clarify what problem we are discussing? All I wrote
APwas that with an assignment the search for the lefthand variable
APdepends on whether the lefthand side is a simple variable or
APmore complicated.

What do you mean with `the lefthand variable'? Especially when talking
about `complicated lefthand sides'?

The name on the left side of an assignment that refers to a variable,
as opposed to names that are attributes.

>>APSure people may prefer to speak about (re)binding
APvs mutating variables, but just because I didn't use the prefered terms,
APstarting to doubt my understanding of the language, seems a bit
APpremature IMO. I'm sure there are areas where my understanding of
APthe language is shaky, metaclasses being one of them, but understanding
APhow names are searched doesn't seem to be one of them.

You didn't understand it in your OP. Maybe your understanding has gained in
the meantime?

I don't rule out that I gained some understanding without noticing it.
Maybe my choice of words was poor then.

--
Antoon Pardon

Jul 7 '06 #25

On 2006-07-07, Fredrik Lundh <fr*****@pythonware.comwrote:

Antoon "I'm no nincompoop, but I play one on the internet" Pardon wrote:

>I don't see the contradiction. That Namespaces and names lookup are
fundamentel parts of the Python language, doesn't mean that
the right behaviour can't be implemented in multiple ways and
doesn't contradict that a specific explanation depend on a specific
implementation instead of just on language definition.

the behaviour *is* defined in the language definition, and has nothing
to do with a specific implementation.

As far as I can see I didn't write anything that contradicts this.
It is possible that at some point my choice of wording was bad
and that I gave the impression that i somehow wanted to contradict
this. If that happened my appologies.

have you even read the language
reference ? do you even know that it exists ?

You mean this, I suppose:

http://docs.python.org/ref/naming.html

--
Antoon Pardon

Jul 7 '06 #26

On 2006-07-07, Fredrik Lundh <fr*****@pythonware.comwrote:

Antoon Pardon wrote:

>>have any of your "my mental model of how Python works is more important
than how it actually works" ever had a point ?

Be free to correct me. But just suggesting that I'm wrong doesn't help
me in changing my mental model.

over the years, enough people have wasted enough time on trying to get
you to understand how Python works, in various aspects. if you really
were interested in learning, you would have learned something by now,
and you wouldn't keep repeating the same old misunderstandings over and
over again.

May be I misunderstand, maybe I sometimes have difficulties making my self
clear. If you already made up your mind which is it, that is fine by me.
I just don't see the point of just posting a response that just
boils down to: You are wrong. Even if you have given up on me, others
might be helped if you took the trouble of explainig what was wrong.

Well, that was just what I was thinking.

--
Antoon Pardon

Jul 7 '06 #27

Fredrik Lundh wrote:

Bruno Desthuilliers wrote:

>Certainly not. Nested scopes allow closures, which allow decorators and
lot of *very* useful things.

decorators can be trivially implemented as classes, of course. it's a
bit unfortunate that many people seem to think that decorators *have* to
be implemented as nested functions, rather than arbitrary callables.

Your of course right - and I should know better since AFAIK (IOW please
correct me if I'm wrong), closures and classes are somewhat
interchangeable.

OTHO, using closures can make things far more simple - just like having
functions being objects is not absolutely necessary for having HOF-like
features, but can make HOF much more simple. If you take back all these
kind of features from Python, you end up with something that's not
really better than Java - and then me run away screaming !-)

Thanks for the correction anyway.
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom.gro'.split('@')])"

Jul 7 '06 #28

Terry Reedy

"Antoon Pardon" <ap*****@forel.vub.ac.bewrote in message
news:sl********************@rcpc42.vub.ac.be...

others might be helped if you took the trouble of explaining
what was wrong.

Aside from F., I tried to explain what I think you said wrong. Did you
read it? Did it help any?

tjr

Jul 7 '06 #29

>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:

>APOn 2006-07-07, Piet van Oostrum <pi**@cs.uu.nlwrote:

>>>>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:

APCould you maybe clarify what problem we are discussing? All I wrote
APwas that with an assignment the search for the lefthand variable
APdepends on whether the lefthand side is a simple variable or
APmore complicated.

>>>
What do you mean with `the lefthand variable'? Especially when talking
about `complicated lefthand sides'?

>APThe name on the left side of an assignment that refers to a variable,
APas opposed to names that are attributes.

So let me get it clear:
In a.b = c, a is the lefthand variable, but b is not?

If that is what you mean then I interpret your statement

>AP`with an assignment the search for the lefthand variable
APdepends on whether the lefthand side is a simple variable or
APmore complicated'

as meaning that the search for `a' in `a.b = c' would be different than the
search for `a' in `a = b'. Well, it is not. But I can understand the
confusion. Namely, `a = b' introduces a binding for `a' in the local scope,
unless `a' was declared global. So the search will find `a' in the local
scope and it stops there. On the other hand `a.b = c' will not introduce a
binding for `a'. So the search for `a' may stop in the local space (if
there was another binding for `a' in the local scope) or it may need to
continue to outer scopes. The difference, however is not the
complicatedness of the lefthand side but whether the local scope contains a
binding for the variable.

--
Piet van Oostrum <pi**@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email: pi**@vanoostrum.org

Jul 7 '06 #30

On 2006-07-07, Terry Reedy <tj*****@udel.eduwrote:

>
"Antoon Pardon" <ap*****@forel.vub.ac.bewrote in message
news:sl********************@rcpc42.vub.ac.be...
>And if Nested variables are harmfull,

I don't know if anyone said that they were, but Guido obviously does not
think so, or he would not have added them. So skip that.

>what is then the big difference between rebinding them and mutating them

A variable is a name. Name can be rebound (or maybe not) but they cannot
be mutated. Only objects (with mutation methods) can be mutated. In other
words, binding is a namespace action and mutation is an objectspace action.
In Python, at least, the difference is fundamental.

Or, in other other words, do not be fooled by the convenient but incorrect
abbreviated phrase 'mutate a nested variable'.

I'm not fooled by that phrase. I just think the mutate vs rebind
explanation is not complete.

If we have two statements "a = b" and "c.d = b" the fact that a is being
rebound while c is mutated doesn't explain why we allow c to be searched
out of the local scope.

By only stating that the first statement is a rebinding and the second
is a mutation and that this is a fundamental difference in python you
seem to suggest that this fundamental differenence implies this
difference in searching scopes. Python could have made the choice
that in an assignment the variable on the left side was always to
be searched in local space so that code like the following would
throw: UnboundLocalError: local variable 'c' referenced before assignment

c = SomeObject
def f():
c.a = 5

Now I'm not arguing python should have made this choice. But the
possibility shows IMO this has more to do with search policies
of names than with the distinction between a rebinding and a mutation.

AFAIK when nested scopes where introduced everyone agreed that scopes
had to have access to outer scopes. There were voices that supported
allowing a rebinding in an outer scope but no consensus on how to
do this was reached, so this possibility was dropped. So we can't
rebind an outer scope variable but we can mutate such a variable
because for mutation we only need access.

>I understand that python evolved and that this sometimes results
in things that in hindsight could have been done better.

So does Guido. That is one explicit reason he gave for not choosing any of
the nunerous proposals for the syntax and semantics of nested scope write
access. In the face of anti-consensus among the developers and no
particular personal preference, he decided, "Better to wait than roll the
dice and make the wrong, hard to reverse, choice now". (Paraphrased quote)

>>I have to wonder if someone really thought this through at design time

Giving the actual history of, if anything, too many people thinking too
many different thoughts, this is almost funny.

Maybe I didn't made myself clear enough, but I never meant to imply
people hadn't thought thouroughly about this. If I gave you this
impression I appologize. What I was wondering about was that those
that had thought about it, would have reached a certain conclusion
that seemed suggested.

Recently however, Guido has rejected most proposals to focus attention on
just a few variations and possibly gain a consensus. So I think there is
at least half a chance that some sort of nested scope write access will
appear in 2.6 or 3.0.

Well I have browsed the discussion, which is why I react so lately to
this, and there is one thing I wonder about. As far as I can see no
suggestion removes the difference in the default search. The following
code will still work and won't need an outer statement. (or global,
nonlocal or whatever it will be)

c = SomeObject
def f():
c.a = 5

I don say they have to change this, but since it seemed decided this
was a python3k thing, i think the question deserved to be raised.

But I'm glad with this turn of events anyhow.

Just my 2 cent.

--
Antoon Pardon

Jul 8 '06 #31

On 2006-07-07, Piet van Oostrum <pi**@cs.uu.nlwrote:

>>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:

>>APOn 2006-07-07, Piet van Oostrum <pi**@cs.uu.nlwrote:
>>>>>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:

APCould you maybe clarify what problem we are discussing? All I wrote
APwas that with an assignment the search for the lefthand variable
APdepends on whether the lefthand side is a simple variable or
APmore complicated.
>>>>
What do you mean with `the lefthand variable'? Especially when talking
about `complicated lefthand sides'?

>>APThe name on the left side of an assignment that refers to a variable,
APas opposed to names that are attributes.

So let me get it clear:
In a.b = c, a is the lefthand variable, but b is not?

Yes, b is an attribute of a

If that is what you mean then I interpret your statement
>>AP`with an assignment the search for the lefthand variable
APdepends on whether the lefthand side is a simple variable or
APmore complicated'

as meaning that the search for `a' in `a.b = c' would be different than the
search for `a' in `a = b'.

It is conceptually different. In the line 'a = b' you don't need to
search for the scope of a. You know it is the current scope, if you
want to know the scope of b on the other hand, you need to search
for statement where it is assigned to.

Sure you can set things up in the interpreter so that the same search
routine is used, but that is IMO an implementation detail.

Well, it is not. But I can understand the
confusion. Namely, `a = b' introduces a binding for `a' in the local scope,
unless `a' was declared global. So the search will find `a' in the local
scope and it stops there. On the other hand `a.b = c' will not introduce a
binding for `a'. So the search for `a' may stop in the local space (if
there was another binding for `a' in the local scope) or it may need to
continue to outer scopes. The difference, however is not the
complicatedness of the lefthand side but whether the local scope contains a
binding for the variable.

The complicatedness of the lefthand side, decided on whether the
variable was introduced in the local scope or not during startup
time. So that complicatedness decided whether the search was
to stop at the local level or not.

--
Antoon Pardon

Jul 8 '06 #32

>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:

>APIt is conceptually different. In the line 'a = b' you don't need to
APsearch for the scope of a. You know it is the current scope, if you

Except when it has been declared global.

>APwant to know the scope of b on the other hand, you need to search
APfor statement where it is assigned to.

>APSure you can set things up in the interpreter so that the same search
AProutine is used, but that is IMO an implementation detail.

>>Well, it is not. But I can understand the
confusion. Namely, `a = b' introduces a binding for `a' in the local scope,
unless `a' was declared global. So the search will find `a' in the local
scope and it stops there. On the other hand `a.b = c' will not introduce a
binding for `a'. So the search for `a' may stop in the local space (if
there was another binding for `a' in the local scope) or it may need to
continue to outer scopes. The difference, however is not the
complicatedness of the lefthand side but whether the local scope contains a
binding for the variable.

>APThe complicatedness of the lefthand side, decided on whether the
APvariable was introduced in the local scope or not during startup
APtime. So that complicatedness decided whether the search was
APto stop at the local level or not.

No, it doesn't. There could be another binding in the same scope. The
complicatedness of this particular assignment doesn't decide anything about
how to search for 'a', but rather the presence or absence of a binding
anywhere in the scope.
--
Piet van Oostrum <pi**@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email: pi**@vanoostrum.org

Jul 9 '06 #33

On 2006-07-09, Piet van Oostrum <pi**@cs.uu.nlwrote:

>>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:

>>APIt is conceptually different. In the line 'a = b' you don't need to
APsearch for the scope of a. You know it is the current scope, if you

Except when it has been declared global.

Yes, I ignored that possibilitym because as far as I understood we
were discussing variables in intermediate scopes.

>>APwant to know the scope of b on the other hand, you need to search
APfor statement where it is assigned to.

>>APSure you can set things up in the interpreter so that the same search
AProutine is used, but that is IMO an implementation detail.

>>>Well, it is not. But I can understand the
confusion. Namely, `a = b' introduces a binding for `a' in the local scope,
unless `a' was declared global. So the search will find `a' in the local
scope and it stops there. On the other hand `a.b = c' will not introduce a
binding for `a'. So the search for `a' may stop in the local space (if
there was another binding for `a' in the local scope) or it may need to
continue to outer scopes. The difference, however is not the
complicatedness of the lefthand side but whether the local scope contains a
binding for the variable.

>>APThe complicatedness of the lefthand side, decided on whether the
APvariable was introduced in the local scope or not during startup
APtime. So that complicatedness decided whether the search was
APto stop at the local level or not.

No, it doesn't. There could be another binding in the same scope.

Indeed there could be. But I hoped you would understand I was just
keeping things simple, with a simple example.

The
complicatedness of this particular assignment doesn't decide anything about
how to search for 'a', but rather the presence or absence of a binding
anywhere in the scope.

I'll word it differently. If the compiler encounters a line like 'a = b'
then you know from that line alone that the search space for a will be
limited to the local scope. If you encounter a line like 'a.c = b' then
you have no such knowledge. A line like 'a = b' will cause the compilor
that at call time variable a will be added to the local scope, a line like
'a.c = b' will not have that effect. So a line like 'a = b' has an
influence on what the search space is for variable a, while a line
like 'a.c = b' doesn't. So the complicatedness on the leftside decides
whether or not the compilor will take certain actions with regards to
the search space of the variable on the left side.

And yes I'm again ignoring global.

--
Antoon Pardon

Jul 9 '06 #34

On 2006-07-08, Dennis Lee Bieber <wl*****@ix.netcom.comwrote:

On 8 Jul 2006 18:52:56 GMT, Antoon Pardon <ap*****@forel.vub.ac.be>
declaimed the following in comp.lang.python:

>>
I'm not fooled by that phrase. I just think the mutate vs rebind
explanation is not complete.

If we have two statements "a = b" and "c.d = b" the fact that a is being
rebound while c is mutated doesn't explain why we allow c to be searched
out of the local scope.

The "search" is not different, per se... It is only after the object
is found that the difference becomes apparent -- a rebinding changes an
object's ID, and is not permitted for a non-local UNLESS a "global"
statement appears before any usage of the name.

"c.d =..." has absolutely no effect on the ID of object C; in that
aspect it is a read-only look-up of "c". IOWs, the same look-up as would
be used if "c" were on the RHS of the statement.

>be searched in local space so that code like the following would
throw: UnboundLocalError: local variable 'c' referenced before assignment

c = SomeObject
def f():
c.a = 5

What would you say the behavior should be for:

c.a = c.b
vs
la = c.b

The look-up of "c" is the same on both sides of the statement; local
(not found) then global (found), THEN the operation is applied. On both
sides "c" is a read-only look-up (that is, no changes to the ID of "c"
-- no rebinding -- take place).

Are you suggesting we need to use "global c" in order to have "c.b"
on the RHS? By that logic, we would also need "global sys" to reference
"sys.argv" inside a function definition. Remember -- the modules loaded
by "import" are just more "SomeObject" samples...

I think you are misunderstanding what I was getting at. This example was
not meant to illustrate how I think python should behave. I thought I
had made that clear.

When someone gets confused over the difference between rebinding or mutating
a variable on an intermediate scope, the explanation he mostly seems to get
boils down to: one is rebinding, the other is mutation, this is a fundametal
difference in python.

My impression is that they seem to say that the fundamental difference
between mutation and rebinding implies the specific behaviour python
has now. IMO this explanation is incomplete. The python developers
could have chosen that a line like 'c.a = ...' would have resulted
in c being included in the local scope. Then rebinding and mutation
would still be fundamentally different from each other but the specific
confusion over why 'k[0] = ...' worked as expeced but 'k = ...' didn't,
will disappear.

So I only used this example as an illustration why I think the usual
explanation is not completed. This example is not to be taken as an
illustration of how I think python should behave.

--
Antoon Pardon

Jul 9 '06 #35

>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:

>APWhen someone gets confused over the difference between rebinding or
APmutating a variable on an intermediate scope, the explanation he
APmostly seems to get boils down to: one is rebinding, the other is
APmutation, this is a fundametal difference in python.

>APMy impression is that they seem to say that the fundamental difference
APbetween mutation and rebinding implies the specific behaviour python
APhas now. IMO this explanation is incomplete. The python developers
APcould have chosen that a line like 'c.a = ...' would have resulted
APin c being included in the local scope. Then rebinding and mutation
APwould still be fundamentally different from each other but the specific
APconfusion over why 'k[0] = ...' worked as expeced but 'k = ...' didn't,
APwill disappear.

That seems nonsense to me. If they had chosen that 'c.a = ...' would imply
that c would become a local variable, what would the value of c have to be
then, if there was no prior direct assignment to c? Would it become a new
binding like in 'c = ...'? From what class should it become an instance? Or
would it become a copy of the value that 'c' has in an outer scope? I don't
know any programming language where an assignment to c.a does create a new
c, rather than modifying the existing value of c. That would have been a
very strange language design. Similarly for 'k[0] = ...'. What would happen
with the other elements of k?

There are no situations in Python where an assignment to c.a or c[0]
suddenly lets spring c into existence. You always need an already existing
binding for c for this to be valid. And it always uses that binding, and
doesn't move or copy it to a different block.

Some of the confusing originates from the fact that assignment in Python is
subtly different from assignment in other programming languages. In most
languages variables denote memory locations or collections of memory
locations. An assignment then means changing the contents of those memory
locations. In python variables are bound to values, and assignment means
(re)binding the name to a possibly different value. With an example: most
other languages have boxes with the name of the variable on them. An
assignment changes the contents of the box. In Python you have values
(objects) and an assignment means sticking a label with the name of the
variable on the object. Often the difference is unnoticable, but there are
subtle cases where this really makes a difference.

When the lefthandside of an assignment is an object attribute, subscription
or slice then the assignment is syntactic sugar for a mutation operation
implemented by the object, which usually changes the value of the object
but could do something completely different.

(Finally, the lefthandside can also be a [nested] tuple or list, in which
case it is a collection of parallel assignments. And oh yes, there are also
other binding operations e.g. a function or class definition.)
--
Piet van Oostrum <pi**@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email: pi**@vanoostrum.org

Jul 9 '06 #36

This is probably my last response to you in this thread. My impression
is that for the moment nothing productive can come from this exchange.
I have the feeling that you are not reading so much with the interntion
of understanding what I want to say, but with the intention of
confirming your suspition that I just don't have a clue.

It seems this is turing into some competition where I have
somehow to defend my understanding of python an you trying to
show how little I really understand. Since I don't feel the
need to prove myself here, I will simply bow out.

On 2006-07-09, Piet van Oostrum <pi**@cs.uu.nlwrote:

>>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:

>>APWhen someone gets confused over the difference between rebinding or
APmutating a variable on an intermediate scope, the explanation he
APmostly seems to get boils down to: one is rebinding, the other is
APmutation, this is a fundametal difference in python.

>>APMy impression is that they seem to say that the fundamental difference
APbetween mutation and rebinding implies the specific behaviour python
APhas now. IMO this explanation is incomplete. The python developers
APcould have chosen that a line like 'c.a = ...' would have resulted
APin c being included in the local scope. Then rebinding and mutation
APwould still be fundamentally different from each other but the specific
APconfusion over why 'k[0] = ...' worked as expeced but 'k = ...' didn't,
APwill disappear.

That seems nonsense to me. If they had chosen that 'c.a = ...' would imply
that c would become a local variable, what would the value of c have to be
then, if there was no prior direct assignment to c?

I'm sorry to see you missed it, but since I had answered this already in
this thread I saw at the moment no need to repeat it: There would be no
value for c, the line would raise an UnboundLocalError.

I also don't understand why you take the trouble of attacking this
possibility. It's wasn't presented as a suggestion for changing python.
It was used as an illustration of why I think some explanation needs
to be worked out more. So even if this turns out to be the worst
possible that could ever happen to python, unless you think people
needing the original explanation will grasp the implication of this
possibility immediately, the point I wanted to illustrate seems to
stand.

--
Antoon Pardon

Jul 10 '06 #37