473,738 Members | 6,332 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Nested scopes, and augmented assignment

Hi,

The following might be documented somewhere, but it hit me unexpectedly
and I couldn't exactly find this in the manual either.

Problem is, that I cannot use augmented assignment operators in a
nested scope, on variables from the outer scope:

PythonWin 2.4.3 (#69, Mar 29 2006, 17:35:34) [MSC v.1310 32 bit
(Intel)] on win32.
Portions Copyright 1994-2004 Mark Hammond (mh******@skipp inet.com.au) -
see 'Help/About PythonWin' for further copyright information.
>>def foo():
.... def nestedfunc(bar) :
.... print bar, ';', localvar
.... localvar += 1
.... localvar=0
.... nestedfunc('bar ')
....
>>foo()
bar =Traceback (most recent call last):
File "<interacti ve input>", line 1, in ?
File "<interacti ve input>", line 6, in foo
File "<interacti ve input>", line 3, in nestedfunc
UnboundLocalErr or: local variable 'localvar' referenced before
assignment
>>>
This is entirely counter-intuitive to me, and searching the manual for
nested scoping rules and for augmented assignment rules, I still feel
that I couldn't have predicted this from the docs.

Is this an implementation artifact, bug, or should it really just
follow logically from the language definition?

Regards,

--Tim

Jul 4 '06 #1
37 2785
Tim N. van der Leeuw wrote:
Hi,

The following might be documented somewhere, but it hit me unexpectedly
and I couldn't exactly find this in the manual either.

Problem is, that I cannot use augmented assignment operators in a
nested scope, on variables from the outer scope:
<snip/>
Is this an implementation artifact, bug, or should it really just
follow logically from the language definition?
From the docs:

"""
An augmented assignment expression like x += 1 can be rewritten as x = x + 1
to achieve a similar, but not exactly equal effect. In the augmented
version, x is only evaluated once. Also, when possible, the actual
operation is performed in-place, meaning that rather than creating a new
object and assigning that to the target, the old object is modified
instead.
"""

The first part is the important one. If you expand

x += 1

to

x = x + 1

it becomes clear under the python scoping rules that x is being treated as a
local to the inner scope.

There has been a discussion about this recently on python-dev[1] and this
NG - google for it.

Regards,

Diez
[1] http://mail.python.org/pipermail/pyt...ne/065902.html
Jul 4 '06 #2
On 2006-07-04, Diez B. Roggisch <de***@nospam.w eb.dewrote:
Tim N. van der Leeuw wrote:
>Hi,

The following might be documented somewhere, but it hit me unexpectedly
and I couldn't exactly find this in the manual either.

Problem is, that I cannot use augmented assignment operators in a
nested scope, on variables from the outer scope:

<snip/>
>Is this an implementation artifact, bug, or should it really just
follow logically from the language definition?

From the docs:

"""
An augmented assignment expression like x += 1 can be rewritten as x = x + 1
to achieve a similar, but not exactly equal effect. In the augmented
version, x is only evaluated once. Also, when possible, the actual
operation is performed in-place, meaning that rather than creating a new
object and assigning that to the target, the old object is modified
instead.
"""

The first part is the important one. If you expand

x += 1

to

x = x + 1

it becomes clear under the python scoping rules that x is being treated as a
local to the inner scope.

There has been a discussion about this recently on python-dev[1] and this
NG - google for it.
Well no matter what explanation you give to it, and I understand how it
works, I keep finding it strange that something like

k = [0]
def f(i):
k[0] += i
f(2)

works but the following doesn't

k = 0
def f(i):
k += i
f(2)
Personnaly I see no reason why finding a name/identifier on the
leftside of an assignment should depend on whether the name
is the target or prefix for the target.
But maybe that is just me.

--
Antoon Pardon
Jul 5 '06 #3
Antoon Pardon wrote:
(snip)
Well no matter what explanation you give to it, and I understand how it
works,
I'm not sure of this.
I keep finding it strange that something like

k = [0]
def f(i):
k[0] += i
f(2)

works but the following doesn't

k = 0
def f(i):
k += i
f(2)
Personnaly I see no reason why finding a name/identifier on the
leftside of an assignment should depend on whether the name
is the target or prefix for the target.
It's not about "finding a name/identifier", it's about the difference
between (re)binding a name and mutating an object.
--
bruno desthuilliers
python -c "print '@'.join(['.'.join([w[::-1] for w in p.split('.')]) for
p in 'o****@xiludom. gro'.split('@')])"
Jul 5 '06 #4
Bruno Desthuilliers wrote:
It's not about "finding a name/identifier", it's about the difference
between (re)binding a name and mutating an object.
the difference between binding and performing an operation on an object
(mutating or not), in fact.

this is Python 101.

</F>

Jul 5 '06 #5
On 2006-07-05, Bruno Desthuilliers <on***@xiludom. growrote:
Antoon Pardon wrote:
(snip)
>Well no matter what explanation you give to it, and I understand how it
works,

I'm not sure of this.
Should I care about that?
>I keep finding it strange that something like

k = [0]
def f(i):
k[0] += i
f(2)

works but the following doesn't

k = 0
def f(i):
k += i
f(2)
Personnaly I see no reason why finding a name/identifier on the
leftside of an assignment should depend on whether the name
is the target or prefix for the target.

It's not about "finding a name/identifier", it's about the difference
between (re)binding a name and mutating an object.
The two don't contradict each other. Python has chosen that it won't
rebind variables that are out of the local scope. So if the lefthand
side of an assignment is a simple name it will only search in the
local scope for that name. But if the lefthand side is more complicated
if will also search the outerscopes for the name.

Python could have chosen an approach with a "nested" keyword, to allow
rebinding a name in an intermediate scope. It is not that big a deal
that it hasn't, but I keep finding the result strange and somewhat
counterintuitiv e.

Let me explain why:

Suppose someone is rather new of python and writes the following
code, manipulating vectors:

A = 10 * [0]
def f(B):
...
for i in xrange(10):
A[i] += B[i]
...

Then he hears about the vector and matrix modules that are around.
So he rewrites his code naively as follows:

A = NullVector(10):
def f(B):
...
A += B
...

And it won't work. IMO the principle of least surprise here would
be that it should work.

--
Antoon Pardon
Jul 5 '06 #6
Antoon Pardon wrote:
Python could have chosen an approach with a "nested" keyword
sure, and Python could also have been invented by aliens, powered by
space potatoes, and been illegal to inhale in Belgium.

have any of your "my mental model of how Python works is more important
than how it actually works" ever had a point ?

</F>

Jul 5 '06 #7
>>>>Antoon Pardon <ap*****@forel. vub.ac.be(AP) wrote:
>APOn 2006-07-05, Bruno Desthuilliers <on***@xiludom. growrote:
>>Antoon Pardon wrote:
(snip)
Well no matter what explanation you give to it, and I understand how it
works,

I'm not sure of this.
>APShould I care about that?
Yes, because as long as you don't understand it, you are in for unpleasant
surprises.
>>It's not about "finding a name/identifier", it's about the difference
between (re)binding a name and mutating an object.
>APThe two don't contradict each other. Python has chosen that it won't
APrebind variables that are out of the local scope. So if the lefthand
APside of an assignment is a simple name it will only search in the
APlocal scope for that name. But if the lefthand side is more complicated
APif will also search the outerscopes for the name.
No. It will always use the same search order. But a variable that is bound
inside the function (with an asignment) and is not declared global, is in
the local namespace. A variable that is not assigned to inside the function
is not in the local namespace. An assignment to A[i] is not rebinding A, so
it doesn't count to make the variable A local.
>APPython could have chosen an approach with a "nested" keyword, to allow
APrebinding a name in an intermediate scope. It is not that big a deal
APthat it hasn't, but I keep finding the result strange and somewhat
APcounterintui tive.
Maybe it would have been nice if variables could have been declared as
nested, but I think it shows that nested variables have to be used with
care, similar to globals. Especially not allowing rebinding in intermediate
scopes is a sound principle (`Nested variables considered harmful').
If you need to modify the objects which are bound to names in intermediate
scopes, use methods and give these objects as parameters.
>APLet me explain why:
>APSuppose someone is rather new of python and writes the following
APcode, manipulating vectors:
>AP A = 10 * [0]
AP def f(B):
AP ...
AP for i in xrange(10):
AP A[i] += B[i]
AP ...
>APThen he hears about the vector and matrix modules that are around.
APSo he rewrites his code naively as follows:
>AP A = NullVector(10):
AP def f(B):
AP ...
AP A += B
AP ...
>APAnd it won't work. IMO the principle of least surprise here would
APbe that it should work.
Well, A = f(B) introduces a local variable A. Therefore also A = A+B.
And therefore also A += B.

The only way to have no surprises at all would be if local variables would
have to be declared explicitly, like 'local A'. But that would break
existing code. Or maybe the composite assignment operators should have been
exempted. Too late now!

You have the same problem with:

A = [10]
def inner():
A.append(2)

works but

A = [10]
def inner():
A += [2]

doesn't. The nasty thing in this example is that although A += [2] looks
like a rebinding syntactically, semantically there is no rebinding done.

I think the cleaner solution is (use parameters and don't rebind):
def inner(A):
A.append(2)

--
Piet van Oostrum <pi**@cs.uu.n l>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C 4]
Private email: pi**@vanoostrum .org
Jul 5 '06 #8
On 2006-07-05, Fredrik Lundh <fr*****@python ware.comwrote:
Antoon Pardon wrote:
>Python could have chosen an approach with a "nested" keyword

sure, and Python could also have been invented by aliens, powered by
space potatoes, and been illegal to inhale in Belgium.
At one time one could have reacted the same when people suggested
python could use a ternary operator. In the mean time a ternary
operator is in the pipeline. If you don't want to discuss how
python could be different with me, that is fine, but I do no
harm discussing such things with others.
have any of your "my mental model of how Python works is more important
than how it actually works" ever had a point ?
Be free to correct me. But just suggesting that I'm wrong doesn't help
me in changing my mental model.

--
Antoon Pardon
Jul 6 '06 #9
On 2006-07-05, Piet van Oostrum <pi**@cs.uu.nlw rote:
>>>>>Antoon Pardon <ap*****@forel. vub.ac.be(AP) wrote:
>>APOn 2006-07-05, Bruno Desthuilliers <on***@xiludom. growrote:
>>>Antoon Pardon wrote:
(snip)
Well no matter what explanation you give to it, and I understand how it
works,

I'm not sure of this.
>>APShould I care about that?

Yes, because as long as you don't understand it, you are in for unpleasant
surprises.
Well if someone explains what is wrong about my understanding, I
certainly care about that (although I confess to sometimes being
impatient) but someone just stating he is not sure I understand?
>>>It's not about "finding a name/identifier", it's about the difference
between (re)binding a name and mutating an object.
>>APThe two don't contradict each other. Python has chosen that it won't
APrebind variables that are out of the local scope. So if the lefthand
APside of an assignment is a simple name it will only search in the
APlocal scope for that name. But if the lefthand side is more complicated
APif will also search the outerscopes for the name.

No. It will always use the same search order.
So if I understand you correctly in code like:

c.d = a
b = a

All three names are searched for in all scopes between the local en global
one. That is what I understand with your statement that [python] always
uses the same search order.

My impression was that python will search for c and a in the total current
namespace but will not for b.
But a variable that is bound
inside the function (with an asignment) and is not declared global, is in
the local namespace.
Aren't we now talking about implementation details? Sure the compilor
can set things up so that local names are bound to the local scope and
so the same code can be used. But it seems somewhere was made the
decision that b was in the local scope without looking for that b in
the scopes higher up.

Let me explain a bit more. Suppose I'm writing a python interpreter
in python. One implemantation detail is that I have a list of active
scopes which are directories which map names to objects. At the
start of a new function the scope list is adapted and all local
variables are inserted to the new activated scope and mapped to
some "Illegal Value" object. Now I also have a SearchName function
that will start at the begin of a scope list and return the
first scope in which that name exists. The [0] element is the
local scope. Now we come to the line "b = a"

This could be then executed internally as follows:

LeftScope = SearchName("b", ScopeList)
RightScope = SearchName("a", ScopeList)
LeftScope["b"] = RightScope["a"]

But I don't have to do it this way. I already know in which scope
"b" is, the local one, which has index 0. So I could just as well
have that line exucuted as follows:

LeftScope = ScopeList[0]
RightScope = SearchName("a", ScopeList)
LeftScope["b"] = RightScope["a"]

As far as I understand both "implementation s" would make for
a correct execution of the line "b = a" and because of the
second possibility, b is IMO not conceptually searched for in
the same way as a is searched for, although one could organise
things that the same code is used for both.

Of course it is possible I completely misunderstood how python
is supposed to work and the above is nonesense in which case
I would appreciate it if you correct me.
>>APPython could have chosen an approach with a "nested" keyword, to allow
APrebinding a name in an intermediate scope. It is not that big a deal
APthat it hasn't, but I keep finding the result strange and somewhat
APcounterintu itive.

Maybe it would have been nice if variables could have been declared as
nested, but I think it shows that nested variables have to be used with
care, similar to globals. Especially not allowing rebinding in intermediate
scopes is a sound principle (`Nested variables considered harmful').
If you need to modify the objects which are bound to names in intermediate
scopes, use methods and give these objects as parameters.
But shouldn't we just do programming in general with care? And if
Nested variables are harmfull, what is then the big difference
between rebinding them and mutating them that we should forbid
the first and allow the second?

I understand that python evolved and that this sometimes results
in things that in hindsight could have been done better. But
I sometimes have the impression that the defenders try to defend
those results as a design decision. With your remark above I have
to wonder if someone really thought this through at design time
and came to the conclusion that nested variables are harmfull
and thus may not be rebound but not that harmfull so mutation
is allowed and if so how he came to that conclusion.

>>APLet me explain why:
>>APSuppose someone is rather new of python and writes the following
APcode, manipulating vectors:
>>AP A = 10 * [0]
AP def f(B):
AP ...
AP for i in xrange(10):
AP A[i] += B[i]
AP ...
>>APThen he hears about the vector and matrix modules that are around.
APSo he rewrites his code naively as follows:
>>AP A = NullVector(10):
AP def f(B):
AP ...
AP A += B
AP ...
>>APAnd it won't work. IMO the principle of least surprise here would
APbe that it should work.

Well, A = f(B) introduces a local variable A. Therefore also A = A+B.
And therefore also A += B.

The only way to have no surprises at all would be if local variables would
have to be declared explicitly, like 'local A'. But that would break
existing code. Or maybe the composite assignment operators should have been
exempted. Too late now!
Let me make one thing clear. I'm not trying to get the python people to
change anything. Most I hope for is that they would think about this
behaviour for python 3000.
You have the same problem with:

A = [10]
def inner():
A.append(2)

works but

A = [10]
def inner():
A += [2]

doesn't. The nasty thing in this example is that although A += [2] looks
like a rebinding syntactically, semantically there is no rebinding done.

I think the cleaner solution is (use parameters and don't rebind):
def inner(A):
A.append(2)
For what it is worth. My proposal would be to introduce a rebinding
operator, ( just for the sake of this exchange written as := ).

A line like "b := a", wouldn't make b a local variable but would
search for the name b in all active scopes and then rebind b
there. In terms of my hypothetical interpreter something like
the above.

LeftScope = SearchName("b", ScopeList)
RightScope = SearchName("a", ScopeList)
LeftScope["b"] = RightScope["a"]

With the understanding that b wouldn't be inserted in the local
scope unless there was an assignment to be somewhere else in
the function.

The augmented assignments could then be redefined in terms of the
rebinding operator instead of the assignment.

I'm not sure this proposal would eliminate all surprises but as
far as I can see it wouldn't break existing code. But I don't think
this proposal would have a serious chance.

In any case thanks for your contribution.

--
Antoon Pardon
Jul 6 '06 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
2039
by: Doug Tolton | last post by:
I have a function that returns a tuple: def checkdoc(self, document): blen = document bates = document normal, truncated, semicolon = 0,0,0 for bat in bates: if len(bat) == 2 * blen: semicolon += 1
3
2340
by: Nils Grimsmo | last post by:
hi, i'm having some trouble nesting functions. consider the following: def h(): x = 1 def g(): print x # ok, x is taken from h g()
6
2567
by: Andy Baker | last post by:
Hi there, I'm learning Python at the moment and trying to grok the thinking behind it's scoping and nesting rules. I was googling for nested functions and found this Guido quote: (http://www.python.org/search/hypermail/python-1993/0343.html) "This is because nested function definitions don't have access to the local variables of the surrounding block -- only to the globals of the
5
1807
by: Dave Benjamin | last post by:
I ran into an odd little edge case while experimenting with functions that create classes on the fly (don't ask me why): >>> def f(x): ... class C(object): ... x = x ... print C.x ... >>> f(5) Traceback (most recent call last):
4
1527
by: Pierre Barbier de Reuille | last post by:
Hello, a discussion began on python-dev about this. It began by a bug report, but is shifted and it now belongs to this discussion group. The problem I find with augmented assignment is it's too complex, it's badly explained, it's error-prone. And most of all, I don't see any use-case for it ! The most common error is to consider that :
0
8968
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8787
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
1
9259
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9208
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8208
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
6053
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4569
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4824
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3279
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.