473,320 Members | 1,978 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Augument assignment versus regular assignment

I do the following. First create lists x,y,z. Then add an element to x
using the augumented assignment operator. This causes all the other
lists to be changed also.
But if I use the assignment x=x+[4] instead of using the augumented
assignment, the y and z lists do not change.
Why is that?
This does not happen when I work with integer data type for example, as
shown below.

Thanks for your help
Nagy
>>x=y=z=[]
x+=[2]
x
[2]
>>y
[2]
>>z
[2]
>>x=x+[4]

x
[2, 4]
>>y
[2]
>>z
[2]
>>a=b=4
b
4
>>a+=2
a
6
>>b
4

Jul 8 '06 #1
35 2393
nagy wrote:
I do the following. First create lists x,y,z. Then add an element to x
using the augumented assignment operator. This causes all the other
lists to be changed also.
But if I use the assignment x=x+[4] instead of using the augumented
assignment, the y and z lists do not change.
Why is that?
This does not happen when I work with integer data type for example, as
shown below.

Thanks for your help
Nagy

>>>>x=y=z=[]
In this example, the '[]' creates a new list object. x, y, and z are all
set to reference that object.
>>>>x+=[2]
This does an "in-place" operation on that list, modifying (or
"mutating") the object directly.
>>>>x
[2]
>>>>y

[2]
>>>>z

[2]
>>>>x=x+[4]
This creates a new list that is the concatenation of the list created
above (the list [2]) with a new list (the list [4]). This brand new list
is bound to the name 'x'. The names 'y' and 'z' are left unchanged. That
is, they still point to the original list.
>>>>
x

[2, 4]
>>>>y

[2]
>>>>z

[2]
>>>>a=b=4
This binds the names 'a' and 'b' to the integer object 4.
>>>>b

4
>>>>a+=2
This attempts to mutate the integer object 4, by adding 2 to it.
However, numbers in Python are immutable, and so the in-place operation
fails. Thus, it creates a new integer object equal to 6 (actually,
CPython keeps a cache of certain smaller integer objects and reuses
them, but this does not matter in practice). This new integer object is
bound to the name 'a'. The name 'b' remains bound to the original 4 object.
>>>>a

6
>>>>b

4
Jul 8 '06 #2
Thanks, Kirk.
I considered the += as only a shorthand notation for the assignment
operator.
Since for lists + is simply a concatetation, I am not sure it x=x+[2]
is creating a brand
new list. Could you refer me to any documentation on this?
Thanks,
Nagy
Kirk McDonald wrote:
nagy wrote:
I do the following. First create lists x,y,z. Then add an element to x
using the augumented assignment operator. This causes all the other
lists to be changed also.
But if I use the assignment x=x+[4] instead of using the augumented
assignment, the y and z lists do not change.
Why is that?
This does not happen when I work with integer data type for example, as
shown below.

Thanks for your help
Nagy

>>>x=y=z=[]

In this example, the '[]' creates a new list object. x, y, and z are all
set to reference that object.
>>>x+=[2]

This does an "in-place" operation on that list, modifying (or
"mutating") the object directly.
>>>x
[2]
>>>y
[2]
>>>z
[2]
>>>x=x+[4]

This creates a new list that is the concatenation of the list created
above (the list [2]) with a new list (the list [4]). This brand new list
is bound to the name 'x'. The names 'y' and 'z' are left unchanged. That
is, they still point to the original list.
>>>
x
[2, 4]
>>>y
[2]
>>>z
[2]
>>>a=b=4

This binds the names 'a' and 'b' to the integer object 4.
>>>b
4
>>>a+=2

This attempts to mutate the integer object 4, by adding 2 to it.
However, numbers in Python are immutable, and so the in-place operation
fails. Thus, it creates a new integer object equal to 6 (actually,
CPython keeps a cache of certain smaller integer objects and reuses
them, but this does not matter in practice). This new integer object is
bound to the name 'a'. The name 'b' remains bound to the original 4 object.
>>>a
6
>>>b
4
Jul 8 '06 #3
nagy wrote:
Thanks, Kirk.
I considered the += as only a shorthand notation for the assignment
operator.
Since for lists + is simply a concatetation, I am not sure it x=x+[2]
is creating a brand
new list. Could you refer me to any documentation on this?
Yes:

http://docs.python.org/ref/augassign.html
"An augmented assignment expression like x += 1 can be rewritten as x =
x + 1 to achieve a similar, but not exactly equal effect. In the
augmented version, x is only evaluated once. Also, when possible, the
actual operation is performed in-place, meaning that rather than
creating a new object and assigning that to the target, the old object
is modified instead."

This behavior is only logical. Consider:
>>x = [2]
y = x + [4]
After these operations, we have two lists: x (the list [2]) and y (the
list [2, 4]). This is because the expression "x + [4]" creates a new
list. We then bind this new list to the name 'y', and leave the name 'x'
alone.

If we then say this:
>>x = x + [6]
We are doing much the same operation. We are creating a new list (the
list [2, 6]), and binding it to the name 'x'. The list [2], previously
bound to 'x', is no longer bound to anything, so Python frees it.

The augmented assignment, as I went over previously, attempts to modify
the list object directly. Any names bound to the object (or any other
objects that reference the object) will see the changes.

-Kirk McDonald
Jul 8 '06 #4
Looks like x=x+[2] creats a new list to me:
>>b = [8,5,6]
x = b
x = x + [2]
print b,x
[8, 5, 6] [8, 5, 6, 2]

-Chris
On Sat, Jul 08, 2006 at 11:56:11AM -0700, nagy wrote:
Thanks, Kirk.
I considered the += as only a shorthand notation for the assignment
operator.
Since for lists + is simply a concatetation, I am not sure it x=x+[2]
is creating a brand
new list. Could you refer me to any documentation on this?
Thanks,
Nagy
Kirk McDonald wrote:
nagy wrote:
I do the following. First create lists x,y,z. Then add an element to x
using the augumented assignment operator. This causes all the other
lists to be changed also.
But if I use the assignment x=x+[4] instead of using the augumented
assignment, the y and z lists do not change.
Why is that?
This does not happen when I work with integer data type for example, as
shown below.
>
Thanks for your help
Nagy
>
>
>>>>x=y=z=[]
In this example, the '[]' creates a new list object. x, y, and z are all
set to reference that object.
>>>>x+=[2]
This does an "in-place" operation on that list, modifying (or
"mutating") the object directly.
>>>>x
>
[2]
>
>>>>y
>
[2]
>
>>>>z
>
[2]
>
>>>>x=x+[4]
This creates a new list that is the concatenation of the list created
above (the list [2]) with a new list (the list [4]). This brand new list
is bound to the name 'x'. The names 'y' and 'z' are left unchanged. That
is, they still point to the original list.
>>>>
>>>>x
>
[2, 4]
>
>>>>y
>
[2]
>
>>>>z
>
[2]
>
>>>>a=b=4
This binds the names 'a' and 'b' to the integer object 4.
>>>>b
>
4
>
>>>>a+=2
This attempts to mutate the integer object 4, by adding 2 to it.
However, numbers in Python are immutable, and so the in-place operation
fails. Thus, it creates a new integer object equal to 6 (actually,
CPython keeps a cache of certain smaller integer objects and reuses
them, but this does not matter in practice). This new integer object is
bound to the name 'a'. The name 'b' remains bound to the original 4 object.
>>>>a
>
6
>
>>>>b
>
4
>

--
http://mail.python.org/mailman/listinfo/python-list
Jul 8 '06 #5

nagy wrote:
Thanks, Kirk.
I considered the += as only a shorthand notation for the assignment
operator.
Since for lists + is simply a concatetation, I am not sure it x=x+[2]
is creating a brand
new list. Could you refer me to any documentation on this?
Thanks,
Nagy
My habit is to check the id.
>>x = [1,2]
id(x)
-1209327188
>>x += [4]
x
[1,2,4]
>>id(x)
-1209327188
>>x = x + [6]
x
[1,2,4,6]
>>id(x)
-1209334664

So it looks as if x += [] modifies the list in place, while x = x + []
creates a new list.

I am not sure if this is 100% guaranteed, as I have noticed in the past
that id's can be reused under certain circumstances. Perhaps one of the
resident gurus can comment.

Frank Millman

Jul 9 '06 #6
Frank Millman wrote:
nagy wrote:
>>Thanks, Kirk.
I considered the += as only a shorthand notation for the assignment
operator.
Since for lists + is simply a concatetation, I am not sure it x=x+[2]
is creating a brand
new list. Could you refer me to any documentation on this?
Thanks,
Nagy


My habit is to check the id.

>>>>x = [1,2]
id(x)

-1209327188
>>>>x += [4]
x

[1,2,4]
>>>>id(x)

-1209327188
>>>>x = x + [6]
x

[1,2,4,6]
>>>>id(x)

-1209334664

So it looks as if x += [] modifies the list in place, while x = x + []
creates a new list.

I am not sure if this is 100% guaranteed,
It is. This is true for any mutable type.
as I have noticed in the past
that id's can be reused under certain circumstances. Perhaps one of the
resident gurus can comment.
Jul 9 '06 #7
Frank Millman wrote:
So it looks as if x += [] modifies the list in place, while x = x + []
creates a new list.
objects can override the += operator (by defining the __iadd__ method),
and the list type maps __iadd__ to extend. other containers may treat
+= differently, but in-place behaviour is recommended by the language
reference:

An augmented assignment expression like x += 1 can be rewritten as
x = x + 1 to achieve a similar, but not exactly equal effect. In
the augmented version, x is only evaluated once. Also, when possible,
the actual operation is performed in-place, meaning that rather than
creating a new object and assigning that to the target, the old object
is modified instead.

</F>

Jul 9 '06 #8
On 2006-07-09, Fredrik Lundh <fr*****@pythonware.comwrote:
Frank Millman wrote:
>So it looks as if x += [] modifies the list in place, while x = x + []
creates a new list.

objects can override the += operator (by defining the __iadd__ method),
and the list type maps __iadd__ to extend. other containers may treat
+= differently, but in-place behaviour is recommended by the language
reference:

An augmented assignment expression like x += 1 can be rewritten as
x = x + 1 to achieve a similar, but not exactly equal effect. In
the augmented version, x is only evaluated once. Also, when possible,
the actual operation is performed in-place, meaning that rather than
creating a new object and assigning that to the target, the old object
is modified instead.
What does it mean that x is only evaluated once. I have an avltree module,
with an interface much like a directory. So I added print statements to
__setitem__ and __getitem__ and then tried the following code.
>>from avltree import Tree
t=Tree()
t['a'] = 1
__setitem__, key = a
>>t['a']
__getitem__, key = a
1
>>t['a'] = t['a'] + 1
__getitem__, key = a
__setitem__, key = a
>>t['a'] += 1
__getitem__, key = a
__setitem__, key = a
>>t['b'] = []
__setitem__, key = b
>>t['b'] = t['b'] + [1]
__getitem__, key = b
__setitem__, key = b
>>t['b'] += [2]
__getitem__, key = b
__setitem__, key = b

So to me it seems that when we substitute t['a'] or t['b'] for x,
x is evaluated twice with the augmented version, just like it
is with the not augmented version.

--
Antoon Pardon

Jul 10 '06 #9
In article <sl********************@rcpc42.vub.ac.be>,
Antoon Pardon <ap*****@forel.vub.ac.bewrote:
>On 2006-07-09, Fredrik Lundh <fr*****@pythonware.comwrote:
>Frank Millman wrote:
>>So it looks as if x += [] modifies the list in place, while x = x + []
creates a new list.

objects can override the += operator (by defining the __iadd__ method),
and the list type maps __iadd__ to extend. other containers may treat
+= differently, but in-place behaviour is recommended by the language
reference:

An augmented assignment expression like x += 1 can be rewritten as
x = x + 1 to achieve a similar, but not exactly equal effect. In
the augmented version, x is only evaluated once. Also, when possible,
the actual operation is performed in-place, meaning that rather than
creating a new object and assigning that to the target, the old object
is modified instead.

What does it mean that x is only evaluated once. I have an avltree module,
with an interface much like a directory. So I added print statements to
__setitem__ and __getitem__ and then tried the following code.
>>>from avltree import Tree
t=Tree()
t['a'] = 1
__setitem__, key = a
>>>t['a']
__getitem__, key = a
1
>>>t['a'] = t['a'] + 1
__getitem__, key = a
__setitem__, key = a
>>>t['a'] += 1
__getitem__, key = a
__setitem__, key = a
>>>t['b'] = []
__setitem__, key = b
>>>t['b'] = t['b'] + [1]
__getitem__, key = b
__setitem__, key = b
>>>t['b'] += [2]
__getitem__, key = b
__setitem__, key = b

So to me it seems that when we substitute t['a'] or t['b'] for x,
x is evaluated twice with the augmented version, just like it
is with the not augmented version.
$ cat x.py

def getindex(ind = 0):
print 'getindex() called'
return ind

a = [0, 1, 2, 3, 4, 5]
a[getindex(0)] = a[getindex(0)] + 17
print a
a[getindex(1)] += 22
print a

$ python x.py
getindex() called
getindex() called
[17, 1, 2, 3, 4, 5]
getindex() called
[17, 23, 2, 3, 4, 5]

In this case, getindex() is a rather pointless function, but it could
be an expensive one or one with side effects or even one which alters
state, so that it gives different values on subsequent calls with the
same argument.

The += version finds the object to be operated upon once, the expanded
version does it twice.
--
Jim Segrave (je*@jes-2.demon.nl)

Jul 10 '06 #10
On 2006-07-10, Jim Segrave <je*@nl.demon.netwrote:
In article <sl********************@rcpc42.vub.ac.be>,
Antoon Pardon <ap*****@forel.vub.ac.bewrote:
>>On 2006-07-09, Fredrik Lundh <fr*****@pythonware.comwrote:
>>Frank Millman wrote:

So it looks as if x += [] modifies the list in place, while x = x + []
creates a new list.

objects can override the += operator (by defining the __iadd__ method),
and the list type maps __iadd__ to extend. other containers may treat
+= differently, but in-place behaviour is recommended by the language
reference:

An augmented assignment expression like x += 1 can be rewritten as
x = x + 1 to achieve a similar, but not exactly equal effect. In
the augmented version, x is only evaluated once. Also, when possible,
the actual operation is performed in-place, meaning that rather than
creating a new object and assigning that to the target, the old object
is modified instead.

What does it mean that x is only evaluated once. I have an avltree module,
with an interface much like a directory. So I added print statements to
__setitem__ and __getitem__ and then tried the following code.
>>>>from avltree import Tree
t=Tree()
t['a'] = 1
__setitem__, key = a
>>>>t['a']
__getitem__, key = a
1
>>>>t['a'] = t['a'] + 1
__getitem__, key = a
__setitem__, key = a
>>>>t['a'] += 1
__getitem__, key = a
__setitem__, key = a
>>>>t['b'] = []
__setitem__, key = b
>>>>t['b'] = t['b'] + [1]
__getitem__, key = b
__setitem__, key = b
>>>>t['b'] += [2]
__getitem__, key = b
__setitem__, key = b

So to me it seems that when we substitute t['a'] or t['b'] for x,
x is evaluated twice with the augmented version, just like it
is with the not augmented version.

$ cat x.py

def getindex(ind = 0):
print 'getindex() called'
return ind

a = [0, 1, 2, 3, 4, 5]
a[getindex(0)] = a[getindex(0)] + 17
print a
a[getindex(1)] += 22
print a

$ python x.py
getindex() called
getindex() called
[17, 1, 2, 3, 4, 5]
getindex() called
[17, 23, 2, 3, 4, 5]

In this case, getindex() is a rather pointless function, but it could
be an expensive one or one with side effects or even one which alters
state, so that it gives different values on subsequent calls with the
same argument.

The += version finds the object to be operated upon once, the expanded
version does it twice.
I disagree. The += version only evaluates the index once, but still has
to find the object twice.

Let us start with the following:

t['b'] = []

Now with x being evaluated once, I would expect that

t[getindex('b')] += [1]

would be equivallent to:

_key = getindex('b')
_lst = t[_key]
_lst += [1]

And not to the following:

_key = getindex('b')
_lst = t[_key]
_lst += [1]
t[_key] = _lst
But as far as I can interpret what is happening from the printed lines
the second is happening and not the first. So in this example some
optimisation has happened, by calculating the key only once, but
the search for the object using that precalculated key happens twice,
once with __getitem__ and once with __setitem__.
>>t['b'] = []
__setitem__, key = b
>>t[getindex('b')] += [1]
getindex() called
__getitem__, key = b
__setitem__, key = b

--
Antoon Pardon
Jul 10 '06 #11

"Antoon Pardon" <ap*****@forel.vub.ac.bewrote in message
news:sl********************@rcpc42.vub.ac.be...
I disagree. The += version only evaluates the index once, but still has
to find the object twice.
No it does not *have* to find the object twice and no it does *not* find
the object twice. From the viewpoint of the interpreter, the purpose of
abbreviating 'objectexpression = objectexpression op arg' as
'objectexpression op=arg' is to avoid unnecessary recalculation
But as far as I can interpret what is happening from the printed lines
Your print get/set examples from your black-box testing miss the following
key point. While the interpreter still has to *use* the object(s) twice,
once to get and once to set, it no longer has to *calculate* the objects
twice. If we open the box and look at the compiled pycode, we get, for
example:
>>dis(compile('x[i][j+k]=x[i][j+k]+1', '', 'single'))
1 0 LOAD_NAME 0 (x)
3 LOAD_NAME 1 (i)
6 BINARY_SUBSCR
7 LOAD_NAME 2 (j)
10 LOAD_NAME 3 (k)
13 BINARY_ADD

14 BINARY_SUBSCR
15 LOAD_CONST 0 (1)
18 BINARY_ADD

19 LOAD_NAME 0 (x)
22 LOAD_NAME 1 (i)
25 BINARY_SUBSCR
26 LOAD_NAME 2 (j)
29 LOAD_NAME 3 (k)
32 BINARY_ADD

33 STORE_SUBSCR
>>dis(compile('x[i][j+k]+=1', '', 'single'))
1 0 LOAD_NAME 0 (x)
3 LOAD_NAME 1 (i)
6 BINARY_SUBSCR
7 LOAD_NAME 2 (j)
10 LOAD_NAME 3 (k)
13 BINARY_ADD

14 DUP_TOPX 2

17 BINARY_SUBSCR
18 LOAD_CONST 0 (1)
21 INPLACE_ADD

22 ROT_THREE

23 STORE_SUBSCR

[blank lines added for clarity]

Both forms call binary_subscr to get the number to add to 1 and both call
store_subscr to set the result back in the list. But in the first case,
the expressions for both the source-target list and the index of the value
for the addition are duplicated and re-evaluated. In the second, the
*results* of the first evaluation are duplicated (on the stack) and saved
for the storage operation.

For understanding the detailed operation of CPython, dis is a great help

Terry Jan Reedy

Jul 10 '06 #12
On 2006-07-10, Terry Reedy <tj*****@udel.eduwrote:
>
"Antoon Pardon" <ap*****@forel.vub.ac.bewrote in message
news:sl********************@rcpc42.vub.ac.be...
>I disagree. The += version only evaluates the index once, but still has
to find the object twice.

No it does not *have* to find the object twice and no it does *not* find
the object twice. From the viewpoint of the interpreter, the purpose of
abbreviating 'objectexpression = objectexpression op arg' as
'objectexpression op=arg' is to avoid unnecessary recalculation
But is the viewpoint of the interpreter somehow relevant? IMO the
question is if the actual behaviour is compatible with what people
expect after having read the language reference. The viewpoint of
the interpreter IMO doesn't play a role in answering that question.
>But as far as I can interpret what is happening from the printed lines

Your print get/set examples from your black-box testing miss the following
key point. While the interpreter still has to *use* the object(s) twice,
once to get and once to set, it no longer has to *calculate* the objects
twice.
The language reference doesn't talk about objects. And IMO you
should be carefull if you want to use the word "object" here.
In the line: "foo += 1", you can't talk about the object foo,
since foo will be bound to a different object after the assignment
than it was bound to before.

As I read the language reference the x stands for a target expression.
Now what does it mean to evaluate a target expression like col[key].
IMO it means finding the location of the item in the collection: the
bucket in the directory, the node in the tree ... grosso mode it
boils down to the call to __setitem__ or __getitem__ depending
on where the col[key] was located in the line (or if you prefer
the view from the interpreter it boils down to the BINARY_SUBSCR
and STORE_SUBSCR opcodes).

So if the language reference seems to implies that col[key] will
only be evaluated once in a line like: "col[key] += 1" I expect
only one call from __setitem__ or __getitem__ (or only one
from BINARY_SUBSCR or STORE_SUBSCR)

Now I know python doesn't behave this way, but how python
actually behave can't be used as an argument that this
is the behaviour as described by the language reference.

So my question is: suppose I write my own collector,
where __setitem__ and __getitem__ have the same side effect.
How many times should/will this side effect occur in code like
"col[key] += 1". As I read the language reference it should
happen only once, however that is not what happens. So if
the actual behaviour of python is what we want, which is
what I suspect, then the language reference should
clarify more what the supposed behaviour should be.

Now my reading of the language reference can be faulty,
but if you want to argue that, I would appreciate it
if you could explain how I have to read the language
reference in order to come to the conclusion that the
side effect in this example has to happen twice.

And even in this case would I suggest that the language
reference would better be made clearer, since I doubt
that I'm the only who will read the language reference
this way.

--
Antoon Pardon
Jul 11 '06 #13
>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:
>APAs I read the language reference the x stands for a target expression.
APNow what does it mean to evaluate a target expression like col[key].
APIMO it means finding the location of the item in the collection: the
APbucket in the directory, the node in the tree ... grosso mode it
APboils down to the call to __setitem__ or __getitem__ depending
APon where the col[key] was located in the line (or if you prefer
APthe view from the interpreter it boils down to the BINARY_SUBSCR
APand STORE_SUBSCR opcodes).
>APSo if the language reference seems to implies that col[key] will
APonly be evaluated once in a line like: "col[key] += 1" I expect
APonly one call from __setitem__ or __getitem__ (or only one
APfrom BINARY_SUBSCR or STORE_SUBSCR)
You need both the __setitem__ and the __getitem__ otherwise it won't work.
I think the "evaluated once" clause is for cases like:

f(a)[g(b)] += 1

where f and/or g have side effects. These side effects should then take
place only once. Side effects in __setitem__ and __getitem__ is not what it
is talking about.
--
Piet van Oostrum <pi**@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email: pi**@vanoostrum.org
Jul 11 '06 #14
On 2006-07-11, Piet van Oostrum <pi**@cs.uu.nlwrote:
>>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:
>>APAs I read the language reference the x stands for a target expression.
APNow what does it mean to evaluate a target expression like col[key].
APIMO it means finding the location of the item in the collection: the
APbucket in the directory, the node in the tree ... grosso mode it
APboils down to the call to __setitem__ or __getitem__ depending
APon where the col[key] was located in the line (or if you prefer
APthe view from the interpreter it boils down to the BINARY_SUBSCR
APand STORE_SUBSCR opcodes).
>>APSo if the language reference seems to implies that col[key] will
APonly be evaluated once in a line like: "col[key] += 1" I expect
APonly one call from __setitem__ or __getitem__ (or only one
APfrom BINARY_SUBSCR or STORE_SUBSCR)

You need both the __setitem__ and the __getitem__ otherwise it won't work.
I think the "evaluated once" clause is for cases like:

f(a)[g(b)] += 1

where f and/or g have side effects. These side effects should then take
place only once. Side effects in __setitem__ and __getitem__ is not what it
is talking about.
Well I'll start on an possitive note and accept this. Now I'd like you
to answer some questions.

1) Do you think the langauge reference makes it clear that this is how
the reader has to understand things.

2a) In case you answer yes to question (1). Can you explain me how
I have to read the language reference in order to deduce this
is indeed the way things should be understood.

2b) In case you anser no to question (1). Do you find it unreasonable
that I ask that the language reference would be rewritten here so
that your explanation can be (more easily) deduced from it.

--
Antoon Pardon
Jul 12 '06 #15
>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:
>APWell I'll start on an possitive note and accept this. Now I'd like you
APto answer some questions.
>AP1) Do you think the langauge reference makes it clear that this is how
AP the reader has to understand things.
Yes.
>AP2a) In case you answer yes to question (1). Can you explain me how
AP I have to read the language reference in order to deduce this
AP is indeed the way things should be understood.
Just read what it says. `It is only evaluated once' is quite clear I would
say. Your problem is that you thought __setitem__ is part of evaluation,
but it isn't. It is part of assignment, while __getitem__ is part of
evaluation. See the definitions of __getitem__ and __setitem__ in the
language reference manual.

--
Piet van Oostrum <pi**@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email: pi**@vanoostrum.org
Jul 14 '06 #16
On 2006-07-14 16:07:28, Piet van Oostrum wrote:
>>AP2a) In case you answer yes to question (1). Can you explain me how
AP I have to read the language reference in order to deduce this
AP is indeed the way things should be understood.

Just read what it says. `It is only evaluated once' is quite clear I would
say. Your problem is that you thought __setitem__ is part of evaluation,
but it isn't. It is part of assignment, while __getitem__ is part of
evaluation. See the definitions of __getitem__ and __setitem__ in the
language reference manual.
Sorry to butt in here... I really don't know much more about this than I
read in this thread :)

But wasn't stated earlier that one of the differences between a += b and a
= a + b is that a gets evaluated once in the first case and twice in the
second case? If __getitem__ was part of the evaluation (as you seem to
say), shouldn't it be called twice in the second case? It doesn't seem to
get called twice; see this snippet from an earlier message:
>>t['a'] = t['a'] + 1
__getitem__, key = a
__setitem__, key = a
>>t['a'] += 1
__getitem__, key = a
__setitem__, key = a
Seems like the __get/setitem__ thing has not much to do with what the
manual calls evaluation, but rather with what the name implies: setting and
getting the value of the item. And therefore, since in both the a += b case
and the a = a + b case the value of a is gotten once and set once,
__getitem__ gets called once and __setitem__ gets called once. No?

Thanks,
Gerhard

Jul 14 '06 #17
The problem with understanding augmented assignment is that it directs the
compiler and interpreter to do one or maybe two mostly invisible
optimizations. To me, the effective meaning of 'evalutating once versus
twice' is most easily seen in the byte code generated by what is, remember,
the reference implementation. What it does is what the
less-than-super-clear doc means.

I posted the difference for one expression after looking at three other
pairs. It is easy to examine such pairs in IDLE and, I presume, in other
IDEs.

Terry Jan Reedy

Jul 15 '06 #18
>>>>Gerhard Fiedler <ge*****@gmail.com(GF) wrote:
>GFOn 2006-07-14 16:07:28, Piet van Oostrum wrote:
AP2a) In case you answer yes to question (1). Can you explain me how
API have to read the language reference in order to deduce this
APis indeed the way things should be understood.
>>>
Just read what it says. `It is only evaluated once' is quite clear I would
say. Your problem is that you thought __setitem__ is part of evaluation,
but it isn't. It is part of assignment, while __getitem__ is part of
evaluation. See the definitions of __getitem__ and __setitem__ in the
language reference manual.
>GFSorry to butt in here... I really don't know much more about this than I
GFread in this thread :)
>GFBut wasn't stated earlier that one of the differences between a += b and a
GF= a + b is that a gets evaluated once in the first case and twice in the
GFsecond case? If __getitem__ was part of the evaluation (as you seem to
GFsay), shouldn't it be called twice in the second case? It doesn't seem to
GFget called twice; see this snippet from an earlier message:
>>>>t['a'] = t['a'] + 1
GF__getitem__, key = a
GF__setitem__, key = a
>>>>t['a'] += 1
GF__getitem__, key = a
GF__setitem__, key = a
>GFSeems like the __get/setitem__ thing has not much to do with what the
GFmanual calls evaluation, but rather with what the name implies: setting and
GFgetting the value of the item. And therefore, since in both the a += b case
GFand the a = a + b case the value of a is gotten once and set once,
GF__getitem__ gets called once and __setitem__ gets called once. No?
Yes, in both cases you get the value once, and you set the value once.

In an assignment, the lefthand side is evaluated differently from the
righthand side, of course. Because in the righthand side you need the value
of the object, but in the lefthand side you need only the 'location' (this
is not a Python term).
Therefore in the righthand side __getitem__ is part of the evaluation.
In the lefthand side when it is a[i], only a and i are evaluated, but then
the evaluation stops. Next the assignment is done with __setitem__.

Now if it is an augmented assignment like a[i]+=b, It is only evaluated
once. But we need it both as a lefthand side and a righthand side. So this
means that a and i are evaluated (but only once!). For the lefthand side
this completes the evaluation. And then the results of these are used as
parameters to __getitem__ to complete the evaluation of the righthand side.
Assuming b had already been evaluated, next the assignment is performed
by calling __setitem__.

Your example above doesn't show any difference because t['a'] doesn't have
any side effects. But if you use for both t and the index a function that
prints something you will see the difference.
--
Piet van Oostrum <pi**@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email: pi**@vanoostrum.org
Jul 15 '06 #19
On 2006-07-14, Piet van Oostrum <pi**@cs.uu.nlwrote:
>>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:
>>APWell I'll start on an possitive note and accept this. Now I'd like you
APto answer some questions.
>>AP1) Do you think the langauge reference makes it clear that this is how
AP the reader has to understand things.

Yes.
>>AP2a) In case you answer yes to question (1). Can you explain me how
AP I have to read the language reference in order to deduce this
AP is indeed the way things should be understood.

Just read what it says. `It is only evaluated once' is quite clear I would
say.
If it is so clear, why don't you explain it?
Your problem is that you thought __setitem__ is part of evaluation,
No I think the other way around. The evaluation is part of __setitem__
In so far as it makes sense to talk about the evaluation of a target
in python
but it isn't. It is part of assignment,
No the assignment is part of __setitem__
while __getitem__ is part of evaluation.
How can __getitem__ be part of the evaluation of col['t'] in
a statment like

col['t'] = 5

The language reference about augmented assigments is not only
talking about evaluating a language element as an expression,
it is also talking about evaluating that language element as
a target. Now I'm not so sure that it makes sense to talk about
evaluating a target in python, but I see no way to avoid that
interpretation in:

An augmented assignment expression like x += 1 can be rewritten as
x = x + 1 to achieve a similar, but not exactly equal effect. In
the augmented version, x is only evaluated once.

x is only evaluated once in a statement like: x+= 1
This suggests x is evaluated twice in a statement like: x = x + 1
The only way to eavlaute x twice in that statement is that
x is evaluated once as a target.
So the langauage reference is here talking about (among other things)
evaluating a target.

What the language reference should have said IMO is that in case x
is an attribute reference, index or slicing, the primary expression
will be evaluated only once, as will be the index or slice in the
two latter cases.

This wording doesn't introduce two different kinds of evaluation
as the original wording of the language reference does.

--
Antoon Pardon
Jul 17 '06 #20
On 2006-07-15, Terry Reedy <tj*****@udel.eduwrote:
The problem with understanding augmented assignment is that it directs the
compiler and interpreter to do one or maybe two mostly invisible
optimizations. To me, the effective meaning of 'evalutating once versus
twice' is most easily seen in the byte code generated by what is, remember,
the reference implementation. What it does is what the
less-than-super-clear doc means.
But what does one do, if the question is raised whether or not the
code generated actually behaves as described by the language reference?

Shouldn't the language reference be clear enough to be understandable
without the help of byte code generated by the reference implementation?
Otherwise the reference implemenation is being used as part of the
language definition and then it can never be wrong.

--
Antoon Pardon
Jul 17 '06 #21
Antoon Pardon wrote:
>
What the language reference should have said IMO is that in case x
is an attribute reference, index or slicing, the primary expression
will be evaluated only once, as will be the index or slice in the
two latter cases.
I think the difficulty here for the author of this particular section
of the reference is in describing the mechanisms at work whilst keeping
the description at a conceptual level which can be directly connected
to the source text in a program. However, apart from providing a vague
intuition about how augmented assignment must work, the description
doesn't answer all questions effectively, particularly in cases where
the actual assignments are not simple local/global name binding
operations.

One way to consider augmented assignment is as a statement formulated
in the following way:

setvalue(namespace, name, op(getvalue(namespace, name), expr))

Consider an augmented assignment on a local name:

a += b

This could be phrased as follows:

setname(locals, "a", add(getname(locals, "a"), b))

Or really:

setname(locals, "a", add(getname(locals, "a"), getname(locals, "b")))

Consider an augmented assignment on an attribute:

a.b += c

This could also be phrased similarly:

setattr(a, "b", add(getattr(a, "b"), c))

And consider an augmented assignment on an item:

a[b] += c

This too is phrased similarly:

setitem(a, b, add(getitem(a, b), c))

So, as long as you're willing to accept that the setvalue class of
operations (setname, setattr, setitem) aren't really evaluating the
target of the assignment - strictly, the thing being replaced in the
assignment - then only the getvalue class of operations (getname,
getattr, getitem) are causing the evaluation of the target. And since
setname is just a normal name binding assignment which we know doesn't
cause the target to be evaluated, we can assume that the other
operations in that class behave similarly.

Now, one aspect of evaluation has been skipped in the above
explanation: what if we have a "complicated" expression on the left
hand side? Consider this:

a.f().x += y

We could naively write this as follows:

setattr(a.f(), "x", add(getattr(a.f(), "x"), y))

However, it looks like we are evaluating at least part of the left hand
side twice. Thus, we need to be a bit clearer about what really
happens:

namespace = a.f()
setattr(namespace, "x", add(getattr(namespace, "x"), y))

In order to avoid double evaluation, we first obtain the target
namespace (or itemspace, I suppose). Then, we perform the operations as
stated above.

So, to conclude, augmented assignment involves the evaluation of any
expression which gives the target namespace (trivial in the case of
local/global name binding, non-trivial otherwise), then a compound
operation of the form given above involving two classes of operations,
each providing operations to act on names, attributes and items.

I doubt that this explanation is simple, clear or efficient enough for
the language reference, but then any explanation shouldn't be
constrained in terms of complexity in a way that, for example, a
tutorial explanation should be. Despite cries of "you must get it now!"
from various quarters, I think this thread has been informative, and
I'd like to thank you for bringing this matter to the group's
attention, Antoon.

Paul

Jul 17 '06 #22
>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:
>APOn 2006-07-14, Piet van Oostrum <pi**@cs.uu.nlwrote:
>>Just read what it says. `It is only evaluated once' is quite clear I would
say.
>APIf it is so clear, why don't you explain it?
I have done that in another thread.
>>Your problem is that you thought __setitem__ is part of evaluation,
>APNo I think the other way around. The evaluation is part of __setitem__
APIn so far as it makes sense to talk about the evaluation of a target
APin python
How can evaluation be part of __setitem__? (If we talk about targets). The
LRM says clearly that it is part of assignment.
>>but it isn't. It is part of assignment,
>APNo the assignment is part of __setitem__
__setitem__( self, key, value)
Called to implement assignment to self[key]
>>while __getitem__ is part of evaluation.
>APHow can __getitem__ be part of the evaluation of col['t'] in
APa statment like
>AP col['t'] = 5
It isn't, because there is a difference between evaluation in the
lefthandside and in the righthandside. See the description of assignment.
>APThe language reference about augmented assigments is not only
APtalking about evaluating a language element as an expression,
APit is also talking about evaluating that language element as
APa target. Now I'm not so sure that it makes sense to talk about
APevaluating a target in python, but I see no way to avoid that
APinterpretation in:
In the description of assignment the distinction is made for each case of
the lefthandside. So that for example when the lefthandside is a
subscription, only the primary and the index are evaluated, but no
__getitem__ is called.
>AP An augmented assignment expression like x += 1 can be rewritten as
AP x = x + 1 to achieve a similar, but not exactly equal effect. In
AP the augmented version, x is only evaluated once.
>APx is only evaluated once in a statement like: x+= 1
APThis suggests x is evaluated twice in a statement like: x = x + 1
APThe only way to eavlaute x twice in that statement is that
AP x is evaluated once as a target.
APSo the langauage reference is here talking about (among other things)
APevaluating a target.
yes.
>APWhat the language reference should have said IMO is that in case x
APis an attribute reference, index or slicing, the primary expression
APwill be evaluated only once, as will be the index or slice in the
APtwo latter cases.
You should read the part about Augmented assignment statements after
carefully reading the part about assignments. Then you know already that
evaluation of the target is only about the primary and the index.
--
Piet van Oostrum <pi**@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email: pi**@vanoostrum.org
Jul 17 '06 #23

"Antoon Pardon" <ap*****@forel.vub.ac.bewrote in message
news:sl********************@rcpc42.vub.ac.be...
On 2006-07-15, Terry Reedy <tj*****@udel.eduwrote:
>The problem with understanding augmented assignment is that it directs
the
compiler and interpreter to do one or maybe two mostly invisible
optimizations. To me, the effective meaning of 'evalutating once versus
twice' is most easily seen in the byte code generated by what is,
remember,
the reference implementation. What it does is what the
less-than-super-clear doc means.

But what does one do, if the question is raised whether or not the
code generated actually behaves as described by the language reference?

Shouldn't the language reference be clear enough to be understandable
without the help of byte code generated by the reference implementation?
Otherwise the reference implemenation is being used as part of the
language definition and then it can never be wrong.

--
Antoon Pardon
--
http://mail.python.org/mailman/listinfo/python-list


Jul 17 '06 #24
"Antoon Pardon" <ap*****@forel.vub.ac.bewrote in message
news:sl********************@rcpc42.vub.ac.be...
On 2006-07-15, Terry Reedy <tj*****@udel.eduwrote:
>The problem with understanding augmented assignment is that it directs
the
compiler and interpreter to do one or maybe two mostly invisible
optimizations. To me, the effective meaning of 'evalutating once versus
twice' is most easily seen in the byte code generated by what is,
remember,
the reference implementation. What it does is what the
less-than-super-clear doc means.

But what does one do, if the question is raised whether or not the
code generated actually behaves as described by the language reference?
One can just as well ask whether or not the docs actually describe the
behavior of the interpreter ;-). In this case, not as well as one might
like. Or, whether docs (and reasonable interpretation thereof) and
implementation match, which I claim they do it this case.
Shouldn't the language reference be clear enough to be understandable
without the help of byte code generated by the reference implementation?
Normally, code experiments, such as you attempted, should be enough. But,
as I said, trying to document mostly invisibly behavior is more difficult.
Based on looking at the byte code, I would put it this way: the difference
between assignment with duplicated source and target expressions and
augmented assigment with the same expression is that the interpreter
eliminates as much duplicate calculation as possible. But precisely what
that mean seems a bit interpreter dependent to me.

Should we expect a reference writen in (ambiguous) natural language to be
unambiguously clear? I think not; that is why formal grammers and
notations were invented. I learned Python with the doc in one window and
the interactive interpreter in another.
Otherwise the reference implemenation is being used as part of the
language definition
Exactly. People doing other implementations use it along with the docs.
and then it can never be wrong.
No, 'part of' does not mean 'governing'. A Python 'bug' is a discrepancy
between doc and code. Usually, normal experienced judgment is enough to
decide which is faulty and needs to be changed. Occasionaly, both are
plausibly correct and such cases get referred to Guido or the developer's
list for discussion and a decision.

Terry Jan Reedy


Jul 17 '06 #25
On 2006-07-17, Piet van Oostrum <pi**@cs.uu.nlwrote:
>>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:
>>APOn 2006-07-14, Piet van Oostrum <pi**@cs.uu.nlwrote:
>>>Just read what it says. `It is only evaluated once' is quite clear I would
say.
>>APIf it is so clear, why don't you explain it?

I have done that in another thread.
Before I respond I would like you to be more specific
about where you gave that explanation. The subject
of the thread would be fine or a message-id perhaps.

--
Antoon Pardon
Jul 18 '06 #26
On 2006-07-17, Paul Boddie <pa**@boddie.org.ukwrote:
Antoon Pardon wrote:
>>
What the language reference should have said IMO is that in case x
is an attribute reference, index or slicing, the primary expression
will be evaluated only once, as will be the index or slice in the
two latter cases.

I think the difficulty here for the author of this particular section
of the reference is in describing the mechanisms at work whilst keeping
the description at a conceptual level which can be directly connected
to the source text in a program.
Sure! However the author of the augmented assignment reference uses
a concept that seems not be a pythonic concept: a target evaluation.
Call it an lvalue, a lefthand evaluation if you prefer, but in
the assignment reference it is never mentioned. Each time the
word evaluation is used, it is the evaluation of an expression, never
a target. Sure sometimes such a expression is part of a target, but
the evaluation never reaches a point where the result is the evaluation
of a target: So why doesn't the autor of the augmented assignment reference
limit himself to the same concepts used in the assignment reference?

Now maybe I'm just not bright enough, so maybe you can explain what
something like col['t'] is evaluated to in a statement like:

col['t'] = exp
However, apart from providing a vague
intuition about how augmented assignment must work, the description
doesn't answer all questions effectively, particularly in cases where
the actual assignments are not simple local/global name binding
operations.

One way to consider augmented assignment is as a statement formulated
in the following way:

setvalue(namespace, name, op(getvalue(namespace, name), expr))

Consider an augmented assignment on a local name:

a += b

This could be phrased as follows:

setname(locals, "a", add(getname(locals, "a"), b))

Or really:

setname(locals, "a", add(getname(locals, "a"), getname(locals, "b")))

Consider an augmented assignment on an attribute:

a.b += c

This could also be phrased similarly:

setattr(a, "b", add(getattr(a, "b"), c))

And consider an augmented assignment on an item:

a[b] += c

This too is phrased similarly:

setitem(a, b, add(getitem(a, b), c))

So, as long as you're willing to accept that the setvalue class of
operations (setname, setattr, setitem) aren't really evaluating the
target of the assignment - strictly, the thing being replaced in the
assignment - then only the getvalue class of operations (getname,
getattr, getitem) are causing the evaluation of the target. And since
setname is just a normal name binding assignment which we know doesn't
cause the target to be evaluated, we can assume that the other
operations in that class behave similarly.
Well maybe I'm missing your point entirely but would you be so friendly,
as to rephrase the following statements too:

a = a + b
a.b = a.b + c
a[b] = a[b] + c

Because as far as I can see the rephrasing of these statements will
result in exactly the same results as their augmented assignment
counter parts. So following your logic above, we would have to
come to the conclusion that the "targets" of these statements are
evaluated once too. This while the language reference suggests
they are evaluated twice.

IMO the language reference sugest that the stament:

a[i] += [b]

is more effective than:

a[i] = a[i] + [b]

IMO it suggests that the augmented assignment version is
just as effective as:

a[i].append(b)

Because, i hope, we agree that a[i] is evaluated only
once here!

So we have here three statements all which result
in a similar result one in which the
number of evaluations of a[i] is under dispute,

one in which the number of evaluations is suggested
to be two

one in which the number of evaluations is one.

Yet the work done in the augmented version of this
example is almost the same as in the normal assignment,
the difference is negligible IMO and both version
differ about the same with the append version which
will have to do about half what the other versions had to
do, depending on how expensive the __xetitem__ methods
are.

Now, one aspect of evaluation has been skipped in the above
explanation: what if we have a "complicated" expression on the left
hand side? Consider this:

a.f().x += y

We could naively write this as follows:

setattr(a.f(), "x", add(getattr(a.f(), "x"), y))

However, it looks like we are evaluating at least part of the left hand
side twice. Thus, we need to be a bit clearer about what really
happens:

namespace = a.f()
setattr(namespace, "x", add(getattr(namespace, "x"), y))

In order to avoid double evaluation, we first obtain the target
namespace (or itemspace, I suppose). Then, we perform the operations as
stated above.
Yes, and what you are doing here is eliminating the double evaluation
of the primary. In a statement like a.f().x = ... the a.f() part
is the primary according to the assignment reference.
So, to conclude, augmented assignment involves the evaluation of any
expression which gives the target namespace (trivial in the case of
local/global name binding, non-trivial otherwise), then a compound
operation of the form given above involving two classes of operations,
each providing operations to act on names, attributes and items.
This seems not be complete, if the target is a list, there is no target
namespace. That is why I used "primary" in my proposal, because
it catches all cases and it is also the term used in the assigment
reference.
I doubt that this explanation is simple, clear or efficient enough for
the language reference,
Would you mind explaning, what you think is not simple, clear or
efficient enough about my proposed wording? Just curious.
but then any explanation shouldn't be
constrained in terms of complexity in a way that, for example, a
tutorial explanation should be. Despite cries of "you must get it now!"
from various quarters, I think this thread has been informative, and
I'd like to thank you for bringing this matter to the group's
attention, Antoon.
Your welcome.

--
Antoon Pardon
Jul 18 '06 #27
On 2006-07-17, Terry Reedy <tj*****@udel.eduwrote:
"Antoon Pardon" <ap*****@forel.vub.ac.bewrote in message
news:sl********************@rcpc42.vub.ac.be...
>On 2006-07-15, Terry Reedy <tj*****@udel.eduwrote:
>>The problem with understanding augmented assignment is that it directs
the
compiler and interpreter to do one or maybe two mostly invisible
optimizations. To me, the effective meaning of 'evalutating once versus
twice' is most easily seen in the byte code generated by what is,
remember,
the reference implementation. What it does is what the
less-than-super-clear doc means.

But what does one do, if the question is raised whether or not the
code generated actually behaves as described by the language reference?

One can just as well ask whether or not the docs actually describe the
behavior of the interpreter ;-). In this case, not as well as one might
like. Or, whether docs (and reasonable interpretation thereof) and
implementation match, which I claim they do it this case.
Well this dispute seems to boil down what is and what is not
involved in evaluating a target, and I have come to doubt that
target evaluation is even a meaningfull concept in python, so
maybe in order that I can understand how you come to that claim,
can you explain what a target evaluates to? So in a statement like

col['t'] = exp

What is the evaluation of col['t']?
>Shouldn't the language reference be clear enough to be understandable
without the help of byte code generated by the reference implementation?

Normally, code experiments, such as you attempted, should be enough. But,
as I said, trying to document mostly invisibly behavior is more difficult.
Based on looking at the byte code, I would put it this way: the difference
between assignment with duplicated source and target expressions and
augmented assigment with the same expression is that the interpreter
eliminates as much duplicate calculation as possible. But precisely what
that mean seems a bit interpreter dependent to me.

Should we expect a reference writen in (ambiguous) natural language to be
unambiguously clear? I think not; that is why formal grammers and
notations were invented. I learned Python with the doc in one window and
the interactive interpreter in another.
But there seems to be no formal system used for explaning the behaviour.
So I think one should take extra care in the language reference to
write in a way that is as clear as possible. IMO there is room for
improvement in the augmented assignment part. That is why I proposed
that the language reference would state that the primary of the
target would be evaluated only once, along with the index/key or slice.
Such wording avoids the notion of a target evaluation, which seems
trouble some and explains what kind of optimisations can be expected.

--
Antoon Pardon
Jul 18 '06 #28
Antoon Pardon wrote:
>
Now maybe I'm just not bright enough, so maybe you can explain what
something like col['t'] is evaluated to in a statement like:

col['t'] = exp
In the notation given earlier, let us say that it would be this:

namespace = col
setitem(namespace, "t", exp)

Note that in my notation I use the term namespace to apply to actual
namespaces (locals, globals, classes, objects) as well as what I call
"itemspaces" (things which store items). In the above, we still obtain
the namespace first, but we don't attempt to fetch the item previously
stored at item "t" (if any was present).

[...]
Well maybe I'm missing your point entirely but would you be so friendly,
as to rephrase the following statements too:

a = a + b
This is where it gets interesting, even though it should not appear to
do so. We might consider writing this as follows:

setname(ns_target, "a", add(getname(ns_source, "a"), b))

Here, I've skipped over how b is obtained, but it'd be something like
getname(locals, "b") or getname(globals, "b"). Since recent versions of
Python perform some special tricks to disallow assignments involving
mixed namespaces (considering a is global and that the above code is in
a function), we probably can assume the same namespace is at work. But
really, the most pertinent question relates to the evaluation of the
namespaces themselves: a simple name binding operation involves the
local or global namespaces which can be obtained without side-effects.
Thus, the simple assignment above is mostly equivalent to the augmented
assignment...

a += b

....since ns_target == ns_source and their evaluation can be considered
free of side-effects. In other words, we can rewrite it thus:

assert ns_target == ns_source
namespace = ns_target
setname(namespace, "a", add(getname(namespace, "a"), b))
a.b = a.b + c
We could consider writing this:

setattr(a, "b", add(getattr(a, "b"), c))

Once again, I've skipped over how c is obtained, since it isn't central
to the assignment. As above, it appears that the namespace is the same
on both sides of the assignment (given Python's assignment rules),
although by using an opaque identifier the above example doesn't really
permit us to illustrate this. We could expand the identifier a to
investigate further:

setattr(getname(ns_target, "a"), "b", add(getattr(getname(ns_source,
"a"), "b"), c))

Which according to the above reduces to this:

setattr(getname(namespace, "a"), "b", add(getattr(getname(namespace,
"a"), "b"), c))

And to this:

ns_a = getname(namespace, "a")
setattr(ns_a, "b", add(getattr(ns_a, "b"), c))

Which shows that the augmented assignment is probably equivalent. See
below for a better example.
a[b] = a[b] + c
And again, we could consider writing this:

setitem(a, b, add(getitem(a, b), c))

Once again, I've skipped over how b and c are obtained. Consider this
as an exercise! ;-) And as noted above, the example doesn't really
permit us to illustrate this point - you could expand the terms and
produce something that suggests that this simple case is equivalent to
an augmented assignment.
Because as far as I can see the rephrasing of these statements will
result in exactly the same results as their augmented assignment
counter parts. So following your logic above, we would have to
come to the conclusion that the "targets" of these statements are
evaluated once too. This while the language reference suggests
they are evaluated twice.
Quoting from the reference: "In the augmented version, x is only
evaluated once." Let us look in depth at some of the unillustrated
issues from above.
IMO the language reference sugest that the stament:

a[i] += [b]
Here, we can be sure that the same namespace is involved:

namespace = a
setitem(namespace, i, add(getitem(namespace, i), [b]))
is more effective than:

a[i] = a[i] + [b]
Whilst I, knowing that a appears twice and believing that a always
produces the same result, might write the same as above, let us
consider a more complicated expression instead of a (where the function
f returns a list):

f()[i] = f()[i] + [b]

Here, since we can make no guarantees that f() evaluates to the same
thing every time it is called, we must evaluate it twice to honour the
intent of the statement:

setitem(f(), i, add(getitem(f(), i), [b]

To see this clearly...

f1 = f()
f2 = f()
setitem(f1, i, add(getitem(f2, i), [b])

Meanwhile, in the case of the augmented assignment...

f()[i] += [b]

....the author has more or less stated that he/she wants f() to be
evaluated only once - anything else would be surprising:

namespace = f()
setitem(namespace, i, add(getitem(namespace, i), b))
IMO it suggests that the augmented assignment version is
just as effective as:

a[i].append(b)

Because, i hope, we agree that a[i] is evaluated only
once here!
The rule is that if the namespace (what you call the primary, I think)
is mentioned once, it is evaluated only once.

[...]
Yes, and what you are doing here is eliminating the double evaluation
of the primary. In a statement like a.f().x = ... the a.f() part
is the primary according to the assignment reference.
Indeed. The language reference uses very simple opaque identifiers to
supposedly illustrate its point but, as we saw above, such examples
actually obstruct the path to understanding. That a simple identifier x
obtained from some namespace is not evaluated twice is almost
tangential to understanding the principles involved.

Paul

Jul 18 '06 #29

"Antoon Pardon" <ap*****@forel.vub.ac.bewrote in message
news:sl********************@rcpc42.vub.ac.be...
On 2006-07-17, Terry Reedy <tj*****@udel.eduwrote:
> Or, whether docs (and reasonable interpretation thereof) and
implementation match, which I claim they do it this case.
The claim, in reference to the CPython implementation, that you refer to
below.
Well this dispute seems to boil down what is and what is not
involved in evaluating a target,
Yes, and that is interpreter/implementation dependent.
and I have come to doubt that
target evaluation is even a meaningfull concept in python,
Abstractly, it is whatever the interpreter does before it actually attaches
an object to the name or slot. Again, the details are interpreter
dependent.
so maybe in order that I can understand how you come to that claim,
I looked at the CPython bytecode and saw that for augmented assigment, it
saved on the stack the internal information it needed for get and set
instead of recalculating it after the operation. This is what I expected
and what I think the docs imply.
can you explain what a target evaluates to?
The internal information the interpreter needs to do the assignment
(binding).
So in a statement like
col['t'] = exp
What is the evaluation of col['t']?
Try some personal introspection. When you act as a Python interpreter,
what do you do?
So I think one should take extra care in the language reference to
write in a way that is as clear as possible. IMO there is room for
improvement in the augmented assignment part.
I already agreed. I might someday suggest a change.

Terry Jan Reedy

Jul 18 '06 #30
>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:
>APOn 2006-07-17, Piet van Oostrum <pi**@cs.uu.nlwrote:
>>>>>>>Antoon Pardon <ap*****@forel.vub.ac.be(AP) wrote:
APOn 2006-07-14, Piet van Oostrum <pi**@cs.uu.nlwrote:
>>>
>Just read what it says. `It is only evaluated once' is quite clear I would
>say.
APIf it is so clear, why don't you explain it?
>>>
I have done that in another thread.
>APBefore I respond I would like you to be more specific
APabout where you gave that explanation. The subject
APof the thread would be fine or a message-id perhaps.
<m2************@ordesa.cs.uu.nl(In fact it is in this same thread).
--
Piet van Oostrum <pi**@cs.uu.nl>
URL: http://www.cs.uu.nl/~piet [PGP 8DAE142BE17999C4]
Private email: pi**@vanoostrum.org
Jul 18 '06 #31
On 2006-07-18, Terry Reedy <tj*****@udel.eduwrote:
>
"Antoon Pardon" <ap*****@forel.vub.ac.bewrote in message
news:sl********************@rcpc42.vub.ac.be...
>On 2006-07-17, Terry Reedy <tj*****@udel.eduwrote:
>> Or, whether docs (and reasonable interpretation thereof) and
implementation match, which I claim they do it this case.

The claim, in reference to the CPython implementation, that you refer to
below.
>Well this dispute seems to boil down what is and what is not
involved in evaluating a target,

Yes, and that is interpreter/implementation dependent.
Well I can agree that some things are implemantation dependent.
But shouldn't a language reference describe behaviour in terms or
implementation independent things?
>and I have come to doubt that
target evaluation is even a meaningfull concept in python,

Abstractly, it is whatever the interpreter does before it actually attaches
an object to the name or slot. Again, the details are interpreter
dependent.
Well I can sort of agree with this. IMO there are two possible views
here, (not necessarily mutual exclusive) But if you agree with the first
IMO the language reference shouldn't make use of the concept to
describe behaviour.

1) target evaluation is not a pythonic concept.

2) It is whatever the interpreter does before it actually
attaches an object, to a name/slot/... (in a primary).
>so maybe in order that I can understand how you come to that claim,

I looked at the CPython bytecode and saw that for augmented assigment, it
saved on the stack the internal information it needed for get and set
instead of recalculating it after the operation. This is what I expected
and what I think the docs imply.
>can you explain what a target evaluates to?

The internal information the interpreter needs to do the assignment
(binding).
Yes but that seems to be an entirly internal interpreter affair. If
you stay at the level of the byte code, you will not find any opcode
that will result in or manipulate a target evaluation.

If you look at the language reference for the assignment, you will
not find the notion of a target evaluation mentioned there either.

That is because as far as I can see, an assignment is essentially
a ternary operation in python. An assignment needs a (name)space/scope,
an index/key/name and an object and those three are combined into
an assignment. Sometimes it looks like only two elements
are given, but that is because the space is implicit in cases
of a rebinding, (the STORE_GLOBAL and STORE_FAST opcode).

Talking about a target evaluation IMO only makes sense if
you view an assigment as a binary operation.
>So in a statement like
col['t'] = exp
What is the evaluation of col['t']?

Try some personal introspection. When you act as a Python interpreter,
what do you do?
I do the evaluation of the target in the __setitem__ method
(or the STORE_SUBSCR opcode).

Let as look what the compilor makes of it:
[blank lines added for clarity]
>>dis(compile("col['t'] = exp", '', 'single'))
1 0 LOAD_NAME 0 (exp)
3 LOAD_NAME 1 (col)
6 LOAD_CONST 0 ('t')

9 STORE_SUBSCR

The bytecodes at 0, 3 and 6 don't do any evaluation, they
just put things on the stack.

So what does the STORE_SUBSCR at location 9 do (or __setitem__)?

Well it will first do a number of preparations, like searching
for a bucket in a dictionary or a node in a tree or list, may
be even create one if a suitable wasn't available. And after
that is done, the object will be somehow attached.

So IMV those preparation before the attachment, belong to
whatever the interpreter does before it actually attaches
an object to a name/slot.

So the evaluation of the target is part of what is done by
STORE_SUBSCR or __setitem__.

Now you can object to the fact that I have divided the work
within an opcode. But if you do that, there seems to be
no place left to talk about a target evaluation in this
example.

So as a conclusion I would say one has two options:

1) View target evaluations as not a pythonic concept

2) Accept that target evaluation is done by STORE_SUBSCR/__setitem__

--
Antoon Pardon
Jul 19 '06 #32
Antoon Pardon wrote:
The language reference doesn't talk about objects. And IMO you
should be carefull if you want to use the word "object" here.
In the line: "foo += 1", you can't talk about the object foo,
since foo will
possibly
be bound to a different object after the assignment
than it was bound to before.
witness
>>class Foo(list) :
def __iadd__(self,other) :
self.append(other)
return self

>>bar = foo = Foo()
foo += 1
foo is bar
True
>>foo
[1]

while of course
>>bar = foo = 0
foo += 1
foo is bar
False

Best, BB
--
"On naît tous les mètres du même monde"
Jul 19 '06 #33

"Antoon Pardon" <ap*****@forel.vub.ac.bewrote in message
news:sl********************@rcpc42.vub.ac.be...
>Try some personal introspection. When you act as a Python interpreter,
what do you do?

I do the evaluation of the target in the __setitem__ method
(or the STORE_SUBSCR opcode).
I meant mentally, in your mind ;-)
>
Let as look what the compilor makes of it:
[blank lines added for clarity]
>>>dis(compile("col['t'] = exp", '', 'single'))

1 0 LOAD_NAME 0 (exp)
3 LOAD_NAME 1 (col)
6 LOAD_CONST 0 ('t')

9 STORE_SUBSCR

The bytecodes at 0, 3 and 6 don't do any evaluation, they
just put things on the stack.

So what does the STORE_SUBSCR at location 9 do (or __setitem__)?

Well it will first do a number of preparations, like searching
for a bucket in a dictionary or a node in a tree or list, may
be even create one if a suitable wasn't available. And after
that is done, the object will be somehow attached.

So IMV those preparation before the attachment, belong to
whatever the interpreter does before it actually attaches
an object to a name/slot.

So the evaluation of the target is part of what is done by
STORE_SUBSCR or __setitem__.

Now you can object to the fact that I have divided the work
within an opcode.
But I won't. The amount of duplication that can be factored out with
augmented assignment depends on the granularity of operations. And the
granularity of operations depends on the interpreter. Hence my claim that
the details are necessarily interpreter dependent.

Terry Jan Reedy

Jul 19 '06 #34
On 2006-07-19, Terry Reedy <tj*****@udel.eduwrote:
>
"Antoon Pardon" <ap*****@forel.vub.ac.bewrote in message
news:sl********************@rcpc42.vub.ac.be...
>So IMV those preparation before the attachment, belong to
whatever the interpreter does before it actually attaches
an object to a name/slot.

So the evaluation of the target is part of what is done by
STORE_SUBSCR or __setitem__.

Now you can object to the fact that I have divided the work
within an opcode.

But I won't. The amount of duplication that can be factored out with
augmented assignment depends on the granularity of operations.
I can agree with that. But until now each time I tried to
suggest that the STORE_SUBSCR or __setitem__ are involved
in the evaluation of the target, I was told I shouldn't
look at things that way.
And the
granularity of operations depends on the interpreter. Hence my claim that
the details are necessarily interpreter dependent.
Well my impression from reading the language reference is that
it suggests that a greater amount of duplication is factored out
than the granularity of the CPython implementation allows.

But since you already agreed that there is room for improvement
in the augmented assignment reference I will leave it at that.

I would like to thank everybody for their contributions, but I
think everything worth while has been said, so I will no longer
contribute to this thread.

--
Antoon Pardon
Jul 20 '06 #35

"Antoon Pardon" <ap*****@forel.vub.ac.bewrote in message
news:sl********************@rcpc42.vub.ac.be...
On 2006-07-19, Terry Reedy <tj*****@udel.eduwrote:
>>
But I won't. The amount of duplication that can be factored out with
augmented assignment depends on the granularity of operations.

I can agree with that. But until now each time I tried to
suggest that the STORE_SUBSCR or __setitem__ are involved
in the evaluation of the target, I was told I shouldn't
look at things that way.
I don't think that I have said that, but only that a 'pragmatic' definition
is 'what actually is factored out'.
>And the
granularity of operations depends on the interpreter. Hence my claim
that
the details are necessarily interpreter dependent.

Well my impression from reading the language reference is that
it suggests that a greater amount of duplication is factored out
than the granularity of the CPython implementation allows.
Perhaps all that is meant to be promised is that function calls are not
duplicated, but if so, that could be much clearer. I would have to look at
more different types of expressions and their corresponding byte code
before suggesting anything particular.
I would like to thank everybody for their contributions, but I
think everything worth while has been said, so I will no longer
contribute to this thread.
Agreed.

Terry Jan Reedy

Jul 20 '06 #36

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

23
by: Paul Rubin | last post by:
OK, I want to scan a file for lines matching a certain regexp. I'd like to use an assignment expression, like for line in file: if (g := re.match(pat, line)): croggle(g.group(1)) Since...
31
by: surfunbear | last post by:
I've read some posts on Perl versus Python and studied a bit of my Python book. I'm a software engineer, familiar with C++ objected oriented development, but have been using Perl because it is...
9
by: Dieter Vanderelst | last post by:
Dear all, I'm currently comparing Python versus Perl to use in a project that involved a lot of text processing. I'm trying to determine what the most efficient language would be for our...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.