Is 'everything' a refrence or isn't it?

KraftDiner wrote:

I was under the assumption that everything in python was a refrence...

so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)
so it seems that in order to do this I need to code it like:

lst = [1,2,3]
for i in range(len(lst)):
if lst[i] == 2:
lst[i]=4
print lst

Have I misunderstood something?

To give a short answer: yes.

And now into the details assuming, that your misunderstanding comes from
programming in C before evaluating Python:

If you write
i = 4
following happens:
an integer literal 4 gets into the scope of Python script as an object
of type integer and can be from now on reached using the identifier i .
So i is in this context a _kind of_ pointer/reference to the value
'4', but ... in Python sense not in C sense.

if you write
for i in lst:
then the identifier i 'points' to values in lst, but else as in C
where you can use a pointer to change the value it points to by
assigning to *i or where you can use a reference to change a value it
references, in Python you don't change a value of i because i is an
identifier and therefore has no value in sense of C - it is only a name
used to reach a value.

When you write within the 'for i in lst:' loop
i = 4
you just use the identifier i as a 'pointer' to another Python object
as it was pointing to before.

Let's consider a list.
The 'identifier'(i.e. construct) lst[i] pointing to a list element
behaves in Python like what you understand in C as a reference.

This is the main difference between the meaning of = assignment in
Python and in C and it usually results in problems with understanding it
when someone has programmed C and expects from Python to be like C.

The core of the trouble is probably different understanding of the word
'reference', which used in different contexts (e.g. Python and C) means
different things. Therefore as Scott David Daniels says
"There is a reason people talk about names and bindings."

Claudio

Jan 4 '06 #7

On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:

I was under the assumption that everything in python was a refrence...

so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)

See, this confusion is precisely why I get the urge to slap people who
describe Python as "call by reference". It isn't.

It isn't "call by value" either -- Python never copies objects unless you
explicitly tell it to.

It is "call by object" -- you pass around *objects*. Internally, this is
quite fast, because the entire object doesn't need to be moved, only
pointers to objects, but you don't get the behaviour of either call by
reference or call by value.

In the above code, i is a name bound one at a time to the ints [1,2,3].
When you re-assign i to 4, that doesn't change the object 2 into the
object 4, because ints are immutable. Only the name i is rebound to a new
object 4. That doesn't change objects like lst which include 2 inside them.

See this for more detail:

http://effbot.org/zone/call-by-object.htm

--
Steven.

Jan 4 '06 #8

On Wed, 04 Jan 2006 20:05:38 +0100, Christian Tismer wrote:

KraftDiner wrote:
I was under the assumption that everything in python was a refrence...

This is true.

No it is not.

so if I code this:
lst = [1,2,3]

lst is a reference to a list that holds references to 1, 2, and 3
for i in lst:

i is a reference to every element in the list, one ofter the other.
But this is not a ref to lst any longer, the list gets dereferenced
before getting at its contents, and only references to that are
returned.

See what confusing, complicated hoops people have to jump through to
hammer the round peg of Python's behaviour into the square peg of "call by
reference"?

http://effbot.org/zone/call-by-object.htm

Python does not have references or pointers, except internally where
Python coders can not get to them. It has names and objects. Keep thinking
about "call by reference" and you just confuse yourself and others. Think
about names and objects and it is simple and straight-forward.

--
Steven.

Jan 4 '06 #9

Steven D'Aprano wrote:

On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:

I was under the assumption that everything in python was a refrence...

so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)

See, this confusion is precisely why I get the urge to slap people who
describe Python as "call by reference". It isn't.

It isn't "call by value" either -- Python never copies objects unless you
explicitly tell it to.

It is "call by object" -- you pass around *objects*. Internally, this is
quite fast, because the entire object doesn't need to be moved, only
pointers to objects, but you don't get the behaviour of either call by
reference or call by value.

In the above code, i is a name bound one at a time to the ints [1,2,3].
When you re-assign i to 4, that doesn't change the object 2 into the
object 4, because ints are immutable. Only the name i is rebound to a new
object 4. That doesn't change objects like lst which include 2 inside them.

See this for more detail:

http://effbot.org/zone/call-by-object.htm

It seems to be hard to explain it all in a straighforward way without
using pictures showing what happens when an assignment is processed
demonstrating what is what and how it is called. Probably a small nice
movie could be here the right medium to be handed over to Python newbies
coming from C to give some hints towards proper understanding. Replacing
one word with another is not sufficient to avoid confusion as the
concept is probably not always easy to grasp (I have still problems to
get the idea how it works 'inside' with this local and global
dictionaries and so on).
Lack of knowledge about a good reference was the reason why I decided to
give a reply here inspite of the fact, that so many other postings were
already there, but none (including mine) which covered all aspects, so
that I could say: "Wow! It is an excelent explanation and there is
nothing which must be added to it!".
Is there anywhere an ultimative enlightening explanation of the concept
of identifiers and variables in Python for use in case the next posting
of that kind hits comp.lang.python and it sure will as they are coming
in quite regular.
http://effbot.org/zone/call-by-object.htm is sure an attempt of a good
explanation, but in my eyes still nothing what immediately enlightens.

Claudio

Jan 4 '06 #10

Peter Hansen

Steven D'Aprano wrote:

Python does not have references or pointers, except internally where
Python coders can not get to them. It has names and objects. Keep thinking
about "call by reference" and you just confuse yourself and others. Think
about names and objects and it is simple and straight-forward.

I won't argue the point, but I would point out that the term "name" is
insufficient for whatever it is that is stored inside a list.

What do you call it, if not a reference? The word "binding" isn't
really appropriate here, as it is easily confused with the operation of
binding (i.e. usually what assignment does).

-Peter

Jan 4 '06 #11

Steven D'Aprano <st***@REMOVETHIScyber.com.au> writes:

On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:
I was under the assumption that everything in python was a refrence...
so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst
I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.) See, this confusion is precisely why I get the urge to slap people who
describe Python as "call by reference". It isn't.

Except this doesn't have *anything at all* to do with python being (or
not being) call by reference. This is a confusion about name binding
vs. assignment to a variable. The proper people to slap around for
this case are the ones who talk about assignment to a variable.
It is "call by object" -- you pass around *objects*. Internally, this is
quite fast, because the entire object doesn't need to be moved, only
pointers to objects, but you don't get the behaviour of either call by
reference or call by value.

No, you get *exactly* that behavior from call by reference when you
start passing objects around by reference. If I declare a C object as
"struct foo bar" and do the C "call-by-reference" hack of passing
&bar, I get the exact same behavior I get when I pass an object
referenced by bar to a Python subroutine.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

Jan 4 '06 #12

Peter Hansen <pe***@engcorp.com> writes:

Steven D'Aprano wrote:
Python does not have references or pointers, except internally where
Python coders can not get to them. It has names and objects. Keep thinking
about "call by reference" and you just confuse yourself and others. Think
about names and objects and it is simple and straight-forward. I won't argue the point, but I would point out that the term "name" is
insufficient for whatever it is that is stored inside a list.

Correct. What's stored in a list is a reference.
What do you call it, if not a reference? The word "binding" isn't
really appropriate here, as it is easily confused with the operation
of binding (i.e. usually what assignment does).

To clarify, assignement statements in python are bindings, and don't
do assign values to a variables. Assignment in other languages do that
- which is why people get confused when exposed to what Python does.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

Jan 4 '06 #13

Steven D'Aprano wrote:

On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)

See, this confusion is precisely why I get the urge to slap people who
describe Python as "call by reference". It isn't.

On reading back over my post, I realise that it might
sound like I was mad at KraftDiner. My apologies -- I'm
not, I feel (s)he is the victim of incorrect
information here, not the culprit.

After all, as a Python newbie, how is KraftDiner
supposed to know that when people say "Python is call
by reference", what they actually mean is "Um, except
that it doesn't behave like any call by reference
language you're likely to have used before, and
sometimes it behaves more like call by value"?
--
Steven.

Jan 5 '06 #14

Donn Cave

In article <pa****************************@REMOVETHIScyber.co m.au>,
Steven D'Aprano <st***@REMOVETHIScyber.com.au> wrote:

See what confusing, complicated hoops people have to jump through to
hammer the round peg of Python's behaviour into the square peg of "call by
reference"?

http://effbot.org/zone/call-by-object.htm

Python does not have references or pointers, except internally where
Python coders can not get to them. It has names and objects. Keep thinking
about "call by reference" and you just confuse yourself and others. Think
about names and objects and it is simple and straight-forward.

I agree that it can be simple and straightforward. The particular
terms we use have nothing to do with it, though. Most of us need
a word or two to attach to the concept, because that's how our
brains work, but there are no natural words in the English language
that precisely express Python concepts, and we just have to make
do with one or another of several possible. It's silly to insist
that one particular word resolves all confusions, and another word
creates them. It's sort of unfortunate to let this crusade get to
the point where you're upbraiding people who know a great deal about
the matter because they don't use your favorite word.

As other followups have pointed out, the problem is simply not
understanding what "=" does. While in some obscure way this
may be related to "call by reference" etc., that's certainly not
a helpful digression. To explain what i = 4 does, we can say

creates a named reference "i" to 4
binds the name "i" to 4
etc.

Any of these is delightfully clear if you already understand it,
which is where most of the born again believers are coming from.
People who don't already understand what's being explained, need
to have a little patience and make sure to test their understanding
with a few experiments.

Donn Cave, do**@u.washington.edu

Jan 5 '06 #15

Claudio Grondi wrote:

[snip]

See, this confusion is precisely why I get the urge to slap people who
describe Python as "call by reference". It isn't.

It isn't "call by value" either -- Python never copies objects unless you
explicitly tell it to.

It is "call by object" -- you pass around *objects*. Internally, this is
quite fast, because the entire object doesn't need to be moved, only
pointers to objects, but you don't get the behaviour of either call by
reference or call by value.

....
It seems to be hard to explain it all in a straighforward way without
using pictures showing what happens when an assignment is processed
demonstrating what is what and how it is called.
That's no different from call by reference or call by
value. If you think back to when you were first
learning about pointers, that was confusing too.

Probably a small nice
movie could be here the right medium to be handed over to Python newbies
coming from C to give some hints towards proper understanding. Replacing
one word with another is not sufficient to avoid confusion as the
concept is probably not always easy to grasp (I have still problems to
get the idea how it works 'inside' with this local and global
dictionaries and so on).

No, just saying "call by object" is not sufficient, but
it is vital. If you say "call by reference", the term
comes with a lot of mental baggage -- people think they
understand what call by reference means, how it
behaves, and as soon as they hear CBR they think they
understand how Python behaves. They think "Oh, Python
is CBR just like language Foo, so it must behave just
like language Foo, and this idiom works in language
Foo, so it must work in Python too."

And sometimes it does appear to work, which just makes
it all the more puzzling when it doesn't.

When they hear CBO the term is new and different and
doesn't come with any mental frame, so instead of
thinking they understand Python's behaviour, they think
"Call by what now? How does that work?" instead of
making incorrect assumptions.

--
Steven.

Jan 5 '06 #16

Peter Hansen wrote:

Steven D'Aprano wrote:
Python does not have references or pointers, except internally where
Python coders can not get to them. It has names and objects. Keep
thinking
about "call by reference" and you just confuse yourself and others. Think
about names and objects and it is simple and straight-forward.

I won't argue the point, but I would point out that the term "name" is
insufficient for whatever it is that is stored inside a list.

What do you call it, if not a reference? The word "binding" isn't
really appropriate here, as it is easily confused with the operation of
binding (i.e. usually what assignment does).

I just call it an item of the list. The fact that
CPython uses an array of pointers to objects for lists
is an implementation detail. Unfortunately, while
"reference" is a nice generic English word, in this
context it has too much mental baggage to safely use
around newbies. (Experience Pythonistas is another
story of course.)
--
Steven.

Jan 5 '06 #17

Mike Meyer wrote:

Steven D'Aprano <st***@REMOVETHIScyber.com.au> writes:
On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:
I was under the assumption that everything in python was a refrence...
so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst
I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)
See, this confusion is precisely why I get the urge to slap people who
describe Python as "call by reference". It isn't.

Except this doesn't have *anything at all* to do with python being (or
not being) call by reference. This is a confusion about name binding
vs. assignment to a variable. The proper people to slap around for
this case are the ones who talk about assignment to a variable.

Mike, you are wrong. Not about talk about "assigning to
variables" being harmful, I agree with that. But this
confusion is *exactly* about the call by reference
misunderstanding. How do I know this?

Because the Original Poster said so! He said, to
paraphrase, "Hey, I thought Python was call by
reference, but I tried this, and it didn't work, what
gives???"

This is not an isolated case either.

You can't get any clearer than that. Talk about call by
reference leads to confusion and incorrect assumptions
about Python's behaviour. End of story.

It is "call by object" -- you pass around *objects*. Internally, this is
quite fast, because the entire object doesn't need to be moved, only
pointers to objects, but you don't get the behaviour of either call by
reference or call by value.

No, you get *exactly* that behavior from call by reference when you
start passing objects around by reference.

Who cares what Python does internally? It is strictly
irrelevant whether Python internally passes around
pointers, or copies values, or sometimes one or the other.
If I declare a C object as
"struct foo bar" and do the C "call-by-reference" hack of passing
&bar, I get the exact same behavior I get when I pass an object
referenced by bar to a Python subroutine.

It is not a question of whether Python's behaviour is
*sometimes* the same as call by reference. The question
whether Python's behaviour is *always* the same as CBR,
and it is not. Regardless of the reasons, Python does
not behave like CBR in other languages, and it is
harmful to imply that it does.

Likewise, you shouldn't argue that Python is obviously
call by value just because *sometimes* it has the same
behaviour as CBV.

I mean, come on! A whole bunch of people argue black
and blue that Python "obviously" is call by reference,
and another vocal bunch argue that it is "obviously"
call by value. Isn't that the tiniest hint that Python
-- not the underlying C/Java/Python/whatever
implementation, but Python the language --- does not
fall into either camp, but is doing something that
sometimes looks like one and sometimes like the other?

Just google on "Python is call by value" and "Python is
call by reference" -- including quotes -- to see this
argument come up time and time again, year after year
after year. And all because of some false dichotomy
that CBR and CBV are the only two valid descriptions of
a computer language.

--
Steven.

Jan 5 '06 #18

Mike Meyer wrote:

Peter Hansen <pe***@engcorp.com> writes:

Steven D'Aprano wrote:
Python does not have references or pointers, except internally where
Python coders can not get to them. It has names and objects. Keep thinking
about "call by reference" and you just confuse yourself and others. Think
about names and objects and it is simple and straight-forward.

I won't argue the point, but I would point out that the term "name" is
insufficient for whatever it is that is stored inside a list.

Correct. What's stored in a list is a reference.

Nonsense. What is stored in the list is an object.
Python doesn't have pointers. You are confusing the
underlying C implementation, which implements lists as
an array of pointers, with the high-level Python code,
which doesn't even know what a pointer is. It only has
objects.

Since we are discussing the perspective from a Python
programmer's point of view, it is irrelevant what the C
implementation does. If lists held references, you
could do this:

x = 5
L = [1, 2, x]
# the third item points to the same memory location
# as x; changing it will also change x
L[2] = 7
assert x == 7

which is the behaviour that "call by reference"
implies. But it doesn't work that way in high-level
Python code.

The *reason* it doesn't work is that the object 5 is
immutable. If you only ever passed mutable objects
around, you would think Python was call by reference.
And if you only ever passed immutable objects around,
you would think Python was call by value. But since
Python has both sorts of objects, it displays both
sorts of behaviour.
--
Steven.

Jan 5 '06 #19

Aahz

In article <43**************@REMOVEMEcyber.com.au>,
Steven D'Aprano <st***@REMOVEMEcyber.com.au> wrote:

Nonsense. What is stored in the list is an object. Python doesn't have
pointers. You are confusing the underlying C implementation, which
implements lists as an array of pointers, with the high-level Python
code, which doesn't even know what a pointer is. It only has objects.

Wrong. Python has objects and targets. Names are a subset of targets.
A target is anything that can contain a binding (or reference, whichever
term you prefer) to an object; lists are arrays of targets.

No matter how you slice it ;-), a list does not actually hold objects, it
only holds references to objects.
--
Aahz (aa**@pythoncraft.com) <*> http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing." --Alan Perlis

Jan 5 '06 #20

Steven D'Aprano <st***@REMOVEMEcyber.com.au> writes:

Mike Meyer wrote:
Steven D'Aprano <st***@REMOVETHIScyber.com.au> writes:
On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:
I was under the assumption that everything in python was a refrence...
so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst
I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)
See, this confusion is precisely why I get the urge to slap people who
describe Python as "call by reference". It isn't. Except this doesn't have *anything at all* to do with python being
(or
not being) call by reference. This is a confusion about name binding
vs. assignment to a variable. The proper people to slap around for
this case are the ones who talk about assignment to a variable.

Mike, you are wrong. Not about talk about "assigning to variables"
being harmful, I agree with that. But this confusion is *exactly*
about the call by reference misunderstanding. How do I know this?

Because the Original Poster said so! He said, to paraphrase, "Hey, I
thought Python was call by reference, but I tried this, and it didn't
work, what gives???"

And he's right, and you're wrong. Look at the *code*. There isn't a
single call in it. He may have said "call by reference", but his
example code didn't have anything to do with call by rereference, and
he didn't demonstrate any confusion about the semantics of parameter
passing. He *did* demonstrate confusion about the semantics of the
assignment statement.

It is "call by object" -- you pass around *objects*. Internally, this is
quite fast, because the entire object doesn't need to be moved, only
pointers to objects, but you don't get the behaviour of either call by
reference or call by value.

No, you get *exactly* that behavior from call by reference when you
start passing objects around by reference.

Who cares what Python does internally? It is strictly irrelevant
whether Python internally passes around pointers, or copies values, or
sometimes one or the other.

Who said anything about what Python does internally? I agree, it's
strictly irrelevant. What matters is the *behavior*.

If I declare a C object as
"struct foo bar" and do the C "call-by-reference" hack of passing
&bar, I get the exact same behavior I get when I pass an object
referenced by bar to a Python subroutine.

It is not a question of whether Python's behaviour is *sometimes* the
same as call by reference. The question whether Python's behaviour is
*always* the same as CBR, and it is not.

You're wrong. Python's behavior is *always* the same as CBR, once you
identify what Python is passing a reference to.
Likewise, you shouldn't argue that Python is obviously call by value
just because *sometimes* it has the same behaviour as CBV.
Actually, it's either always the same as CBV, or never the same as
CBV. It's always the same as CBV because you always pass a value - and
the value is always a reference. But every calling mechanism requires
that you always pass a value. It's what the value is that determines
the calling mechanism. You pass a reference for CBR, and a name for
call by name.
I mean, come on! A whole bunch of people argue black and blue that
Python "obviously" is call by reference, and another vocal bunch argue
that it is "obviously" call by value.
I don't think I've seen anyone arguing that Python is
call-by-value. There are people who argue that it's sometimes one and
soemtimes the other, but that's mostly because they have incorrectly
identified what Python is passing a reference to.
Just google on "Python is call by value" and "Python is call by
reference" -- including quotes -- to see this argument come up time
and time again, year after year after year. And all because of some
false dichotomy that CBR and CBV are the only two valid descriptions
of a computer language.

I tried the google, and I don't see a lot of arguments. I see people
declaiming one way or another, and not very many of those. I don't
think it's an apparent dichotomy has much to do with it. I know there
are other things than CBV and CBR, and even understand Jensen's
device.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

Jan 5 '06 #21

Shalabh Chaturvedi

KraftDiner wrote:

I was under the assumption that everything in python was a refrence...

For a good writeup on the 'name-binding' behavior others are talking about:

http://starship.python.net/crew/mwh/...jectthink.html

Cheers,
Shalabh

Jan 5 '06 #22

Stuart D. Gathman

On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:

I was under the assumption that everything in python was a refrence...

so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)
...
Have I misunderstood something?

It might help to do a translation to equivalent C:

int _i1 = 1;
int _i2 = 2;
int _i3 = 3;
int _i4 = 4;
int* lst[NLST] = { &_i1,&_i2,&_i3 };
int _idx; /* internal iterator */
for (_idx = 0; _idx < NLST; ++_idx) {
int *i = lst[_idx];
if (*i == *_i2)
i = &_i4;
}
for (_idx = 0; _idx < NLST; ++_idx)
printf("%d\n",*lst[_idx]);

--
Stuart D. Gathman <st****@bmsi.com>
Business Management Systems Inc. Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

Jan 5 '06 #23

Steven D'Aprano <st***@REMOVEMEcyber.com.au> writes:

Mike Meyer wrote:
Peter Hansen <pe***@engcorp.com> writes:
Steven D'Aprano wrote:
Python does not have references or pointers, except internally where
Python coders can not get to them. It has names and objects. Keep thinking
about "call by reference" and you just confuse yourself and others. Think
about names and objects and it is simple and straight-forward.
I won't argue the point, but I would point out that the term "name" is
insufficient for whatever it is that is stored inside a list. Correct. What's stored in a list is a reference.

Nonsense. What is stored in the list is an object. Python doesn't have
pointers. You are confusing the underlying C implementation, which
implements lists as an array of pointers, with the high-level Python
code, which doesn't even know what a pointer is. It only has objects.

I didn't say a single word about C, or pointers, or anything else to
do with the implementation.
Since we are discussing the perspective from a Python
programmer's point of view, it is irrelevant what the C implementation
does. If lists held references, you could do this:

x = 5
L = [1, 2, x]
# the third item points to the same memory location
# as x; changing it will also change x
L[2] = 7
assert x == 7

which is the behaviour that "call by reference" implies. But it
doesn't work that way in high-level Python code.
Now you're talking nonsense. Call by reference is about *parameter
passing*, not about the behavior of assignment statements or
lists. How the above code behaves has *nothing* to do with how Python
passes parameters.
The *reason* it doesn't work is that the object 5 is immutable.
The immutability of 5 has *nothing* to do with it. If you replace 5
and 7 with mutable objects, you get the exact same behavior:

x = [5]
L = [1, 2, x]
L[2] = [7]
assert x == set([7])

You seem to be suffering from the newbie confusion of thinking that a
Python assignment statement writes a value into the lvalue. It doesn't
- it makes the lvalue a reference to the object the right side
evaluates to ("binds a name to a value", except sometimes the lvalue
isn't a name). I'm pretty sure you know the difference.

The same thing applies to CBR/CBV. The immutability of the objects
being passed ain't got nothing to do with it. It's the fact that
assignment statements bind values instead of write them into a
variable that makes Python's CBR behavior sometimes act like CBV.
If you only ever passed mutable objects around, you would think
Python was call by reference.
True. But you'd still run into things that you can do with CBR in
other languages that you can't do in Python. That's not because Python
isn't CBR, it's because those other languages have things that Python
doens't have.
And if you only ever passed immutable objects around, you would
think Python was call by value.
You might. Then again, you might also understand the concepts well
enough to realize that there isn't any difference between CBR and CBV
when you're passing immutable objects.
But since Python has both sorts of objects, it displays both sorts
of behaviour.

Again, the mutability (or lack thereof) of the objects being passed
doesn't have anything to do with it. The things you can do in other
languages using CBR that you can't do in Python - like changing the
binding of a name in the callers namespace - you can't do whether the
name is bound to a mutable object or an immutable object. That Python
looks like CBV when you pass immutable objects is because CBR looks
like CBV when you pass immutable objects.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

Jan 5 '06 #24

Bryan Olson

Steven D'Aprano wrote:

Mike Meyer wrote:

[...]

Correct. What's stored in a list is a reference.

Nonsense. What is stored in the list is an object.

According to the Python Language Reference:

Some objects contain references to other objects; these are
called containers. Examples of containers are tuples, lists
and dictionaries.
[http://docs.python.org/ref/objects.html]

--
--Bryan

Jan 5 '06 #25

Bryan Olson

Mike Meyer wrote:

Steven D'Aprano writes:

[...]

Because the Original Poster said so! He said, to paraphrase, "Hey, I
thought Python was call by reference, but I tried this, and it didn't
work, what gives???"

And he's right, and you're wrong. Look at the *code*. There isn't a
single call in it. He may have said "call by reference"

Actually he didn't. Steven added "call by" in his paraphrase.
--
--Bryan

Jan 5 '06 #26

Donn Cave

Quoth Steven D'Aprano <st***@REMOVEMEcyber.com.au>:
| Mike Meyer wrote:
[... dragging out his own terminology hangup ]
|> Except this doesn't have *anything at all* to do with python being (or
|> not being) call by reference. This is a confusion about name binding
|> vs. assignment to a variable. The proper people to slap around for
|> this case are the ones who talk about assignment to a variable.
|
| Mike, you are wrong. Not about talk about "assigning to
| variables" being harmful, I agree with that. But this
| confusion is *exactly* about the call by reference
| misunderstanding. How do I know this?
|
| Because the Original Poster said so! He said, to
| paraphrase, "Hey, I thought Python was call by
| reference, but I tried this, and it didn't work, what
| gives???"

But he really did not SAY "call by reference"! The actual quote:
"I was under the assumption that everything in python was a refrence..."
and
"(After reading that 'everything' is a refrence.)"

Just give it up. And you too, Mike! "variable" is the commonplace
term for the things we fling around in our programs, and you have to
accept that we will use that word whether we're talking about Python,
Java, C or what have you. Of course the semantics of variables - how
they actually relate to values - may be radically different from one
language to another, but it universally plays essentially the same role
in programming (or mathematics), a thing whose value may vary according
to the logic of the program.

C and FORTRAN don't own this word, and it isn't just their version
against the way Python and some other languages do it. Each language
has its own angle on it, and they're all going to be called variables.

Donn Cave, do**@drizzle.com

Jan 5 '06 #27

Stuart D. Gathman wrote:

On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:

I was under the assumption that everything in python was a refrence...

so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)
...
Have I misunderstood something?

It might help to do a translation to equivalent C:

int _i1 = 1;
int _i2 = 2;
int _i3 = 3;
int _i4 = 4;
int* lst[NLST] = { &_i1,&_i2,&_i3 };
int _idx; /* internal iterator */
for (_idx = 0; _idx < NLST; ++_idx) {
int *i = lst[_idx];
if (*i == *_i2)

^-- I have trouble with this line. Is it as is should be? I suppose
it is not.

Claudio i = &_i4;
}
for (_idx = 0; _idx < NLST; ++_idx)
printf("%d\n",*lst[_idx]);

Jan 5 '06 #28

Ben Sizer

Steven D'Aprano wrote:

On reading back over my post, I realise that it might
sound like I was mad at KraftDiner. My apologies -- I'm
not, I feel (s)he is the victim of incorrect
information here, not the culprit.

After all, as a Python newbie, how is KraftDiner
supposed to know that when people say "Python is call
by reference", what they actually mean is "Um, except
that it doesn't behave like any call by reference
language you're likely to have used before, and
sometimes it behaves more like call by value"?

But, if you separate the calling mechanism from the assignment
mechanism, then Python does behave like every other call by reference
language. The problem is that people expect to then be able to change
the value of the referred object with the assignment operator. It's the
assignment semantics that differ from languages such as C++ and Java,
not the calling mechanism. In C++, assignment means copying a value. In
Python, assignment means reassigning a reference.

The unfortunate problem is that people think of 'pass by reference' in
association with 'assignment is a mutating operation', when really the
2 concepts are orthogonal and Python replaces the second with
'assignment is a reference reseating operation'. The only reason I
stress this is because with this in mind, Python is consistent, as
opposed to seeming to have 2 different modes of operation.

--
Ben Sizer

Jan 5 '06 #29

Dan Sommers

On Wed, 04 Jan 2006 22:38:06 -0500,
"Stuart D. Gathman" <st****@bmsi.com> wrote:

On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:
I was under the assumption that everything in python was a refrence...

so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)
...
Have I misunderstood something?
It might help to do a translation to equivalent C: int _i1 = 1;
int _i2 = 2;
int _i3 = 3;
int _i4 = 4;
int* lst[NLST] = { &_i1,&_i2,&_i3 };
Okay so far.
int _idx; /* internal iterator */
for (_idx = 0; _idx < NLST; ++_idx) {
int *i = lst[_idx];
[snip]

That's the C idiom

for i in range(len(lst))

we all complain about here. How about (untested):

/*
iterate over the list, binding i to each item in succession;
_idx is internal to the interpreter;
separate the definition of i from the assignment of i for clarity
*/

int **_idx;
for( _idx = lst; _idx < lst + NLST; ++_idx ) {
int *i;
i = *_idx;

/* compare "the item to which i is bound" to "a constant" */
if( *i == *(&_i2) )
/* rebind i to _i4 */
i = &_i4;
}
for (_idx = 0; _idx < NLST; ++_idx)
printf("%d\n",*lst[_idx]);

for( _idx = lst; _idx < lst + NLST; ++_idx ) {
int *i = *_idx;
printf( "%d\n", *i );
}

Regards,
Dan

--
Dan Sommers
<http://www.tombstonezero.net/dan/>

Jan 5 '06 #30

Claudio Grondi schrieb:

Stuart D. Gathman wrote:
for (_idx = 0; _idx < NLST; ++_idx) {
int *i = lst[_idx];
if (*i == *_i2)

^-- I have trouble with this line. Is it as is should be? I suppose
it is not.

i think he meant

if (*i == _i2)

but i think python does

if (i == &_i2)

because there is only one integer object holding the value 2, so
it is sufficient to compare the addresses (i'm not sure about this,
perhaps someone smarter can clarify?).

--
David.

Jan 5 '06 #31

Ich schrieb:

but i think python does

if (i == &_i2)

because there is only one integer object holding the value 2, so
it is sufficient to compare the addresses (i'm not sure about this,
perhaps someone smarter can clarify?).

well, as far as i can see the relevant function is

static int
int_compare(PyIntObject *v, PyIntObject *w)
{
register long i = v->ob_ival;
register long j = w->ob_ival;
return (i < j) ? -1 : (i > j) ? 1 : 0;
}

in Objects/intobject.c, which does compare by value. so, this is
either special-cased elsewhere or not optimized (should/can it be?).

--
David.

Jan 5 '06 #32

Dan Sommers wrote:

On Wed, 04 Jan 2006 22:38:06 -0500,
"Stuart D. Gathman" <st****@bmsi.com> wrote:

On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:
I was under the assumption that everything in python was a refrence...

so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)
...
Have I misunderstood something?

It might help to do a translation to equivalent C:

int _i1 = 1;
int _i2 = 2;
int _i3 = 3;
int _i4 = 4;
int* lst[NLST] = { &_i1,&_i2,&_i3 };

Okay so far.

int _idx; /* internal iterator */
for (_idx = 0; _idx < NLST; ++_idx) {
int *i = lst[_idx];

[snip]

That's the C idiom

for i in range(len(lst))

we all complain about here. How about (untested):

/*
iterate over the list, binding i to each item in succession;
_idx is internal to the interpreter;
separate the definition of i from the assignment of i for clarity
*/

int **_idx;
for( _idx = lst; _idx < lst + NLST; ++_idx ) {
int *i;
i = *_idx;

/* compare "the item to which i is bound" to "a constant" */
if( *i == *(&_i2) )
/* rebind i to _i4 */
i = &_i4;
}

for (_idx = 0; _idx < NLST; ++_idx)
printf("%d\n",*lst[_idx]);

for( _idx = lst; _idx < lst + NLST; ++_idx ) {
int *i = *_idx;
printf( "%d\n", *i );
}

Regards,
Dan

This code appears to me better than that original one, but it exceeds at
the moment my capacity to check it. Weren't it a good idea to use PyPy
to translate this Python code into C? It should exactly show how it
works, right?
I would be glad to hear how to accomplish this, because I have heard, it
is not easy to separate and use that part of PyPy which does the
translation.

Claudio

Jan 5 '06 #33

Ich schrieb:

well, as far as i can see the relevant function is
in Objects/intobject.c, which does compare by value. so, this is
either special-cased elsewhere or not optimized (should/can it be?).

it is special-cased, but still compares by value. the relevant
parts from "Python/ceval.c":

case COMPARE_OP:
if (PyInt_CheckExact(w) && PyInt_CheckExact(v)) {
a = PyInt_AS_LONG(v);
b = PyInt_AS_LONG(w);
switch (oparg) {
case PyCmp_EQ: res = a == b; break;
}
}

sorry if i am being noisy.

--
David.

Jan 5 '06 #34

Duncan Booth

David Murmann wrote:

but i think python does

if (i == &_i2)

because there is only one integer object holding the value 2, so
it is sufficient to compare the addresses (i'm not sure about this,
perhaps someone smarter can clarify?).

No, Python itself only allocates one integer object holding the value 2,
but that isn't necessarily the only object which can compare equal to 2,
nor even the only integer object with the value of 2 (a C extension could
create another).

I seem to remember that at one time Python tried to 'optimise' the
comparison by saying that if the addresses are equal the objects are equal,
and if the addresses are not equal call the __eq__ method, but this doesn't
work in general as it is possible for an object to not be equal to itself.
(e.g. a NaN or a user defined class).

A suitable short equivalent for the comparison is:

if (*i == *(&_i2)) ...

but a closer equivalent of what Python really does might be:

object *_tmp = &_i2;
if ((PyInt_CheckExact(i) && PyInt_CheckExact(_tmp)) ?
*i==*_tmp : PyObject_RichCompare(i, _tmp, PyCmp_EQ)) ...

i.e. check the types (not that you could actually do that with the C
translation of the code) for two integers which fit in the range of C longs
and compare the values if possible, otherwise call some horrendously
complex comparison code.

Jan 5 '06 #35

Dan Sommers schrieb:

int **_idx;
for( _idx = lst; _idx < lst + NLST; ++_idx ) {
int *i;
i = *_idx;

/* compare "the item to which i is bound" to "a constant" */
if( *i == *(&_i2) )
/* rebind i to _i4 */
i = &_i4;
}

for( _idx = lst; _idx < lst + NLST; ++_idx ) {
int *i = *_idx;
printf( "%d\n", *i );
}

i don't see what is so fundamentally different in this version compared
to the one from Stuart Gathman (except it works, but i think it was a typo).
do you want to stress that python uses an iterator (if so, i would have
renamed _idx to _iter, because it ain't not index anymore)? whats the point
of not starting the for loop at 0? and is there a difference between
(*i == _idx) and (*i == *(&_idx))?

--
David.

Jan 5 '06 #36

"Ben Sizer" <ky*****@gmail.com> writes:

But, if you separate the calling mechanism from the assignment
mechanism, then Python does behave like every other call by reference
language. The problem is that people expect to then be able to change
the value of the referred object with the assignment operator. It's the
assignment semantics that differ from languages such as C++ and Java,
not the calling mechanism. In C++, assignment means copying a value. In
Python, assignment means reassigning a reference.

The unfortunate problem is that people think of 'pass by reference' in
association with 'assignment is a mutating operation', when really the
2 concepts are orthogonal and Python replaces the second with
'assignment is a reference reseating operation'. The only reason I
stress this is because with this in mind, Python is consistent, as
opposed to seeming to have 2 different modes of operation.

Exactly. The critical information is that python's assignment
statement is different from what people are used to from C/C++/etc.
While telling them that "You can't do call by reference because Python
is call by object" may be correct, it leaves out the critical
information. So they're liable to run into the same problem in a
different context - like the one the OP had.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

Jan 5 '06 #37

Dan Sommers

On Thu, 05 Jan 2006 14:28:51 +0100,
David Murmann <da***********@rwth-aachen.de> wrote:

Dan Sommers schrieb:
int **_idx;
for( _idx = lst; _idx < lst + NLST; ++_idx ) {
int *i;
i = *_idx;
/* compare "the item to which i is bound" to "a constant" */
if( *i == *(&_i2) )
/* rebind i to _i4 */
i = &_i4;
}
for( _idx = lst; _idx < lst + NLST; ++_idx ) {
int *i = *_idx;
printf( "%d\n", *i );
}
For reference, Stuart Gathman's looks like this:
int _idx; /* internal iterator */
for (_idx = 0; _idx < NLST; ++_idx) {
int *i = lst[_idx];
i don't see what is so fundamentally different in this version
compared to the one from Stuart Gathman (except it works, but i think
it was a typo) ...
I agree that Stuart's error was a typo.
... do you want to stress that python uses an iterator (if so, i would
have renamed _idx to _iter, because it ain't not index anymore)? whats
the point of not starting the for loop at 0? ...
I was trying to show iteration over a list of items, rather than indexed
access into a list of pointers, but it is no longer as clear to me as it
was at the time that I did a good job of that. ;-)

Another thing I wanted to show is that no amount of changing i would
have any effect on the list, and I missed it in Stuart's code (my
fault).

Or maybe my Python/C ratio is so high that seeing a loop variable used
as an index into a list or an array just bugs me. ;-)

_iter vs. _idx is a non-starter for me because of the baggage associated
with those words.
... and is there a difference between (*i == _idx) and (*i ==
*(&_idx))?

Again, just the emphasis that the comparison is through bindings rather
than directly on the objects, to show that the bindings are completely
independent of the objects. I am pretty sure that there is no semantic
difference between *(&_idx) and _idx in the "normal" case, but there may
be some very odd corner cases if _idx were volatile and/or delcared
register.

Regards,
Dan

--
Dan Sommers
<http://www.tombstonezero.net/dan/>

Jan 5 '06 #38

Bengt Richter

On 4 Jan 2006 10:54:17 -0800, "KraftDiner" <bo*******@yahoo.com> wrote:

I was under the assumption that everything in python was a refrence...

so if I code this:
lst = [1,2,3]
for i in lst:
if i==2:
i = 4
print lst

I though the contents of lst would be modified.. (After reading that
'everything' is a refrence.)
so it seems that in order to do this I need to code it like:

lst = [1,2,3]
for i in range(len(lst)):
if lst[i] == 2:
lst[i]=4
print lst

Have I misunderstood something?

Evidently, as others have already elaborated upon ;-)

I am wondering if it might be helpful to focus on expressions
first, and point out that the result is always a reference to
an object representing the value of the expression.

Since you always get a reference to a value-representing object,
assignment can only have meaning in terms of what you can do with that reference.
You can never assign a value per se. Therefore an assignment is
placing the reference in a place determined by the assignment target syntax.

The simplest is a bare name, and then we speak of binding a name to the
value-representing object. The name belongs to some context, and name space,
and an implementation will determine how to find the named binding site that
will hold the reference, depending on what name space is involved. E.g. the
binding site might be an optimized slot in a function's local name space.

Other assignment syntaxes also create bindings, and also identify what I am
calling binding sites. These sites may be within the representation of another object,
and located by some computation implementing the semantics, such as finding
an appropriate offset into the current representation of a list, or a hash-accessed
bucket/slot of a dict etc.

BTW, the implementation of finding a binding site for a non-bare-name binding
is typically a method call with the method and arguments determined by the trailing
end of the assignment target expression and the object whose method is to be found
determined by the expression up to the trailing end.

Binding site implementation is not part of the language, but the semantics of
binding objects 1:1 or 1:many is.

Mutable objects have assignable binding sites. Immutable objects do not (except
for the "assignment" of initial binding references during their creation). Containing objects
contain binding sites, not values. Binding sites must have references to some
value-representing object or other if they semantically exist, though of course an implementation
may manage space any way convenient if semantics are preserved.

Going back to an expression again, when e.g., a name is evaluated, the name identifies
a binding site somwhere, and the value of any expression (including bare name) is, as we said at the start,
"a reference to an object representing the value of the expression."

In the case of a bare name, we have that reference as soon as we have identified the binding site
associated with the name, since that contains a reference to a value-representing object. If this reference
is now assigned to a binding site identified by another bare name, we have two bare names with respective
associated binding sites having references referring to the same value-representing object.

IOW, python shares value representations by default. Including mutable value representations. Which
means if you want a distinct mutable value equal to a given one, you need an expression that does
more than evaluate to find the reference to the given one -- i.e., creates a new value-representing object
using the given for information, and returns a reference the new.

To a C programmer, it will be tempting to say that "binding sites" are just pointer-data locations,
and references are pointers, and "value-representing-object" is just malloc'ed space for some custom
struct or array or both. This could be true for a particular implementation, but I think "binding site"
and "reference" as I have just used them, are also abstract concepts, capable of alternative, non-C
implementation of the python semantics. So I thought I'd try this as a possible terminology, to see if
it makes it any easier for anyone ;-)

Regards,
Bengt Richter

Jan 5 '06 #39

On Thu, 05 Jan 2006 05:21:24 +0000, Bryan Olson wrote:

Steven D'Aprano wrote:
Mike Meyer wrote:

[...]
Correct. What's stored in a list is a reference.

Nonsense. What is stored in the list is an object.

According to the Python Language Reference:

Some objects contain references to other objects; these are
called containers. Examples of containers are tuples, lists
and dictionaries.
[http://docs.python.org/ref/objects.html]

Is it so hard to understand that the word "reference" has a general,
imprecise meaning in common English (which is clearly how the quote
above is using the word) while still having in the context of assignment
and argument passing a more precise meaning which is dangerously
misleading?

Words are important -- not only for what they mean, but for what the
connotations they carry. For people who come to Python from C-like
languages, the word "reference" means something that is just not true in
the context of Python's behaviour. That's why people come to Python with a
frame that tells that what call by reference implies ("I can do this...")
and then they discover that they often *can't* do that.

It is a crying shame, because reference is a nice, generic word that just
cries out to be used in a nice, generic way, as the Python doc you quoted
is doing. But unfortunately, the term "reference" has been co-opted by C
programmers to effectively mean "pointer to a value".

The aim of the Python doc authors is to communicate information about
Python effectively. That means they have to be aware of words'
connotations and the mental frames they evoke. In the same way a good
programmer must work around bugs in the operating system or libraries, a
good author must work around bugs in people's mental frames which will
cause misunderstanding. That's why "reference" is a bad word to use in the
context of Python containers and argument handling: far from communicating
correct information effectively, it gives many readers a misleading
assumption about how Python code will behave.

The proof of this is the number of times people write to this newsgroup
confused about "call by reference" -- not because they came to Python with
assumptions about its behaviour, but because somebody told that Python was
"call by reference". Or that lists contain "references" to other objects.

In English, I can say "I'm writing a book on Russian history. I've
included a reference to Smith's classic work on the tsars of Russia."
That's a good, generic use of the word "reference". Nobody thinks that
anything I do to modify my book will effect Smith's work. Books don't work
that way.

But in programming, things do work that way. If my class Book contains a
reference to Smith's classic work, I can modify it. (Unless the language
deliberately restricts my ability to modify certain objects, as Python
does with immutable objects.)

That's what programmers expect when you talk about references, especially
if they come from a C (or Pascal) background. In Python, sometimes that's
true, and sometimes it is not, and the only way to tell is by looking at
the object itself, not by thinking about Python's high-level behaviour.

Thinking about Python's behaviour ("it always passes references to
objects") will invoke misleading frames in many programmers' minds. The
word "reference" is misleading and should be avoided, because what the
average non-Python programmer understands by the word is different from
what the experienced Pythonista understands by it.

If we were writing academic papers, we could define "call by reference"
and "objects contain references" any way we liked, and it would be the
responsibility of the readers to ensure they understood *our* meaning. But
we're not -- we're communicating information to ordinary programmers, many
of whom are kids still in school, not academics. Many of them will be
coming to Python with preconceived ideas of the meaning of "call by
reference", "assign a variable", etc. It is *our* responsibility to use
language that will not be misleading, that will not invoke incorrect
frames, that will not lead them up the garden path and be surprised by
Python's behaviour.

If we say "Python is call be reference" (or call by value, as many people
also say) we *know* the consequence will be newbies writing in saying "I
was told Python is call by reference, so I did this, and it didn't work,
is that a bug in Python? What is wrong?" It is not a bug in Python, it is
a bug in their mental model of how Python works, and we put that bug in
their head. Every time that happens, it is *our* fault, not theirs, for
using language guaranteed to mislead. If we use an unfamiliar term like
"call by object", the reader has no preconceived understanding of what it
means and won't be lead to incorrect assumptions.

We know that this will happen because it has happened time and time again
in the past. Are we incapable of learning from experience? Are we
intelligent sentient beings or do we just parrot what was said in the
past with no concern for the consequences of what we say?
--
Steven.

Jan 6 '06 #40

On Wed, 04 Jan 2006 22:51:03 -0500, Mike Meyer wrote:

And if you only ever passed immutable objects around, you would
think Python was call by value.

You might. Then again, you might also understand the concepts well
enough to realize that there isn't any difference between CBR and CBV
when you're passing immutable objects.

Consider this:

def do_nothing(x):
pass

huge_tuple = (None,) * 10000**4
do_nothing(huge_tuple)

If Python made a copy of huge_tuple before passing it to the function, you
would notice.

--
Steven.

Jan 6 '06 #41

Donn Cave

In article <pa****************************@REMOVETHIScyber.co m.au>,
Steven D'Aprano <st***@REMOVETHIScyber.com.au> wrote:
....

We know that this will happen because it has happened time and time again
in the past. Are we incapable of learning from experience? Are we
intelligent sentient beings or do we just parrot what was said in the
past with no concern for the consequences of what we say?

The latter. I know this from the number of times I have read
that parameter passing has something to do with whether the
parameter is a mutable or immutable object.

Donn Cave, do**@u.washington.edu

Jan 6 '06 #42

On Thu, 05 Jan 2006 11:17:44 -0500, Mike Meyer wrote:

While telling them that "You can't do call by reference because Python
is call by object" may be correct,
Good to see you finally concede it.
it leaves out the critical information.
As does "call by reference" or "call by value". No three word phrase can
possibly explain Python's behaviour, or any other language for that
matter. You have to know what "call by whatever" *implies* for it to be
meaningful.
So they're liable to run into the same problem in a
different context - like the one the OP had.

That doesn't follow.

The problem isn't that talk about references is *wrong*, but that it is
misleading. Where "reference" is being used just as a generic term, it is
barely acceptable, but any time it is used in a context where it could be
misconstrued is dangerous precisely because people WILL misunderstand it,
because they *think* they know what references (assignment, call by
reference, whatever) means in this context.

The fact that call by object is unfamiliar is precisely its advantage.
Instead of people jumping to conclusions about what it means, like they do
with "reference", they will stop and think and if need be look for further
information about Python's object model. At the very least, they won't say
"Everything in Python is a reference, so this should work, but it
doesn't".
--
Steven.

Jan 6 '06 #43

Steven D'Aprano <st***@REMOVETHIScyber.com.au> writes:

Thinking about Python's behaviour ("it always passes references to
objects") will invoke misleading frames in many programmers' minds. The
word "reference" is misleading and should be avoided, because what the
average non-Python programmer understands by the word is different from
what the experienced Pythonista understands by it.
Yes, but it's not misleading because "reference" doesn't mean what
they think it means. "reference" means the same thing in Python as it
does in C and similar languages. It's misleading because "variables"
and "assignment" in Python don't do the things they do in C. Some of
the places that this shows up is when you're dealing with call by
reference, or with references in a list.
We know that this will happen because it has happened time and time again
in the past. Are we incapable of learning from experience? Are we
intelligent sentient beings or do we just parrot what was said in the
past with no concern for the consequences of what we say?

No, we learned to say "binding" and "names". We also learned that
while giving newbies false information like "Python doesn't have
references" or "Python isn't call-by-reference" will help them get
past the immediate problem, it sets them up to be confused in some
other area where variables and assignments don't do what they
think. So we've learned to deal with the disease instead of the
symptoms, and tell them how variables in Python behave differently
from variables in C, or Pascal, or ALGOL, or FORTRAN, or whatever.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

Jan 6 '06 #44

Steven D'Aprano <st***@REMOVETHIScyber.com.au> writes:

On Thu, 05 Jan 2006 11:17:44 -0500, Mike Meyer wrote:
While telling them that "You can't do call by reference because Python
is call by object" may be correct, Good to see you finally concede it.

I'm not conceeding anything, because I never said otherwise. In
particular, I still claim that Pythons behavior when passing objects
as parameters is identical to call-by-reference for all the objects
that you can generate in Python.

So they're liable to run into the same problem in a
different context - like the one the OP had.

That doesn't follow.

You're right. That observation is based on watching people repeatedly
beat their head against the difference between variable assignment in
other languages and name binding in object-based languages. Until you
get that difference into their heads, they're going to have
problems. What you use to describe the parameter passing mechanism is
immaterial to the real problem.
The problem isn't that talk about references is *wrong*, but that it is
misleading. Where "reference" is being used just as a generic term, it is
barely acceptable, but any time it is used in a context where it could be
misconstrued is dangerous precisely because people WILL misunderstand it,
because they *think* they know what references (assignment, call by
reference, whatever) means in this context.
This is where we disagree. I think their understanding of references
is dead on. What's broken is their understanding of what variables are
and what assignments mean. Once you fix that, the rest falls into
place.
The fact that call by object is unfamiliar is precisely its advantage.
Instead of people jumping to conclusions about what it means, like they do
with "reference", they will stop and think and if need be look for further
information about Python's object model. At the very least, they won't say
"Everything in Python is a reference, so this should work, but it
doesn't".

What you're advocating is intentionally misleading people to deal with
a symptom. I'd rather give them enough truth to deal with the disease.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

Jan 6 '06 #45

Terry Hancock

On Fri, 06 Jan 2006 12:01:09 +1100
"Steven D'Aprano" <st***@REMOVETHIScyber.com.au> wrote:

On Wed, 04 Jan 2006 22:51:03 -0500, Mike Meyer wrote:
And if you only ever passed immutable objects around,

you would > think Python was call by value.

You might. Then again, you might also understand the
concepts well enough to realize that there isn't any
difference between CBR and CBV when you're passing
immutable objects.

Consider this:

def do_nothing(x):
pass

huge_tuple = (None,) * 10000**4
do_nothing(huge_tuple)

If Python made a copy of huge_tuple before passing it to
the function, you would notice.

Which succinctly demonstrates precisely why a newbie
*should* be told that Python passes references instead of
values. Without this basic knowledge, the newcomer can't
possibly be expected to make intelligent choices of
algorithm.

But it should probably also be made clear that
"reference" means a label assigned to an object, and not a
variable containing a memory location (which is what a C
"pointer" is).

The reference is usually a simple name, but it can also be
an container expression following list, dictionary,
or class instance (spam, spam[0], spam['eggs'], or
spam.ham, for example). Not tuple or string because they
are immutable, and so don't have assignable references.

Cheers,
Terry

--
Terry Hancock (ha*****@AnansiSpaceworks.com)
Anansi Spaceworks http://www.AnansiSpaceworks.com

Jan 6 '06 #46

Ernst Noch

Mike Meyer wrote:

Steven D'Aprano <st***@REMOVETHIScyber.com.au> writes:
Thinking about Python's behaviour ("it always passes references to
objects") will invoke misleading frames in many programmers' minds. The
word "reference" is misleading and should be avoided, because what the
average non-Python programmer understands by the word is different from
what the experienced Pythonista understands by it.

Yes, but it's not misleading because "reference" doesn't mean what
they think it means. "reference" means the same thing in Python as it
does in C and similar languages. It's misleading because "variables"
and "assignment" in Python don't do the things they do in C. Some of
the places that this shows up is when you're dealing with call by
reference, or with references in a list.

Maybe next time showing something like the following trivial snippet
might help demonstrate that the core of the matter doesn't is not the
way python treats parameters?

def func(param, what):

if what:
param['foo'] = 'changed'
else:
temp = param['foo'] # temp is _not_ a reference!
temp = 'changed'

Jan 6 '06 #47

On Thu, 05 Jan 2006 22:18:39 -0600, Terry Hancock wrote:

Consider this:

def do_nothing(x):
pass

huge_tuple = (None,) * 10000**4
do_nothing(huge_tuple)

If Python made a copy of huge_tuple before passing it to
the function, you would notice.
Which succinctly demonstrates precisely why a newbie
*should* be told that Python passes references instead of
values.

"I don't like Chinese food, and pizza gives me gas. Let's have a burger,"
says I.

And Terry helpfully answers: "Pizza gives you gas? Well there you go then,
that's precisely why we have to have Chinese food."

I have never suggested that we should tell newbies that Python is call
by value, and I deny that the only two possible choices for usefully
describing Python's behaviour is by CBV and CBR. The fact that Terry could
read this thread and still imagine that there is such a dichotomy is
worrying.

But it should probably also be made clear that
"reference" means a label assigned to an object, and not a
variable containing a memory location (which is what a C
"pointer" is).

The reference is usually a simple name, but it can also be
an container expression following list, dictionary,
or class instance (spam, spam[0], spam['eggs'], or
spam.ham, for example). Not tuple or string because they
are immutable, and so don't have assignable references.

Are you seriously suggesting that you can't include tuples in a list, or
pass them to functions? Because that's what it sounds like you're saying:
"Python passes references. The reference is usually a simple name, but it
can also be a container... Not tuple or string because they are immutable."

If you don't mean to say that Python can't pass strings to functions, what
do you mean to say?

I'll tell you what I say: Python passes objects to functions or
assignments.

Does this mean that the object is copied? No, I didn't say it copies
objects. I left the nature of the passing mechanism unspoken, which is how
it should be because it is an implementation detail.

Does this mean that the object can be modified in place? Certainly not,
since that depends on the object, not on the nature of Python's high-level
behaviour. It emphasises the object oriented nature of Python, and by its
very language warns the reader not to assume that Python behaves
identically to other languages. It invites further questions, rather than
lulling the reader into jumping to false conclusions.

The emphasis is on the *object*, not the passing mechanism. In C,
everything is mutable, and whether you can change an item depends on
whether you are working with a reference to that item or a copy of the
item. That's a language issue, not a data issue -- in some languages,
you can even choose whether to pass a reference to a function or a copy
of the value.

In Python, whether or not you can change an object depends on the object,
not the language itself: it is a data issue.
--
Steven.

Jan 6 '06 #48

bonono

Steven D'Aprano wrote:

I'll tell you what I say: Python passes objects to functions or
assignments.

Which in C sense, is a reference(or pointer) to some opaque "table"
maintain by the system, identified by id().

Jan 6 '06 #49