Dear all,

I have encountered this weird problem.

I have a class definition with an __init__ argument 'd'
which defaults to {}. This argument is put in the 'self.d'
attribute at initialization

I create two independent instances of this class; the code
is as follows.

class C:
def __init__(self, i=10, d = {}):
self.d = d
self.i = i
def get(self):
print
print self.d
def set(self, dval, ival):
self.d.update(dval)
self.i+=ival

c1=C()
c1.set({'one':1},3)
c1.get()

del c1

c2=C()
c2.set({'two':2},4)
c2.get()


If I run the code I obtain:

{'one': 1}

{'two': 2, 'one': 1}

It seems that the 'self.d' argument of the second instance is the
same of the 'self.d' of the first (deleted!) instance.

Running the code in a debugger I discovered that, when I enter the
__init__ at the second initialization, before doing

self.d = d

the 'd' variable already contains the 'self.d' value of the first
instance and not the default argument {}.

Am I doing some stupid error, or this is a problem ?

Thanks in advance for any help,
Paolo

See Pitfall #5 on http://zephyrfalcon.org/labs/python_pitfalls.html
It also applies to dictionaries (and sets, any mutable object really).

On Thu, Oct 9, 2008 at 1:03 AM, kenneth <kenneth@inwind.itwrote:

Dear all,
>
I have encountered this weird problem.
>
I have a class definition with an __init__ argument 'd'
which defaults to {}. This argument is put in the 'self.d'
attribute at initialization
>
I create two independent instances of this class; the code
is as follows.
>
class C:
def __init__(self, i=10, d = {}):

Change 'd = {}' to 'd=None'
Add the line:
if d is None: d = {}

Cheers,
Chris
--
Follow the path of the Iguana...
http://rebertia.com

self.d = d
self.i = i
def get(self):
print
print self.d
def set(self, dval, ival):
self.d.update(dval)
self.i+=ival
>
c1=C()
c1.set({'one':1},3)
c1.get()
>
del c1
>
c2=C()
c2.set({'two':2},4)
c2.get()
>
>
If I run the code I obtain:
>
{'one': 1}
>
{'two': 2, 'one': 1}
>
It seems that the 'self.d' argument of the second instance is the
same of the 'self.d' of the first (deleted!) instance.
>
Running the code in a debugger I discovered that, when I enter the
__init__ at the second initialization, before doing
>
self.d = d
>
the 'd' variable already contains the 'self.d' value of the first
instance and not the default argument {}.
>
Am I doing some stupid error, or this is a problem ?
>
Thanks in advance for any help,
Paolo
--
http://mail.python.org/mailman/listinfo/python-list
>

kenneth wrote:

the 'd' variable already contains the 'self.d' value of the first
instance and not the default argument {}.
>
Am I doing some stupid error, or this is a problem ?

No, it always contains the default argument because default values are
created just ONE TIME.
http://effbot.org/pyfaq/why-are-defa...en-objects.htm

On Oct 9, 10:14*am, Christian Heimes <li...@cheimes.dewrote:

kenneth wrote:

>

>
No, it always contains the default argument because default values are
created just ONE TIME.http://effbot.org/pyfaq/why-are-defa...etween-objects...

Wow, it's a very "dangerous" behavior ...

Just to know, is this written somewhere in the python documentation or
one has to discover it when his programs fails to work ;-) ?

Paolo

On Thu, Oct 9, 2008 at 1:39 AM, kenneth <kenneth@inwind.itwrote:

On Oct 9, 10:14 am, Christian Heimes <li...@cheimes.dewrote:

>
>
Wow, it's a very "dangerous" behavior ...
>
Just to know, is this written somewhere in the python documentation or
one has to discover it when his programs fails to work ;-) ?

It's mentioned in the tutorial (note the "Important warning"):
http://docs.python.org/tutorial/cont...rgument-values

Cheers,
Chris
--
Follow the path of the Iguana...
http://rebertia.com

>
Paolo
--
http://mail.python.org/mailman/listinfo/python-list
>

In article
<5f3a6fdc-40e5-4450-b65d-066f87f27309@v53g2000hsa.googlegroups.com>,
kenneth <kenneth@inwind.itwrote:

On Oct 9, 10:14*am, Christian Heimes <li...@cheimes.dewrote:

>
>
Wow, it's a very "dangerous" behavior ...
>
Just to know, is this written somewhere in the python documentation or
one has to discover it when his programs fails to work ;-) ?

At least once a week someone discovers this "problem", makes a
post about it here, and then someone points to the spot in the
documentation where it's explained.

Seems to me that people often site the "important warning" in
the tutorial. Of course there's no reason anyone would bother
going through the tutorial - just for fun I looked in the
official Python Reference Manual to see whether they're explicit
about this or require the reader to figure it out from something
else they say.

There's a section titled "7.6 Function definitions". About halfway
through that section there's a _bold face_ statement
"Default parameter values are evaluated when the function definition is
executed.", followed by an explanation of how that can lead to
the sort of problem above.

So I guess it _is_ awfully dangerous. They should really explain
this aspect of the language's behavior to people who don't read
the formal definition and also don't work through the tutorial.

Paolo

--
David C. Ullrich

David C. Ullrich a écrit :

In article
<5f3a6fdc-40e5-4450-b65d-066f87f27309@v53g2000hsa.googlegroups.com>,
kenneth <kenneth@inwind.itwrote:
>

>
At least once a week someone discovers this "problem", makes a
post about it here, and then someone points to the spot in the
documentation where it's explained.
>
Seems to me that people often site the "important warning" in
the tutorial. Of course there's no reason anyone would bother
going through the tutorial

Indeed. No reason at all.

- just for fun I looked in the
official Python Reference Manual to see whether they're explicit
about this or require the reader to figure it out from something
else they say.
>
There's a section titled "7.6 Function definitions". About halfway
through that section there's a _bold face_ statement
"Default parameter values are evaluated when the function definition is
executed.", followed by an explanation of how that can lead to
the sort of problem above.

But there's no reason to read the reference manual neither.

So I guess it _is_ awfully dangerous. They should really explain
this aspect of the language's behavior to people who don't read
the formal definition and also don't work through the tutorial.

You mean : "to people that don't bother reading the FineManual *nor*
searching the newsgroup / ML archives ?"

Well... How to say.. Is there any chance these people will read anything
*at all* ?

Bruno Desthuilliers:

You mean : "to people that don't bother reading the FineManual *nor*
searching the newsgroup / ML archives ?"

Are there ways to change how Python3 manages arguments and functions,
to remove this antifeature of Python, avoiding this common mistake
done by every newbie?
I don't care if it reduces the performance of Python a little.

Bye,
bearophile

bearophileHUGS@lycos.com wrote:

Bruno Desthuilliers:

>
Are there ways to change how Python3 manages arguments and functions,
to remove this antifeature of Python, avoiding this common mistake
done by every newbie?
I don't care if it reduces the performance of Python a little.
>

You can't just copy the default values on every call: you would still get
people confused by the semantics whether you did a shallow or deep copy or
as now no copy. I don't think simply re-executing the default argument
expression on each call works either: that would confuse at least as many
people as the current system.

It would be possible, but extremely annoying to limit default arguments to
being literal constants, or slightly less drastically limit them to being
of types known to be immutable. Of course that blocks you from using any
user defined types unless you invent some kind of scheme for declaring that
a type is safe to use as a default argument.

Another option might just be to generate a warning the first time a program
uses a not obviously immutable default.

I think in this situation the best solution is to expect people to learn to
use the language, and accept that those people who don't RTFM will keep on
asking here. Even if you could change the behaviour of default arguments we
would still get equivalent regular questions from the people who initialise
class attributes with lists or dictionaries.

--
Duncan Booth http://kupuguy.blogspot.com

Duncan Booth:

You can't just copy the default values on every call: you would still get
people confused by the semantics whether you did a shallow or deep copy or
as now no copy.

I think I agree.

I don't think simply re-executing the default argument
expression on each call works either: that would confuse at least as many
people as the current system.

May I ask you why? I think I don't agree, but I am not sure.

It would be possible, but extremely annoying to limit default arguments to
being literal constants,

This is a possible solution, beside re-executing the default argument
expression on each call.

unless you invent some kind of scheme for declaring that
a type is safe to use as a default argument.

Well, it seems functional-style programming may become more common in
the future, and seeing languages like Scala, etc, maybe it can be
useful to add to Python some way to define immutable classes (in an
explicit way). Maybe subclasses of Immutable?

Another option might just be to generate a warning the first time a program
uses a not obviously immutable default.

I don't like this solution much.

Even if you could change the behaviour of default arguments we
would still get equivalent regular questions from the people who initialise
class attributes with lists or dictionaries.

I have seen professional programmers too use class attributes instead
of instance ones...

Well, you can't create a fool-proof language that is useful too, but
in a language that is designed for new programmers too, like Python,
and that doesn't put the running speed as its most important feature,
then I think patching the most known and common pitfalls/traps is
useful, even if you can't patch them all (without turning the language
into something useless).

Bye,
bearophile

On Fri, Oct 10, 2008 at 4:36 AM, <bearophileHUGS@lycos.comwrote:

Bruno Desthuilliers:

>
Are there ways to change how Python3 manages arguments and functions,
to remove this antifeature of Python, avoiding this common mistake
done by every newbie?
I don't care if it reduces the performance of Python a little.

The general idea been discussed ad-nauseum on the list several times
before, including just 2 months ago. See e.g.:

[Python-3000] default argument surprises
http://mail.python.org/pipermail/pyt...st/014658.html

[Python-ideas] proto-PEP: Fixing Non-constant Default Arguments
http://mail.python.org/pipermail/pyt...ry/000121.html

[Python-3000] pre-PEP: Default Argument Expressions
http://mail.python.org/pipermail/pyt...ry/005704.html

Cheers,
Chris
--
Follow the path of the Iguana...
http://rebertia.com

>
Bye,
bearophile
--
http://mail.python.org/mailman/listinfo/python-list
>

bearophileHUGS@lycos.com wrote:

>
May I ask you why? I think I don't agree, but I am not sure.
>

My thought (which may well be wrong) is that people would still expect the
default argument expression to use the values of variables at the time when
the function is defined, not the values at the point of calling it.

e.g. in this hypothetical universe:

1

would certainly be suprising to anyone used to the current behaviour and I
think would also suprise anyone who hadn't read the manual in sufficient
details.

We already get people asking why code like this doesn't return 3:

9

i.e. they expect the variable lookup to be done at function definition time
rather than function call time. This implies to me that some people are
going to get confused whichever way round these things happen although
perhaps it is the behaviour of default arguments that makes them expect
this.

As an aside, making this change to default arguments would mean the
solution usually proposed to the function scoping question above would no
longer work:

3

I wonder whether it is the way the default argument expressions are
embedded inside the function that causes the confusion? If for example
default arguments were defined like this:

class C:
@default(d={})
def __init__(self, i=10, d):
self.d = d
self.i = i

would moving the expression before the 'def' make people less inclined to
be suprised that the object is shared?

Chris Rebert:

The general idea been discussed ad-nauseum on the list several times
before, including just 2 months ago. See e.g.:

Okay, it can't be fixed then.

Bye and thank you,
bearophile

On Thu, 09 Oct 2008 01:39:30 -0700, kenneth (a.k.a. Paolo) wrote:

On Oct 9, 10:14Â*am, Christian Heimes <li...@cheimes.dewrote:

objects...

>
>
Wow, it's a very "dangerous" behavior ...

No, it's very *useful* behaviour.


--
Steven

On Fri, 10 Oct 2008 06:20:35 -0700, bearophileHUGS wrote:

>
May I ask you why? I think I don't agree, but I am not sure.

x = 100
def foo(a, b=x):
return a+b

first = foo(1)
x = 101
second = foo(1)

assert first == second


I think people will be rightly surprised that this fails.

>
This is a possible solution, beside re-executing the default argument
expression on each call.

That's no solution at all, because default arguments should not be
limited to literal constants. That's unacceptable in my opinion.

>
Well, it seems functional-style programming may become more common in
the future, and seeing languages like Scala, etc, maybe it can be useful
to add to Python some way to define immutable classes (in an explicit
way). Maybe subclasses of Immutable?

You're still assuming that the behaviour is a bug. It's not, it's a
feature.

>
I have seen professional programmers too use class attributes instead of
instance ones...

That's only a mistake if you don't mean to use class attributes instead
of instance attributes.



--
Steven

On Oct 10, 12:30*pm, Duncan Booth <duncan.bo...@invalid.invalid>
wrote:

bearophileH...@lycos.com wrote:

>

>

(snip)

I wonder whether it is the way the default argument expressions are
embedded inside the function that causes the confusion? If for example
default arguments were defined like this:
>
class C:
* @default(d={})
* def __init__(self, i=10, d):
* * self.d = d
* * self.i = i
>
would moving the expression before the 'def' make people less inclined to
be suprised that the object is shared?

You could of course define a wrapper to do call-time assignment:

@calltime( d= dict, e= tuple )
def foo( self, d, e ):

If this decorator appeared in the standard library, newbies would be
able to stumble upon it.

I don't think either semantic is more obvious from the syntax alone.
It could mean either thing just as reasonably, and if Python defined
the opposite, we'd be getting opposite complaints.

On one hand, note that the return statement does not follow the same
convention:

....

[0, 1]

It constructs a new object each time. In light of this, the burden of
proof could even fall on Python for the inconsistency. That is,
assuming that it's well- and always defined.

On Oct 11, 4:41*am, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.auwrote:

On Fri, 10 Oct 2008 06:20:35 -0700, bearophileHUGS wrote:

snip

>
That's only a mistake if you don't mean to use class attributes instead
of instance attributes.

Off topic: That gives me an idea for an interesting technique.

class Foo( HasInstanceVars ):
class InstanceVars:
x= 0
y= None
z= [ 0, 1 ]

The __init__ method in HasInstanceVars adds any InstanceVars members
to the instance members. It's not terribly different than using
__init__-- slightly slower, slightly clearer. It could even include a
'__initvars__' variable which adds constructor parameters by name to
the instance. It's marginally problematic how to create new objects
each time Foo is instantiated. You could require factories, pickle-
unpickle the contents, require 'deepcopy' compatibility, execute a
string, or call a function which uniquely executes the class
statement.

0

1

In article <48ef37fe$0$22798$426a34cc@news.free.fr>,
Bruno Desthuilliers <bruno.42.desthuilliers@websiteburo.invalid>
wrote:

David C. Ullrich a écrit :

>
Indeed. No reason at all.
>

>
But there's no reason to read the reference manual neither.
>

>
You mean : "to people that don't bother reading the FineManual *nor*
searching the newsgroup / ML archives ?"

Yes. Also add "don't read any books". I think I started with some
book - the book pointed out the "surprise" you get when you say

L = [[0]] * 10
L[0][0] = 1

or something similar. I was probably surprised once by the
mutable default parameter thing once but having understood
that other example it wasn't hard to see what was happening.

Well... How to say.. Is there any chance these people will read anything
*at all* ?

No. That's exactly the point! Basic Python is so transparent that
you can start using it without reading anything, just looking at
a few examples. _Because_ of that it's their responsibility to
ensure that if you look at a few examples you then have a complete
understanding of the language.

In particular default parameters should work the way the user
expects! The fact that different users will expect different
things here is no excuse...

--
David C. Ullrich

On Oct 14, 1:50*pm, Bruno Desthuilliers
<bdesth.quelquech...@free.quelquepart.frwrote:

David C. Ullrich a écrit :
>
>
>

snip

(snip)

snip

>
If different users expect different - mostly incompatible - things, how
would it be possible to have it working "the way the user expect" ?
Should Python grow some telepathic features to guess the user's
expectations and modifies itself to meet these expectations ?-)

No. Just have a user community that only has one user.

In message <01006451$0$20646$c3e8da3@news.astraweb.com>, Steven D'Aprano
wrote:

On Thu, 09 Oct 2008 01:39:30 -0700, kenneth (a.k.a. Paolo) wrote:
>

<http://effbot.org/pyfaq/why-are-default-values-shared-between-objects>...

>
No, it's very *useful* behaviour.

Can you give an example of how useful it is? Something worth the pain of
newbies tripping over it every week?

Aaron "Castironpi" Brady wrote:
[about how default argument behavior should, in his opinion, be changed]

Say what you like. The language is as it is by choice. Were it, for some
reason, to change we would then be receiving posts every week that
didn't understand the *new* behavior.

Sometimes people just have to learn to confirm with reality instead of
requiring reality to confirm with their preconceptions. This is one such
case.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

On Oct 15, 11:33*pm, Steve Holden <st...@holdenweb.comwrote:

Aaron "Castironpi" Brady wrote:
>
[about how default argument behavior should, in his opinion, be changed]
>
Say what you like. The language is as it is by choice. Were it, for some
reason, to change we would then be receiving posts every week that
didn't understand the *new* behavior.
>
Sometimes people just have to learn to confirm with reality instead of
requiring reality to confirm with their preconceptions. This is one such
case.
>
regards
*Steve
--
Steve Holden * * * *+1 571 484 6266 * +1 800 494 3119
Holden Web LLC * * * * * * *http://www.holdenweb.com/

I am not convinced it should either stay or go, but it's hard to argue
one way or the other about something so deeply entrenched. However,
what are your thoughts, whatever the default behavior is, on a
decorator that provides the alternative? That is, a decorator that
either reevaluates default arguments each time when the language
evaluates them once, or a decorator that evaluates arguments once,
when the languages evaluates them each time?

P.S.

we would then be receiving posts every week that
didn't understand the *new* behavior.

That is not obvious and I don't know of any empirical evidence that
entails it. Hard to search the standard library for that figure.

On Wed, Oct 15, 2008 at 9:43 PM, Aaron Castironpi Brady
<castironpi@gmail.comwrote:

On Oct 15, 11:33 pm, Steve Holden <st...@holdenweb.comwrote:

>
I am not convinced it should either stay or go, but it's hard to argue
one way or the other about something so deeply entrenched. However,
what are your thoughts, whatever the default behavior is, on a
decorator that provides the alternative? That is, a decorator that
either reevaluates default arguments each time when the language
evaluates them once, or a decorator that evaluates arguments once,
when the languages evaluates them each time?
>
P.S.

That is not obvious and I don't know of any empirical evidence that
entails it. Hard to search the standard library for that figure.

Although primitive and likely somewhat flawed, you may find the
statistics in the "Compatibility Issues" section of
http://mail.python.org/pipermail/pyt...ry/005704.html
to be of interest.

Cheers,
Chris
--
Follow the path of the Iguana...
http://rebertia.com

--
http://mail.python.org/mailman/listinfo/python-list
>

Chris Rebert:

Although primitive and likely som