mutable default parameter problem [Prothon]

[Mark Hahn]

As we are addressing the "warts" in Python to be fixed in Prothon, we have
come upon the
mutable default parameter problem. For those unfamiliar with the problem,
it can be seen in this Prothon code sample where newbies expect the two
function calls below to both print [ 1 ] :

def f( list=[ ] ):
print list.append!(1)

f() # prints [ 1 ]
f() # prints [ 1, 1 ]

It is more than just a newbie problem. Even experts find themselves having
to do things like this which is a waste of programming effort:

def f( list = None ):
if list == None: list = [ ]

We have three proposals in the Prothon mailing list right now to fix this.
I'd like to bounce these off of the Python list also since this will
possibly make a big difference in Python code ported over to Prothon and we
can always use more advice.

1) Only allow immutable objects as default values for formal parameters. In
Prothon immutable objects are well-defined since they have an immutable flag
that write-protects them. This (as the other solutions below) would only
solve the problem in a shallow way as one could still have something like a
tuple of mutable objects and see the problem at a deeper level. If the
newbie is going to be dealing with something this complex though then they
are dealing with the overall problem of references versus copies and that is
a bigger overall issue.

2) Evaluate the default expression once at each call time when the default
value is needed. The default expression would be evaluated in the context
of the function definition (like a closure).

3) Evaluate the expression at definition time as it is done now, but at call
time do a defaultValue.co py() operation. This would be a shallow copy so
again it would be a shallow solution.

Choice 2 is my favorite in that it matches the dynamic nature of Prothon,
but it is an expensive solution. Choice 1 is the least expensive solution
but it is limiting to the user. Choice 1 does not help the second code
sample above. Choice 3 is a good compromise since an object.copy() is
pretty fast in Prothon.

Comments? How much Python code would these different proposals break?

[Marcin 'Qrczak' Kowalczyk]

On Tue, 15 Jun 2004 22:01:19 -0800, Troy Melhase wrote:

Or even this:

shared_cache = {}

def F(a, b, cache=shared_ca che):
...

This would work in the 2nd scheme.

--
__("< Marcin Kowalczyk
\__/ qr****@knm.org. pl
^^ http://qrnik.knm.org.pl/~qrczak/

[Marcin 'Qrczak' Kowalczyk]

On Tue, 15 Jun 2004 22:01:19 -0800, Troy Melhase wrote:

$ find /usr/lib/python2.3/ -name "*.py" -exec grep "def.*=\[\]" {} \; | wc

And see 67 instances just in the standard library.

I don't know how you counted them:

[qrczak ~/src/python/Python-2.3.4]$ egrep 'def.*= ?\[\]' **/*.py | wc -l
45
[qrczak ~/src/python/Python-2.3.4]$ egrep 'def.*= ?None' **/*.py | wc -l
1420

Now consider that many of the Nones are a workaround for the current
Python behavior.

I agree that it's probably impractical to change Python rules because
some code relies on the current behavior. OTOH evaluating the default
argument each time when the function is applied is technically better.
This is one of warts which is hard to fix because of compatibility.

--
__("< Marcin Kowalczyk
\__/ qr****@knm.org. pl
^^ http://qrnik.knm.org.pl/~qrczak/

[Andrea Griffini]

On 16 Jun 2004 07:52:04 GMT, Rob Williscroft <rt*@freenet.co .uk>
wrote:

But python has static variables.
That's what I mean with "I'm new to python" :-)
def another( x ):
y = getattr( another, 'static', 10 )
another.static = x
return y

print another(1), another(2), another(4)
I like more as an example:

def foo(x): ... if not hasattr(foo,'li st'):
... foo.list = []
... foo.list.append (x)
... print foo.list
... foo(1) [1] foo(2) [1, 2] foo(3) [1, 2, 3]

In C++ you get the "if" part for free, but the python
approach is still goodlooking enough.

C++:

static int y = 12;

Python:

if not hasattr(foo,'y' ):
foo.y = 12

The python "static" also worked as I expected when
they're inside locally defined functions returned as
callable objects.
It seems to me in python "everything is an object" leads to
"everything is a dictionary" (except when it isn't:).

This language looks better every day :-) ...

But if there are those better-looking statics, why so
much use of that modifiable-default-of-a-fake-parameter
ugly trick ? Is this something that became legal
only recently ?

I found the description of that 'wart' and the corresponding
'trick' it in some doc about python (don't remember which
one), and even started using it myself! shame on me!

Now that I see this other approach, the function object
attributes look way better IMO...
Why pushing the uglyness instead of the beauty ?

Historical reasons may be ? Those are behind a lot of
C++ horrible parts...
Andrea

[Mark Hahn]

Andrea Griffini wrote:

That's what I mean with "I'm new to python" :-)

def another( x ):
y = getattr( another, 'static', 10 )
another.static = x
return y

print another(1), another(2), another(4)

I like more as an example:

>>> def foo(x):

... if not hasattr(foo,'li st'):
... foo.list = []
... foo.list.append (x)
... print foo.list

In Prothon:

def foo(x):
print foo.list.append !(x)
foo.list = []

(Sorry. I couldn't resist bragging.)

[Peter Hansen]

Mark Hahn wrote:

Andrea Griffini wrote:

I like more as an example:

>>> def foo(x):

... if not hasattr(foo,'li st'):
... foo.list = []
... foo.list.append (x)
... print foo.list

In Prothon:
def foo(x):
print foo.list.append !(x)
foo.list = []

(Sorry. I couldn't resist bragging.)

About what?

Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)]

def foo(x): .... foo.list.append (x)
.... print foo.list
.... foo.list = []
foo('test')

['test']

(Oh, did you mean bragging about how a hard-to-see exclamation
mark causes append() to return the sequence? I thought
maybe it was about the function attribute or something.)

-Peter

[Mark Hahn]

Peter Hansen wrote:

In Prothon:
def foo(x):
print foo.list.append !(x)
foo.list = []

(Sorry. I couldn't resist bragging.)

About what?

Python 2.3.3 (#51, Dec 18 2003, 20:22:39) [MSC v.1200 32 bit (Intel)]

>>> def foo(x): ... foo.list.append (x)
... print foo.list
... >>> foo.list = []
>>> foo('test')

['test']

(Oh, did you mean bragging about how a hard-to-see exclamation
mark causes append() to return the sequence? I thought
maybe it was about the function attribute or something.)

Actually, I didn't know if the function attribute assignment outside the
function would work in Python or not. I guess I'll know better than to try
to play one-upmanship with Python next time. I did say I was sorry :-)

FYI: It's not that the exclamation mark causes append to return the
sequence. The exclamation mark is always there and the sequence is always
returned. The exclamation mark is the universal symbol for in-place
modification. This is straight from Ruby and solves the problem that caused
Guido to not allow sequences to be returned. And, yes, I do think that's
worth bragging about ;-)

[Andrea Griffini]

On Wed, 16 Jun 2004 12:40:10 -0700, "Mark Hahn" <ma**@prothon.o rg>
wrote:

In Prothon:

def foo(x):
print foo.list.append !(x)
foo.list = []

(Sorry. I couldn't resist bragging.)

The very first thing I tried was assigning foo.list
outside of the function (and, by the way, that works in
python too); this however doesn't mimic C++ static,
as initialization of the static local variable is done
in C++ when (and only IF) the function is entered.
The value used for initialization can for example
depend on local parameters or global state at *that time*.

Using "hasattr" seemed ugly to me at first, but after
all you need an additional flag anyway, so why not
checking the presence of a certain key in foo.__dict__ ?
That way both initialization and setting the flag are
done at the same time using just one clean statement.

The only (very small) syntax price is the if (that in
C++ is implicit in the overused keyword "static").

Andrea

[David Bolen]

Andrea Griffini <ag****@tin.i t> writes:

But if there are those better-looking statics, why so
much use of that modifiable-default-of-a-fake-parameter
ugly trick ? Is this something that became legal
only recently ?

Function attributes were in fact somewhat recently added (as of Python
2.1, so circa April of 2001).

-- David

[Michele Simionato]

"Mark Hahn" <ma**@prothon.o rg> wrote in message news:<5L2Ac.26$ u%3.13@fed1read 04>...

FYI: It's not that the exclamation mark causes append to return the
sequence. The exclamation mark is always there and the sequence is always
returned. The exclamation mark is the universal symbol for in-place
modification. This is straight from Ruby and solves the problem that caused
Guido to not allow sequences to be returned. And, yes, I do think that's
worth bragging about ;-)

I think the esclamation mark comes from Scheme if not from a more
ancient language. It is certainly not a new idea. OTOH, it is a good
idea, no question
about that. Same for "?" in booleans.
Michele Simionato

[Dave Brueck]

Mark wrote:

I like more as an example:

>>> def foo(x):

... if not hasattr(foo,'li st'):
... foo.list = []
... foo.list.append (x)
... print foo.list

In Prothon:

def foo(x):
print foo.list.append !(x)
foo.list = []

(Sorry. I couldn't resist bragging.)

About what?

-Dave