473,549 Members | 2,935 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Bizarre behavior with mutable default arguments

I've found some bizzare behavior when using mutable values (lists,
dicts, etc) as the default argument of a function. I want to get the
community's feedback on this. It's easiest to explain with code.

This example is trivial and has design issues, but it demonstrates a
problem I've seen in production systems:

def main(argv = ['TESTING']):
'print out arguments with BEGIN and END'
argv.insert(1, "BEGIN")
argv.append("EN D")
for arg in argv: print arg

if __name__ == '__main__':
from sys import argv, exit
exit(main(argv) )
Now lets try this out at the terminal:
>>import example
example.main( )
BEGIN
TESTING
END
>>example.mai n(["TESTING"])
BEGIN
TESTING
END
>>example.main( )
BEGIN
BEGIN
TESTING
END
END
>>example.mai n(["TESTING"])
BEGIN
TESTING
END

The function does different things if you call it with ["TESTING"] as
the argument, even though that is identical to the default value!! It
seems that default values are only allocated once. If the default
value is mutable and is changed during the function's execution, this
has the side effect of making the default value change on each
subsequent execution.

Is this functionality intended? It seems very unintuitive. This has
caused a bug in my programs twice so far, and both times I was
completely mystified until I realized that the default value was
changing.

I'd like to draw up a PEP to remove this from py3k, if I can get some
community backing.
--Buck
Dec 29 '07 #1
35 2177
Is this functionality intended?

Google for "Python mutable default arguments" (you can actually
leave out Python).

It's part of the language semantics, yes.

Regards,
Martin
Dec 29 '07 #2
On Dec 29, 12:50 pm, bukzor <workithar...@g mail.comwrote:
Is this functionality intended? It seems very unintuitive. This has
caused a bug in my programs twice so far, and both times I was
completely mystified until I realized that the default value was
changing.
it is only unintuitive when you do not know about it

once you realize how it works and what it does it can actually be very
useful

i.

Dec 29 '07 #3
On Dec 29, 1:11 pm, "Martin v. Löwis" <mar...@v.loewi s.dewrote:
Google for "Python mutable default arguments"
and a mere 30 minutes later this thread is already one of the results
that come up
Dec 29 '07 #4
Here's the answer to the question:
http://www.python.org/doc/faq/genera...etween-objects

It looks like Guido disagrees with me, so the discussion is closed.


For the record, I still think the following would be an improvement to
py3k:

In python25:
def f(a=None):
if a is None: a = []
...

In py3k becomes:
def f(a=[])
...
In python25 (this function from the FAQ linked above):
def f(a, _cache={}):
# Callers will never provide a third parameter for this function.
(then why is it an argument?)
...

In py3k becomes:
_cache = {}
def f(a):
global _cache
...

This follows the "explicit is better" and "one best way" principles of
Python, and greatly improves the intuitiveness. Also since the first
example is much more common, it reduces the overall verbosity of the
language.

Just my parting two cents,
--Buck
Dec 29 '07 #5
On Sat, 29 Dec 2007 11:14:30 -0800, bukzor wrote:

In python25 (this function from the FAQ linked above):
def f(a, _cache={}):
# Callers will never provide a third parameter for this function.
(then why is it an argument?)
The caller might want to provide it's own pre-prepared cache. Say, for
testing.

I think that this behaviour is a little unintuitive, and by a little I
mean a lot. Nevertheless, I am used to it, and I don't see any reason to
change it. There's very little about programming that is intuitive --
there's no intuitive reason to think that dictionary lookups are O(1)
while list lookups are O(n).

In the absence of a better solution, I'm very comfortable with keeping
the behaviour as is. Unfortunately, there's no good solution in Python to
providing functions with local storage that persists across calls to the
function:
(1) Use a global variable.

cache = {}
def foo():
global cache
print cache
(2) Use a function attribute.

def foo():
print foo.cache
foo.cache = {}
def foo():
try:
foo.cache
except AttributeError:
foo.cache = {}
print foo.cache

(3) Use an argument that isn't actually an argument.

def foo(cache={}):
print cache
#1, the global variable, is probably the worst solution of the lot.
Global variables are rightly Considered Harmful.
#2 has the disadvantages that you initialize the value *after* you write
the code that relies on it. Either that, or you waste time on every call
checking to see it if has been initialized. Also, like recursive
functions, it is difficult to rename the function.
#3 is, I think, the least-worse solution, but I would hardly call it
ideal.
_cache = {}
def f(a):
global _cache
...

This follows the "explicit is better" and "one best way" principles of
Python,
Declaring an argument is equally explicit.

And you are confused -- the Zen doesn't say "one best way". People so
often get it wrong.

The Zen says:

"There should be one-- and PREFERABLY only one --OBVIOUS way to do it."
(Emphasis added.)

At least you're not saying "there should be only one way to do it". I
give you credit for that!

and greatly improves the intuitiveness. Also since the first
example is much more common, it reduces the overall verbosity of the
language.
I question that it is "much more common". How do you know? Where's your
data?

--
Steven
Dec 30 '07 #6
On Sat, 29 Dec 2007 09:50:53 -0800, bukzor wrote:
I've found some bizzare behavior when using mutable values (lists,
dicts, etc) as the default argument of a function.
This FAQ is so Frequently Asked that I sometimes wonder if Python should,
by default, print a warning when it compiles a function with a list or
dict as as default value.

There's precedence for such a thing: the sum() built-in (un)helpfully
raises an exception if you try to use it on strings.

I say unhelpfully because the one time I wanted to use sum() on strings
was specifically to demonstrate the difference between O(n**2) behaviour
and O(n). I was quite put out that Python, which normally allows you to
shoot yourself in the foot if you insist, was so unnecessarily protective
in this case. Give me a warning, if you wish, but don't stop me.

--
Steven
Dec 30 '07 #7
I think that this behaviour is a little unintuitive, and by a little I
mean a lot.
Thanks for acknowledging it.
I question that it is "much more common". How do you know? Where's your
data?
I did a dumb grep of my Python25/Lib folder and found 33 occurances of
the first pattern above. (Use None as the default value, then check
for None and assign empty list/dict)

Although I spent at least double the amount of time looking for the
second pattern, I found no occurances. (Use dict/list as default value
and modify it in place.) Every single function that used a list or
dict as a default value treated these variables as read-only. However,
I did find two new ways to accomplish the above (further violating the
Zen).

/c/Python25/Lib/site-packages/wx-2.8-msw-ansi/wx/lib/
customtreectrl. py:
def FillArray(self, item, array=[]):
if not array:
array = []

/c/Python25/Lib/site-packages/wx-2.8-msw-ansi/wx/lib/floatcanvas/
FloatCanvas.py:
def __init__(self, ObjectList=[], InForeground = False, IsVisible =
True):
self.ObjectList = list(ObjectList )

--Buck
Dec 30 '07 #8
Just for completeness, the mutable default value problem also affects
classes:

class c:
def __init__(self, list = []):
self.list = list
self.list.appen d("LIST END")
def __repr__(self):
return "<Class a: %s>" % self.list
>>import example2
print example2.c()
<Class a: ['LIST END']>
>>print example2.c([])
<Class a: ['LIST END']>
>>print example2.c()
<Class a: ['LIST END', 'LIST END']>
>>print example2.c([])
<Class a: ['LIST END']>

Again, we get different results if we supply an argument that is
identical to the default value. There are many instances in the
standard library where class values are assigned directly from the
initializer, which has list or dict default values, so there is chance
for errors cropping up here.

The error scenario is this:
1. Use a mutable value as default value in a class constructor.
2. Assign class property from constructor arguments.
3. Instantiate class using default value.
4. Modify class property in place.
5. Instantiate (again) class using default value.

The second instance will behave strangely because data from the first
instance has leaked over. The standard library is not affected because
it avoids one of these five steps. Most classes simply don't have
mutable default values (1). Those that do generally treat them as read-
only (4). Some classes are not useful using the default values (3).
Some classes are not useful to be instantiated twice (5). The classes
that don't avoid the problem at one of these four steps have to avoid
it at (2) by using one of the three above patterns.

--Buck
Dec 30 '07 #9
On Dec 30, 3:21 pm, bukzor <workithar...@g mail.comwrote:
Just for completeness, the mutable default value problem also affects
classes:
Simply, because methods are functions, and can have default arguments.
You don't need to nail *another* zillion theses to the cathedral
door :-)
Dec 30 '07 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
1271
by: jianbing.chen | last post by:
I ran into a similar situation like the following (ipython session). Can anyone please explain why the behavior? Thanks in advance. In : def foo(b=): ....: b.append(3) ....: return b ....: In : foo()
20
1265
by: Jasper | last post by:
I'm stumped. I'm calling a method that has keyword args, but not setting them, and yet one of them starts off with data?! The class definition begins like so: class BattleIntentionAction( BattleAction ): def __init__( self, factionName, location, tactic='hold', targetFacName='', terrainArgs=, garrisonIds= ): self.terrainArgs =...
24
2486
by: Steven D'Aprano | last post by:
Sometimes it seems that barely a day goes by without some newbie, or not- so-newbie, getting confused by the behaviour of functions with mutable default arguments. No sooner does one thread finally, and painfully, fade away than another one starts up. I suggest that Python should raise warnings.RuntimeWarning (or similar?) when a function...
0
7715
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
0
7956
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
1
7469
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
7808
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
6040
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5368
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
5087
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3480
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1057
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.