scared about refrences...

[SpreadTooThin]

I'm really worried that python may is doing some things I wasn't
expecting... but lets see...

if I pass a list to a function def fn(myList):

and in that function I modify an element in the list, then does the
callers list get modied as well.

def fn(list):
list[1] = 0

myList = [1, 2, 3]
print myList
fn(myList)
print myList

>>[1,2,3]
[1,0,3]

How can I avoid this? In this case this is a really simplified example
but the effects are the same...
How do I specify or create deep copies of objects that may contain
other objects that may contain other
object that may contain other objects....

[Nick Vatamaniuc]

I think you are afraid of references because you are not trusting your
own code. You are afraid it will do "magic" behind the scenes and mess
everything up. One way around that is to simply write better code and
test often.

If everything was copied when passed around it would be pretty awful --
imagine passing around a 1Gb worth of data to a function...

In Python all the primitives are copied and all other entities are
references. If you want to just copy a list you can :
1) use the list class: new_list=list(o ld_list)
2) use the [:] syntax: new_list=old_li st[:]
3) if your list has nested lists, for example, the above methods will
not copy everything, so you need a deep copy -- copy.deepcopy()

Hope this helps,
Nick V.
SpreadTooThin wrote:

I'm really worried that python may is doing some things I wasn't
expecting... but lets see...

if I pass a list to a function def fn(myList):

and in that function I modify an element in the list, then does the
callers list get modied as well.

def fn(list):
list[1] = 0

myList = [1, 2, 3]
print myList
fn(myList)
print myList

>[1,2,3]
[1,0,3]

How can I avoid this? In this case this is a really simplified example
but the effects are the same...
How do I specify or create deep copies of objects that may contain
other objects that may contain other
object that may contain other objects....

[SpreadTooThin]

J. Clifford Dyer wrote:

SpreadTooThin wrote:

Steven D'Aprano wrote:

On Mon, 30 Oct 2006 13:10:47 -0800, SpreadTooThin wrote:

How do I specify or create deep copies of objects that may contain
other objects that may contain other object that may contain other
objects... .
See the `copy` module especially `copy.deepcopy( )`.

This appears to be the right thing to do to me.. (but what do I know?)
Yes, copy.deepcopy() is the thing you want.

But remember Fredrik's advice that well-designed Python code should not
need to copy data structures often. I don't think I've ever needed to use
deepcopy, and rarely copy.copy().

In general, functions should not modify their caller's data. So this is
bad practice:

def print_list(alis t):
"""Print a sorted list"""
alist.sort() # modifies the caller's data -- bad!
for index, value in enumerate:
print "Value %s at index %d" % (index, value)

This is better:

def print_list(alis t):
"""Print a sorted list"""
alist = alist[:] # makes a local shallow copy of the list
alist.sort() # safe to modify now
for index, value in enumerate:
print "Value %s at index %d" % (index, value)

But notice that you only need a shallow copy, not a deep copy, because you
aren't modifying the objects within the list, only the list itself.

I tried this which more closely resembles my project but this doesn't
work:
Unfortunately my crystal ball is back at the shop being repaired, so
you'll have to explain what "doesn't work" means in this case. Does it
raise an exception? If so, please post the exception. Does it do something
different from what you expected? Then what did you expect, and what did
it do?

I seems that some of the objects in the list don't get along well with
deep copy..
See my second example post that used deepcopy... When run blows up...

When it blows up, is there a lot of shrapnel, or just smoke and fire?
Is the shrapnel mostly metal, or is it plastic and glass?

In short, if we don't know what's happening, we can't help.
* Did the program spit out a bunch of text you didn't understand?
If so, show us the text. That text may be incomprehensibl e at first,
but it contains crucial clues.

* Did it close your python window without a word?
Tell us.

* Did your computer freeze up?
Tell us.

If you don't tell us what went wrong *exactly*, you won't get a
satisfactory answer.

I would assume that looking at the code you should be able to tell..
Silly me.. Here.. is the log.. If I were helping.. I would have cut
and pasted the code myself and ran it.. instead of trying to interpret
this...

array('H', [1, 2, 3]) ['a', 'b', 'c']
Traceback (most recent call last):
File
"/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py",
line 1806, in runMain
self.dbg.runfil e(debug_args[0], debug_args)
File
"/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py",
line 1529, in runfile
h_execfile(file , args, module=main, tracer=self)
File
"/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py",
line 590, in __init__
execfile(file, globals, locals)
File "/Volumes/Data/Users/bjobrien/Desktop/pythonDICOM/Text-1.py",
line 20, in __main__
test(t)
File "/Volumes/Data/Users/bjobrien/Desktop/pythonDICOM/Text-1.py",
line 16, in test
t = copy.deepcopy(x )
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 174, in deepcopy
y = copier(x, memo)
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 305, in _deepcopy_inst
state = deepcopy(state, memo)
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 174, in deepcopy
y = copier(x, memo)
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 268, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 185, in deepcopy
y = copier(x, memo)
TypeError: __deepcopy__() takes no arguments (1 given)

Cheers,
Cliff

[Gabriel Genellina]

At Monday 30/10/2006 20:37, Nick Vatamaniuc wrote:

>In Python all the primitives are copied and all other entities are
references.

What do you mean?
primitives==bui ltin classes?
entities==objec ts?
In Python, objects are never automatically copied, either builtin or
user-defined. Even 123 is an object. Names provide references to objects.
Perhaps you were thinking of *another* language.
--
Gabriel Genellina
Softlab SRL

_______________ _______________ _______________ _____
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar

[J. Clifford Dyer]

SpreadTooThin wrote:

J. Clifford Dyer wrote:

>SpreadTooThi n wrote:

>>Steven D'Aprano wrote:
On Mon, 30 Oct 2006 13:10:47 -0800, SpreadTooThin wrote:
I seems that some of the objects in the list don't get along well with
deep copy..
See my second example post that used deepcopy... When run blows up...

When it blows up, is there a lot of shrapnel, or just smoke and fire?
Is the shrapnel mostly metal, or is it plastic and glass?

In short, if we don't know what's happening, we can't help.
* Did the program spit out a bunch of text you didn't understand?
If so, show us the text. That text may be incomprehensibl e at first,
but it contains crucial clues.

* Did it close your python window without a word?
Tell us.

* Did your computer freeze up?
Tell us.

If you don't tell us what went wrong *exactly*, you won't get a
satisfactory answer.

I would assume that looking at the code you should be able to tell..
Silly me.. Here.. is the log.. If I were helping.. I would have cut
and pasted the code myself and ran it.. instead of trying to interpret
this...

I know it seems unnecessary to post the traceback when I could get the
same thing by running your code on my machine, but it's actually useful,
for a couple reasons: First, when I run the code, I might not get an
error, or if I do, it might not be the same error you were getting, and
then we'd be on a wild goose chase. This could be because your python
installation is goofy, or because you copied in your code incorrectly.
Shit happens, and I'd rather not even start down one of those blind
alleys. Third, it provides a useful frame for how to look at your
code. While a traceback might look like a big mess at first, it's
actually pretty easy to skim through once you get used to it, and it
tells me where to focus my attention in your code.

>
array('H', [1, 2, 3]) ['a', 'b', 'c']
Traceback (most recent call last):
File
"/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py",
line 1806, in runMain
self.dbg.runfil e(debug_args[0], debug_args)
File
"/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py",
line 1529, in runfile
h_execfile(file , args, module=main, tracer=self)
File
"/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py",
line 590, in __init__
execfile(file, globals, locals)
File "/Volumes/Data/Users/bjobrien/Desktop/pythonDICOM/Text-1.py",
line 20, in __main__
test(t)
File "/Volumes/Data/Users/bjobrien/Desktop/pythonDICOM/Text-1.py",
line 16, in test
t = copy.deepcopy(x )
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 174, in deepcopy
y = copier(x, memo)
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 305, in _deepcopy_inst
state = deepcopy(state, memo)
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 174, in deepcopy
y = copier(x, memo)
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 268, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 185, in deepcopy
y = copier(x, memo)
TypeError: __deepcopy__() takes no arguments (1 given)

>Cheers,
Cliff

Thanks, that's very helpful. Playing with your code a bit, I narrowed
the problem down to the array.array() structure. Looking at
help(array), there's a method defined called __deepcopy__, which, it
seems, takes no arguments, while deepcopy is passing it one argument.
Looks like a bug in the array module to me. Maybe others with more
experience using array will have some deeper insight.
Cheers,
Cliff

[Peter Otten]

J. Clifford Dyer wrote:

Thanks, that's very helpful.**Playi ng*with*your*co de*a*bit,*I*nar rowed
the problem down to the array.array() structure.**Loo king*at
help(array), there's a method defined called __deepcopy__, which, it
seems, takes no arguments, while deepcopy is passing it one argument.
Looks like a bug in the array module to me.**Maybe*othe rs*with*more

Fixed in subversion: http://www.python.org/sf/1545837

[SpreadTooThin]

J. Clifford Dyer wrote:

SpreadTooThin wrote:

J. Clifford Dyer wrote:

SpreadTooThin wrote:
Steven D'Aprano wrote:
On Mon, 30 Oct 2006 13:10:47 -0800, SpreadTooThin wrote:
I seems that some of the objects in the list don't get along well with
deep copy..
See my second example post that used deepcopy... When run blows up...

When it blows up, is there a lot of shrapnel, or just smoke and fire?
Is the shrapnel mostly metal, or is it plastic and glass?

In short, if we don't know what's happening, we can't help.
* Did the program spit out a bunch of text you didn't understand?
If so, show us the text. That text may be incomprehensibl e at first,
but it contains crucial clues.

* Did it close your python window without a word?
Tell us.

* Did your computer freeze up?
Tell us.

If you don't tell us what went wrong *exactly*, you won't get a
satisfactory answer.

I would assume that looking at the code you should be able to tell..
Silly me.. Here.. is the log.. If I were helping.. I would have cut
and pasted the code myself and ran it.. instead of trying to interpret
this...

I know it seems unnecessary to post the traceback when I could get the
same thing by running your code on my machine, but it's actually useful,
for a couple reasons: First, when I run the code, I might not get an
error, or if I do, it might not be the same error you were getting, and
then we'd be on a wild goose chase. This could be because your python
installation is goofy, or because you copied in your code incorrectly.
Shit happens, and I'd rather not even start down one of those blind
alleys. Third, it provides a useful frame for how to look at your
code. While a traceback might look like a big mess at first, it's
actually pretty easy to skim through once you get used to it, and it
tells me where to focus my attention in your code.

array('H', [1, 2, 3]) ['a', 'b', 'c']
Traceback (most recent call last):
File
"/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py",
line 1806, in runMain
self.dbg.runfil e(debug_args[0], debug_args)
File
"/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py",
line 1529, in runfile
h_execfile(file , args, module=main, tracer=self)
File
"/Volumes/Data/Users/bjobrien/Applications/Komodo.app/Contents/SharedSupport/dbgp/pythonlib/dbgp/client.py",
line 590, in __init__
execfile(file, globals, locals)
File "/Volumes/Data/Users/bjobrien/Desktop/pythonDICOM/Text-1.py",
line 20, in __main__
test(t)
File "/Volumes/Data/Users/bjobrien/Desktop/pythonDICOM/Text-1.py",
line 16, in test
t = copy.deepcopy(x )
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 174, in deepcopy
y = copier(x, memo)
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 305, in _deepcopy_inst
state = deepcopy(state, memo)
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 174, in deepcopy
y = copier(x, memo)
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 268, in _deepcopy_dict
y[deepcopy(key, memo)] = deepcopy(value, memo)
File
"/Library/Frameworks/Python.framewor k/Versions/2.4/lib/python2.4/copy.py",
line 185, in deepcopy
y = copier(x, memo)
TypeError: __deepcopy__() takes no arguments (1 given)

Cheers,
Cliff

Thanks, that's very helpful. Playing with your code a bit, I narrowed
the problem down to the array.array() structure. Looking at
help(array), there's a method defined called __deepcopy__, which, it
seems, takes no arguments, while deepcopy is passing it one argument.
Looks like a bug in the array module to me. Maybe others with more
experience using array will have some deeper insight.

I don't understand why python would insist that everything must be a
refrence...
It is of course helpful sometime but other times its not... and now
I'm sorta out
of luck...
I don't know how to make this structure immutable... Pickle it? Seems
very
inefficient to me...
Every time I pass a variable now I will worry that it will be changed
by the function...
I haven't worried about things like this since the very early days of
BASIC....
I don't know.. maybe I have more to learn.

Cheers,
Cliff

[Tim Chase]

I don't know how to make this structure immutable... Pickle

it? Seems very inefficient to me...

Well, classes can be made mostly immutable by intercepting the
attempts to write to it...something like

class Foo(object):
def __setattr__( self, name, val ):
raise TypeError("I'm sorry, Dave. You can't do that.")

It might also be feasible to do something like this with a
decorator...?

Every time I pass a variable now I will worry that it will be
changed by the function... I haven't worried about things like
this since the very early days of BASIC.... I don't know..
maybe I have more to learn.

Well, there are at least solutions to this paranoia that occur to me:

-make a deep-copy of the object in question before calling the
function, and then pass the copy to the function so that it can't
alter the original

-trust/read the functions' documentation. Most functions that
alter the contents of their parameters are kind enough to note as
much. If functions go bunging with your parameters without
documenting it, the function's coder needs to be smacked.

-tkc

[Gabriel Genellina]

At Tuesday 31/10/2006 14:16, SpreadTooThin wrote:

>I don't understand why python would insist that everything must be a
refrence...
It is of course helpful sometime but other times its not... and now
I'm sorta out
of luck...
I don't know how to make this structure immutable... Pickle it? Seems
very
inefficient to me...
Every time I pass a variable now I will worry that it will be changed
by the function...
I haven't worried about things like this since the very early days of
BASIC....
I don't know.. maybe I have more to learn.

If you haven't read this yet, you should:
<http://www.effbot.org/zone/python-objects.htm>
--
Gabriel Genellina
Softlab SRL

_______________ _______________ _______________ _____
Correo Yahoo!
Espacio para todos tus mensajes, antivirus y antispam ¡gratis!
¡Abrí tu cuenta ya! - http://correo.yahoo.com.ar

[Fredrik Lundh]

SpreadTooThin wrote:

Every time I pass a variable now I will worry that it will be changed
by the function...

why? who's writing those scary functions that you cannot trust? and
what makes you think they won't abuse any immutable data you give them?

</F>

[Steve Holden]

SpreadTooThin wrote:
[...]

I don't understand why python would insist that everything must be a
refrence...

We can tell that :)

It is of course helpful sometime but other times its not... and now
I'm sorta out
of luck...

There are very good reasons for Python's namespace model. Sure it's
different from some other languages, but it's actually been some
people's preferred semantics for a very long time now (the
recently-deceased Ralph Griswold, may he rest in peace, chose a very
similar model for Icon.

I don't know how to make this structure immutable... Pickle it? Seems
very
inefficient to me...
Every time I pass a variable now I will worry that it will be changed
by the function...
I haven't worried about things like this since the very early days of
BASIC....
I don't know.. maybe I have more to learn.

You do. Firstly, learn to leave your paranoia outside your programming
life. If a function or method makes undocumented changes to its mutable
parameters then it needs changing (or its documentation does).

Adequate testing should reveal such nonsenses before release.

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden