properly delete item during "for item in..."

[Ratko]

Say you have something like this:

for item in myList:
del item

Would this actually delete the item from the list or just decrement
the reference counter because the item in myList is not associated
with name "item" anymore (but still is with myList[itemIndex])? In
other words, is "item" a temporary reference to myList[itemIndex] or
is it actually that reference that myList has stored?

I am not sure if this question even makes any sense anymore. I've been
using python for years and never had any problems (and I don't now
either) but now that I had to revisit c++/STL, I had to deal about
these issues and was wondering how python does it.

Thanks,
Ratko

[Marc 'BlackJack' Rintsch]

On Thu, 17 Jul 2008 09:27:27 -0700, Ratko wrote:

for item in myList:
del item

Would this actually delete the item from the list or just decrement
the reference counter because the item in myList is not associated
with name "item" anymore (but still is with myList[itemIndex])? In
other words, is "item" a temporary reference to myList[itemIndex] or
is it actually that reference that myList has stored?

The latter. Names are always bound to objects, you can't bind a name to
another name or reference in Python.

Ciao,
Marc 'BlackJack' Rintsch

[Gary Herron]

Ratko wrote:

Say you have something like this:

for item in myList:
del item

Would this actually delete the item from the list or just decrement
the reference counter because the item in myList is not associated
with name "item" anymore (but still is with myList[itemIndex])? In
other words, is "item" a temporary reference to myList[itemIndex] or
is it actually that reference that myList has stored?

The 'del' statement does not delete an object, it deletes a reference to
an object. In this case, the variable item is deleted from the scope,
and the referred-to object will have its reference counter decremented
by 1. (But, as you surmise, not to zero, because the list will still
reference it.)

You could remove the object from the list with
del myList[i]
if you knew i. HOWEVER, don't do that while looping through the list!
Changing a list's length will interact badly with the for loop's
indexing through the list, causing the loop to mis the element following
the deleted item.

Gary Herron

I am not sure if this question even makes any sense anymore. I've been
using python for years and never had any problems (and I don't now
either) but now that I had to revisit c++/STL, I had to deal about
these issues and was wondering how python does it.

Thanks,
Ratko
--
http://mail.python.org/mailman/listinfo/python-list

[mk]

Gary Herron wrote:

You could remove the object from the list with
del myList[i]
if you knew i. HOWEVER, don't do that while looping through the list!
Changing a list's length will interact badly with the for loop's
indexing through the list, causing the loop to mis the element following
the deleted item.

Jumping into a thread, I know how not to do it, but not how to do it
properly?

Iterating over a copy may _probably_ work:

>>t=['a', 'c', 'b', 'd']

for el in t[:]:

del t[t.index(el)]

>>t

[]
However, is it really safe? Defining safe as "works reliably in every
corner case for every indexable data type"?
Con: suppose the data structure t is really, really big. Just deleting
some items from t temporarily doubles the memory consumption.

[Ratko]

On Jul 17, 9:57 am, mk <[e-mail]:

Gary Herron wrote:

You could remove the object from the list with
del myList[i]
if you knew i. HOWEVER, don't do that while looping through the list!
Changing a list's length will interact badly with the for loop's
indexing through the list, causing the loop to mis the element following
the deleted item.

Jumping into a thread, I know how not to do it, but not how to do it
properly?

Iterating over a copy may _probably_ work:

>>t=['a', 'c', 'b', 'd']
>>>
>>for el in t[:]:

del t[t.index(el)]

>>t

[]

However, is it really safe? Defining safe as "works reliably in every
corner case for every indexable data type"?

Con: suppose the data structure t is really, really big. Just deleting
some items from t temporarily doubles the memory consumption.

Would this work (safely) then? It does in my test cases but that of
course doesn't prove it works in a general case...

for item in myList:
myList.remove(item)
For dictionaries we can just iterate over values() or items() as
opposed to itervalues() or iteritems() since that's technically a copy
of values or items in the dict, right?
R

[Duncan Booth]

mk <mr****@gmail.comwrote:

Iterating over a copy may _probably_ work:

>t=['a', 'c', 'b', 'd']

for el in t[:]:

del t[t.index(el)]

>t

[]
However, is it really safe? Defining safe as "works reliably in every
corner case for every indexable data type"?

No, because you cannot necessarily copy every indexable data type using
t[:], and not all types will support index. Also it is inefficient to
delete from the start of the list.

If you are working with a list and deleting every object:

del t[:]

will suffice.

If you don't want to delete everything then you could do:

for index, el in enumerate(reversed(t)):
if not wewant(el):
del t[index]

but the pythonic way is just:

t[:] = [ el for el in t if wewant(el) ]

or if you just want the filtered list and don't care about updating the
original:

t = [ el for el in t if wewant(el) ]

Con: suppose the data structure t is really, really big. Just deleting
some items from t temporarily doubles the memory consumption.

No it doesn't. Copying a list doesn't copy any of the elements in the list,
it just copies the references to those element.

[Gary Herron]

Ratko wrote:

On Jul 17, 9:57 am, mk <[e-mail]:

>Gary Herron wrote:

>>You could remove the object from the list with
del myList[i]
if you knew i. HOWEVER, don't do that while looping through the list!
Changing a list's length will interact badly with the for loop's
indexing through the list, causing the loop to mis the element following
the deleted item.

Jumping into a thread, I know how not to do it, but not how to do it
properly?

Iterating over a copy may _probably_ work:

> >>t=['a', 'c', 'b', 'd']

for el in t[:]:

del t[t.index(el)]

> >>t

[]

However, is it really safe? Defining safe as "works reliably in every
corner case for every indexable data type"?

Con: suppose the data structure t is really, really big. Just deleting
some items from t temporarily doubles the memory consumption.

Would this work (safely) then? It does in my test cases but that of
course doesn't prove it works in a general case...

for item in myList:
myList.remove(item)

No. Same problem, The for loop iterates through the list by keeping
and incrementing an internal index. Any modification of the list does
not change the index correspondingly.

One proper way:
newList = []
for item in myList:
if ... whatever...
newList.append(item)
myList = newList

Another, using list comprehension (it's the same thing really as the above):
myList = [item for item in myList if ... whatever...]

>
For dictionaries we can just iterate over values() or items() as
opposed to itervalues() or iteritems() since that's technically a copy
of values or items in the dict, right?

No! In fact the whole point of iteritems and itervalues and iterkeys is
that they *DO NOT* make copies, so changing the dictionary out from
under them is a programming error.

If you use dict.items(), dict.keys() or dict.values(), then you're OK,
because these methods *do* create new lists for both.

Gary Herron

>
R

--
http://mail.python.org/mailman/listinfo/python-list

[Ratko]

For dictionaries we can just iterate over values() or items() as

opposed to itervalues() or iteritems() since that's technically a copy
of values or items in the dict, right?

No! In fact the whole point of iteritems and itervalues and iterkeys is
that they *DO NOT* make copies, so changing the dictionary out from
under them is a programming error.

If you use dict.items(), dict.keys() or dict.values(), then you're OK,
because these methods *do* create new lists for both.

That's what I meant, it just didn't come across correctly I guess.
Thanks for clarifying these issues. I think I have a better
understanding now.
R

[Reedick, Andrew]

-----Original Message-----
From: py********************************@python.org [mailto:python-
li*************************@python.org] On Behalf Of Ratko
Sent: Thursday, July 17, 2008 12:27 PM
To: py*********@python.org
Subject: properly delete item during "for item in..."

Say you have something like this:

for item in myList:
del item

Would this actually delete the item from the list or just decrement
the reference counter because the item in myList is not associated
with name "item" anymore (but still is with myList[itemIndex])? In
other words, is "item" a temporary reference to myList[itemIndex] or
is it actually that reference that myList has stored?

I am not sure if this question even makes any sense anymore. I've been
using python for years and never had any problems (and I don't now
either) but now that I had to revisit c++/STL, I had to deal about
these issues and was wondering how python does it.

Walk the list backwards when deleting.

master = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
print "Deletes nothing"
a = master[:]
print a
for i in a:
del i
print a

print
print

print "Deletes safely from end"
a = master[:]
print a
for i in range(len(a)-1, -1, -1):
print i
if i % 2 == 0:
print " removing ", master[i]
del a[i]
print " ", a,
if master[i] in a:
print "Ooops, deleted wrong thing...",
print
print a

print
print

print "Delete from front. Deletes wrong things and throws an
exception..."
a = master[:]
print a
#for i in range(len(a)-1, -1, -1):
for i in range(len(a)):
print i
if i % 2 == 0:
print " removing ", master[i]
del a[i]
print " ", a,
if master[i] in a:
print "Ooops, deleted wrong thing...",
print
print a