how to remove multiple occurrences of a string within a list?

[bahoo]

Hi,

I have a list like ['0024', 'haha', '0024']
and as output I want ['haha']

If I
myList.remove('0024')

then only the first instance of '0024' is removed.

It seems like regular expressions is the rescue, but I couldn't find
the right tool.

Thanks!
bahoo

[Matimus]

It depends on your application, but a 'set' might really be what you
want, as opposed to a list.

>>s = set(["0024","haha","0024"])
s

set(["0024","haha"])

>>s.remove("0024")
s

set(["haha"])

[kyosohma]

On Apr 3, 1:31 pm, "Matimus" <[e-mail]:

It depends on your application, but a 'set' might really be what you
want, as opposed to a list.

>s = set(["0024","haha","0024"])
s

set(["0024","haha"])>>s.remove("0024")

>s

set(["haha"])

If you want, you can also loop over the list, like so:

counter = 0
your_list = ["0024","haha","0024"]
for i in your_list:
if i == '0024':
your_list.pop(counter)
counter += 1
Mike

[Thomas Nelson]

bahoo wrote:

Hi,

I have a list like ['0024', 'haha', '0024']
and as output I want ['haha']

If I
myList.remove('0024')

then only the first instance of '0024' is removed.

It seems like regular expressions is the rescue, but I couldn't find
the right tool.

Thanks!
bahoo

It's hard to imagine using regular expressions here. Here's a simple
attempt:

def removeall(mylist,obj):
while obj in mylist:
mylist.remove(obj)

Or, if you don't need the changes in place:

[x for x in mylist if x!= obj]

Tom

[Thomas Nelson]

On Apr 3, 1:49 pm, [e-mail] wrote:

On Apr 3, 1:31 pm, "Matimus" <[e-mail]:

It depends on your application, but a 'set' might really be what you
want, as opposed to a list.

>>s = set(["0024","haha","0024"])
>>s

set(["0024","haha"])>>s.remove("0024")

>>s

set(["haha"])

If you want, you can also loop over the list, like so:

counter = 0
your_list = ["0024","haha","0024"]
for i in your_list:
if i == '0024':
your_list.pop(counter)
counter += 1

Mike

This method doesn't work.

>>li = list('hello larry')
count = 0
for i in li:

.... if i=='l':
.... li.pop(count)
.... coun += 1
....
'l'
'l'

>>l

['h', 'e', 'l', 'o', ' ', 'a', 'r', 'r', 'y']

because the indexes change as you pop, it gets some of them but not
all.

[kyosohma]

On Apr 3, 1:49 pm, [e-mail] wrote:

On Apr 3, 1:31 pm, "Matimus" <[e-mail]:

It depends on your application, but a 'set' might really be what you
want, as opposed to a list.

>>s = set(["0024","haha","0024"])
>>s

set(["0024","haha"])>>s.remove("0024")

>>s

set(["haha"])

If you want, you can also loop over the list, like so:

counter = 0
your_list = ["0024","haha","0024"]
for i in your_list:
if i == '0024':
your_list.pop(counter)
counter += 1

Mike

If you want to get really fancy, you could do a list comprehension
too:

your_list = ["0024","haha","0024"]
new_list = [i for i in your_list if i != '0024']

Mike

[7stud]

On Apr 3, 12:20 pm, "bahoo" <[e-mail]:

Hi,

I have a list like ['0024', 'haha', '0024']
and as output I want ['haha']

If I
myList.remove('0024')

then only the first instance of '0024' is removed.

It seems like regular expressions is the rescue, but I couldn't find
the right tool.

Thanks!
bahoo

Here are a couple of ways:

target = "0024"
l = ["0024", "haha", "0024"]

------------------------------------
while(True):
try:
l.remove(target)
except ValueError:
break

print l
-------------------------------------

for index, val in enumerate(l):
if val==target:
del l[index]

print l

[bahoo]

On Apr 3, 2:31 pm, "Matimus" <[e-mail]:

It depends on your application, but a 'set' might really be what you
want, as opposed to a list.

>s = set(["0024","haha","0024"])
s

set(["0024","haha"])>>s.remove("0024")

>s

set(["haha"])

This sounds cool.
But is there a command I can convert the "set" back to a "list"?

[Matimus]

On Apr 3, 12:13 pm, "bahoo" <[e-mail]:

On Apr 3, 2:31 pm, "Matimus" <[e-mail]:

It depends on your application, but a 'set' might really be what you
want, as opposed to a list.

>>s = set(["0024","haha","0024"])
>>s

set(["0024","haha"])>>s.remove("0024")

>>s

set(["haha"])

This sounds cool.
But is there a command I can convert the "set" back to a "list"?

yeah...

>>l = list(s)
l

["haha"]

[Steve Holden]

bahoo wrote:

On Apr 3, 2:31 pm, "Matimus" <[e-mail]:

>It depends on your application, but a 'set' might really be what you
want, as opposed to a list.

>>>>s = set(["0024","haha","0024"])
s

set(["0024","haha"])>>s.remove("0024")

>>>>s

set(["haha"])

This sounds cool.
But is there a command I can convert the "set" back to a "list"?

That would be list(). So what you want is

s = set(["0024","haha","0024"])
s.remove("0024")
l = list(s)

or something like it. It seems, a priori, unlikely that you only want to
remove items with that specific value, Is this part of some larger problem?

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

[irstas]

On Apr 3, 10:13 pm, "bahoo" <[e-mail]:

On Apr 3, 2:31 pm, "Matimus" <[e-mail]:

It depends on your application, but a 'set' might really be what you
want, as opposed to a list.

>>s = set(["0024","haha","0024"])
>>s

set(["0024","haha"])>>s.remove("0024")

>>s

set(["haha"])

This sounds cool.
But is there a command I can convert the "set" back to a "list"?

Beware that converting a list to set and then back to list won't
preserve the order of the items, because the set-type loses the order.

[bahoo]

On Apr 3, 4:21 pm, Steve Holden <[e-mail]:

bahoo wrote:

On Apr 3, 2:31 pm, "Matimus" <[e-mail]:

It depends on your application, but a 'set' might really be what you
want, as opposed to a list.

>>>s = set(["0024","haha","0024"])
s
set(["0024","haha"])>>s.remove("0024")
s
set(["haha"])

This sounds cool.
But is there a command I can convert the "set" back to a "list"?

That would be list(). So what you want is

s = set(["0024","haha","0024"])
s.remove("0024")
l = list(s)

or something like it. It seems, a priori, unlikely that you only want to
remove items with that specific value, Is this part of some larger problem?

regards
Steve
--
Steve Holden [phone] [phone]
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Recent Ramblings http://holdenweb.blogspot.com

Thanks for all the suggestions.
The larger problem is, I have a list of strings that I want to remove
from another list of strings.
So I guess what I will do is, use a for loop, and within the for loop,
do the "list" to "set" and then back to "list".

[bahoo]

On Apr 3, 3:01 pm, "7stud" <[e-mail]:

On Apr 3, 12:20 pm, "bahoo" <[e-mail]:

Hi,

I have a list like ['0024', 'haha', '0024']
and as output I want ['haha']

If I
myList.remove('0024')

then only the first instance of '0024' is removed.

It seems like regular expressions is the rescue, but I couldn't find
the right tool.

Thanks!
bahoo

Here are a couple of ways:

target = "0024"
l = ["0024", "haha", "0024"]

------------------------------------
while(True):
try:
l.remove(target)
except ValueError:
break

print l
-------------------------------------

for index, val in enumerate(l):
if val==target:
del l[index]

print l

This latter suggestion (with the for loop) seems to be buggy: if there
are multiple items in the list "l" equal to "target", then only the
first one will be removed!

Thanks anyways.

[Steven Bethard]

bahoo wrote:

The larger problem is, I have a list of strings that I want to remove
from another list of strings.

If you don't care about the resulting order::

>>items = ['foo', 'bar', 'baz', 'bar', 'foo', 'frobble']
to_remove = ['foo', 'bar']
set(items) - set(to_remove)

set(['frobble', 'baz'])

If you do care about the resulting order::

>>to_remove = set(to_remove)
[item for item in items if item not in to_remove]

['baz', 'frobble']

STeVe

[bahoo]

On Apr 3, 6:05 pm, Steven Bethard <[e-mail]:

bahoo wrote:

The larger problem is, I have a list of strings that I want to remove
from another list of strings.

If you don't care about the resulting order::

>>items = ['foo', 'bar', 'baz', 'bar', 'foo', 'frobble']
>>to_remove = ['foo', 'bar']
>>set(items) - set(to_remove)

set(['frobble', 'baz'])

If you do care about the resulting order::

>>to_remove = set(to_remove)
>>[item for item in items if item not in to_remove]

['baz', 'frobble']

STeVe

This is amazing. I love python!

[Steven D'Aprano]

On Tue, 03 Apr 2007 11:20:33 -0700, bahoo wrote:

Hi,

I have a list like ['0024', 'haha', '0024']
and as output I want ['haha']

If I
myList.remove('0024')

then only the first instance of '0024' is removed.

There are a whole heap of ways of doing this. They all have in common the
idea of doing something over and over again until you're finished.

Here's one way, using a loop in a try...except block.

try:
while True:
myList.remove('0024')
# raises an exception when the target isn't in the list any more
except ValueError:
pass # we're done

It seems like regular expressions is the rescue,

Since when did reg exes operate on lists?

--
Steven D'Aprano

[Hendrik van Rooyen]

"bahoo" <[e-mail]:

On Apr 3, 2:31 pm, "Matimus" <[e-mail]:

It depends on your application, but a 'set' might really be what you
want, as opposed to a list.

>>s = set(["0024","haha","0024"])
>>s

set(["0024","haha"])>>s.remove("0024")

>>s

set(["haha"])

This sounds cool.
But is there a command I can convert the "set" back to a "list"?

Here is a general way to find the duplicates:

>>duplist

[1, 2, 3, 4, 'haha', 1, 2, 3, 4, 5]

>>copylist = duplist[:]
for x in duplist:

del(copylist[copylist.index(x)])
if x in copylist:
fullset.remove(x)

>>fullset

set([5, 'haha'])

>>list(fullset)

[5, 'haha']

>>>

hth - Hendrik

[Amit Khemka]

On 3 Apr 2007 11:20:33 -0700, bahoo <b8*******@yahoo.comwrote:

Hi,

I have a list like ['0024', 'haha', '0024']
and as output I want ['haha']

If I
myList.remove('0024')

then only the first instance of '0024' is removed.

To remove all items with multiple occurances in myList:

list(set(myList) - set([x for x in myList if myList.count(x)>1]))

cheers,
--
----
Amit Khemka -- onyomo.com
Home Page: www.cse.iitd.ernet.in/~csd00377
Endless the world's turn, endless the sun's Spinning, Endless the quest;
I turn again, back to my own beginning, And here, find rest.

[7stud]

On Apr 3, 3:53 pm, "bahoo" <[e-mail]:

target = "0024"
l = ["0024", "haha", "0024"]

for index, val in enumerate(l):
if val==target:
del l[index]

print l

This latter suggestion (with the for loop) seems to be buggy: if there
are multiple items in the list "l" equal to "target", then only the
first one will be removed!

Thanks anyways.

Prove it.

[Steven D'Aprano]

On Wed, 04 Apr 2007 00:59:23 -0700, 7stud wrote:

On Apr 3, 3:53 pm, "bahoo" <[e-mail]:

target = "0024"
l = ["0024", "haha", "0024"]

for index, val in enumerate(l):
if val==target:
del l[index]

print l

This latter suggestion (with the for loop) seems to be buggy: if there
are multiple items in the list "l" equal to "target", then only the
first one will be removed!

Thanks anyways.

Prove it.

Try replacing l = ["0024", "haha", "0024"]
with

l = ["0024", "0024", "haha"]

and re-running the code.
Actually, the description of the bug isn't quite right. The behaviour of
the for loop isn't defined -- sometimes it will work, sometimes it won't,
depending on how many items there are, and which of them are equal to the
target.

The reason it is buggy is that it is deleting items from the same list it
is trying to enumerate over. The only safe way to do that is to iterate
over the list backwards, only deleting items you've already seen and won't
go over again:

for i in range(len(my_list), -1, -1)):
if condition:
del my_list[i]
I don't think reversed() will help you here, but I can't check for sure
because I've only got Python 2.3 on this system.
--
Steven D'Aprano

[mik3l3374]

7stud wrote:

On Apr 3, 3:53 pm, "bahoo" <[e-mail]:

target = "0024"
l = ["0024", "haha", "0024"]

for index, val in enumerate(l):
if val==target:
del l[index]

print l

This latter suggestion (with the for loop) seems to be buggy: if there
are multiple items in the list "l" equal to "target", then only the
first one will be removed!

Thanks anyways.

Prove it.

here's something:

>>l = ["0024", "haha", "0024", "0024", "sfs"]
target = "0024"
for index, val in enumerate(l):

.... print "index: " , index , "value: " , val
.... del l[index]
.... print "after del: " , l
....
index: 0 value: 0024
after del: ['haha', '0024', '0024', 'sfs']
index: 1 value: 0024
after del: ['haha', '0024', 'sfs']
index: 2 value: sfs
after del: ['haha', '0024']

>>l

['haha', '0024']

[mik3l3374]

On Apr 4, 2:20 am, "bahoo" <[e-mail]:

Hi,

I have a list like ['0024', 'haha', '0024']
and as output I want ['haha']

If I
myList.remove('0024')

then only the first instance of '0024' is removed.

It seems like regular expressions is the rescue, but I couldn't find
the right tool.

Thanks!
bahoo

how about this:

>>target = "0024"
l = ["0024", "haha", "0024", "0024", "sfs"]
result = [ item for item in l if item != target]
result

['haha', 'sfs']

[=?ISO-8859-1?Q?Ma=EBl_Benjamin_Mettler?=]

How about:

list(frozenset(['0024', 'haha', '0024']))

mi*******@gmail.com schrieb:

On Apr 4, 2:20 am, "bahoo" <[e-mail]:

>Hi,

I have a list like ['0024', 'haha', '0024']
and as output I want ['haha']

If I
myList.remove('0024')

then only the first instance of '0024' is removed.

It seems like regular expressions is the rescue, but I couldn't find
the right tool.

Thanks!
bahoo

how about this:

>>>target = "0024"
l = ["0024", "haha", "0024", "0024", "sfs"]
result = [ item for item in l if item != target]
result

['haha', 'sfs']

[Bruno Desthuilliers]

bahoo a écrit :

Hi,

I have a list like ['0024', 'haha', '0024']
and as output I want ['haha']

If I
myList.remove('0024')

then only the first instance of '0024' is removed.

It seems like regular expressions is the rescue,

Nope. re wouldn't help here - it works on strings, not on lists (you
need to understand that in Python, strings are *not* 'lists of chars').

Use list comps:
mylist = [item for item in mylist if item != '0024']

or filter() :
mylist = filter(lambda item: item != '0024', mylist)

but I couldn't find
the right tool.

May I second Grant Edward's suggestion in previous thread ? You'd really
do yourself a favor by following a couple Python tutorials. I'd say the
one in the official doc, and Dive into Python.

[Goldfish]

I don't think I would use sets at all. They change the semantic
meaning of the original list. What if you have duplicate entries that
you want to keep? Converting to a set and back will strip out the
duplicates of that.

[7stud]

On Apr 4, 3:08 am, Steven D'Aprano <[e-mail]>
wrote:

On Wed, 04 Apr 2007 00:59:23 -0700, 7stud wrote:

On Apr 3, 3:53 pm, "bahoo" <[e-mail]:

target = "0024"
l = ["0024", "haha", "0024"]

for index, val in enumerate(l):
if val==target:
del l[index]

print l

This latter suggestion (with the for loop) seems to be buggy: if there
are multiple items in the list "l" equal to "target", then only the
first one will be removed!

Thanks anyways.

Prove it.

Try replacing l = ["0024", "haha", "0024"]
with

l = ["0024", "0024", "haha"]

and re-running the code.

Actually, the description of the bug isn't quite right. The behaviour of
the for loop isn't defined -- sometimes it will work, sometimes it won't,
depending on how many items there are, and which of them are equal to the
target.

Thank you for the explanation.

[Ayaz Ahmed Khan]

"kyosohma" typed:

If you want to get really fancy, you could do a list comprehension
too:

your_list = ["0024","haha","0024"]
new_list = [i for i in your_list if i != '0024']

Or, just:

In [1]: l = ["0024","haha","0024"]
In [2]: filter(lambda x: x != "0024", l)
Out[2]: ['haha']

--
Ayaz Ahmed Khan

Do what comes naturally now. Seethe and fume and throw a tantrum.

[Steven Bethard]

Ayaz Ahmed Khan wrote:

"kyosohma" typed:

>If you want to get really fancy, you could do a list comprehension
too:

your_list = ["0024","haha","0024"]
new_list = [i for i in your_list if i != '0024']

Or, just:

In [1]: l = ["0024","haha","0024"]
In [2]: filter(lambda x: x != "0024", l)
Out[2]: ['haha']

Only if you want to make your code harder to read and slower::

$ python -m timeit -s "L = ['0024', 'haha', '0024']"
"[i for i in L if i != '0024']"
1000000 loops, best of 3: 0.679 usec per loop

$ python -m timeit -s "L = ['0024', 'haha', '0024']"
"filter(lambda i: i != '1024', L)"
1000000 loops, best of 3: 1.38 usec per loop

There really isn't much use for filter() anymore. Even in the one place
I would have expected it to be faster, it's slower::

$ python -m timeit -s "L = ['', 'a', '', 'b']" "filter(None, L)"
1000000 loops, best of 3: 0.789 usec per loop

$ python -m timeit -s "L = ['', 'a', '', 'b']" "[i for i in L if i]"
1000000 loops, best of 3: 0.739 usec per loop

STeVe

[Michael J. Fromberger]

In article <11**********************@n59g2000hsh.googlegroups .com>,
"bahoo" <b8*******@yahoo.comwrote:

Hi,

I have a list like ['0024', 'haha', '0024']
and as output I want ['haha']

If I
myList.remove('0024')

then only the first instance of '0024' is removed.

It seems like regular expressions is the rescue, but I couldn't find
the right tool.

If you know in advance which items are duplicated, then there have been
several simple solutions already proposed. Here's another way to tackle
the problem of removing ANY duplicated item from the list (i.e., any
string that appears 1 time).

def killdups(lst):
"""Filter duplicated elements from the input list, and return the
remaining (unique) items in their original order.
"""
count = {}
for elt in lst:
count[elt] = count.get(elt, 0) + 1
return [elt for elt in lst if count[elt] == 1]

This solution is not particularly tricky, but it has the nice properties
that:

1. It works on lists of any hashable type, not just strings,
2. It preserves the order of the unfiltered items,
3. It makes only two passes over the input list.

Cheers,
-M

--
Michael J. Fromberger | Lecturer, Dept. of Computer Science
http://www.dartmouth.edu/~sting/ | Dartmouth College, Hanover, NH, USA

[Alex Martelli]

Amit Khemka <kh********@gmail.comwrote:

On 3 Apr 2007 11:20:33 -0700, bahoo <b8*******@yahoo.comwrote:

Hi,

I have a list like ['0024', 'haha', '0024']
and as output I want ['haha']

If I
myList.remove('0024')

then only the first instance of '0024' is removed.

To remove all items with multiple occurances in myList:

list(set(myList) - set([x for x in myList if myList.count(x)>1]))

This approach is unfortunately quite inefficient, because each call to
myList.count is O(N), and there are O(N) such calls, so the whole thing
is O(N squared). [also, the inner square brackets are not needed, and
cause more memory to be consumed than omitting them would].
A "we-don't-need-no-stinkin'-one-liners" more relaxed approach:

import collections
d = collections.defaultdict(int)
for x in myList: d[x] += 1
list(x for x in myList if d[x]==1)

yields O(N) performance (give that dict-indexing is about O(1)...).
Collapsing this kind of approach back into a "one-liner" while keeping
the O(N) performance is not easy -- whether this is a fortunate or
unfortunate ocurrence is of course debatable. If we had a "turn
sequence into bag" function somewhere (and it might be worth having it
for other reasons):

def bagit(seq):
import collections
d = collections.defaultdict(int)
for x in seq: d[x] += 1
return d

then one-linerness might be achieved in the "gimme nonduplicated items"
task via a dirty, rotten trick...:

list(x for bag in [bagit(myList)] for x in myList if bag[x] == 1)

....which I would of course never mention in polite company...
Alex

[Paul Rubin]

"bahoo" <b8*******@yahoo.comwrites:

I have a list like ['0024', 'haha', '0024']
and as output I want ['haha']

If I
myList.remove('0024')

then only the first instance of '0024' is removed.

[x for x in myList if x != '0024']

[Hendrik van Rooyen]

On Apr 3, 3:53 pm, "bahoo" <[e-mail]:

My first submission handles duplicates, but not triplicates and more.
Here is one that seems a bit more bulletproof:

duplist = [1, 2, 3, 4, 'haha', 1, 2, 3, 4, 5, 1,2,3,4,6,7,7,7,7,7]
copylist = duplist[:]
fullset = set(duplist)
for x in duplist:
del(copylist[copylist.index(x)])
if x in copylist:
if x in fullset:
fullset.remove(x)

print list(fullset)

when it is run, I get:

IDLE 1.1.3 ==== No Subprocess ====

>>>

[5, 6, 'haha']

>>>

Now how would one do it and preserve the original order?
This apparently simple problem is surprisingly FOS...
But list comprehension to the rescue :

>>>[x for x in duplist if duplist.count(x) == 1]

['haha', 5, 6]

>>>

*shakes head* duh... why does it take so long?

: - (

- Hendrik

[Ayaz Ahmed Khan]

"Steven Bethard" typed:

>Or, just:

In [1]: l = ["0024","haha","0024"]
In [2]: filter(lambda x: x != "0024", l)
Out[2]: ['haha']

Only if you want to make your code harder to read and slower::

Slower, I can see. But harder to read?

There really isn't much use for filter() anymore. Even in the one place
I would have expected it to be faster, it's slower::

$ python -m timeit -s "L = ['', 'a', '', 'b']" "filter(None, L)"
1000000 loops, best of 3: 0.789 usec per loop

$ python -m timeit -s "L = ['', 'a', '', 'b']" "[i for i in L if i]"
1000000 loops, best of 3: 0.739 usec per loop

I am getting varying results on my system on repeated runs. What about
itertools.ifilter()?

$ python -m timeit -s "L = ['0024', 'haha', '0024']; import itertools"
"itertools.ifilter(lambda i: i != '1024', L)"
100000 loops, best of 3: 5.37 usec per loop

$ python -m timeit -s "L = ['0024', 'haha', '0024']"
"[i for i in L if i != '0024']"
100000 loops, best of 3: 5.41 usec per loop

$ python -m timeit -s "L = ['0024', 'haha', '0024']" "[i for i in L if i]"
100000 loops, best of 3: 6.71 usec per loop

$ python -m timeit -s "L = ['0024', 'haha', '0024']; import itertools"
"itertools.ifilter(None, L)"
100000 loops, best of 3: 4.12 usec per loop

--
Ayaz Ahmed Khan

Do what comes naturally now. Seethe and fume and throw a tantrum.

[attn.steven.kuo]

On Apr 4, 7:43 pm, [e-mail] (Alex Martelli) wrote:

(snipped)

A "we-don't-need-no-stinkin'-one-liners" more relaxed approach:

import collections
d = collections.defaultdict(int)
for x in myList: d[x] += 1
list(x for x in myList if d[x]==1)

yields O(N) performance (give that dict-indexing is about O(1)...).

Collapsing this kind of approach back into a "one-liner" while keeping
the O(N) performance is not easy -- whether this is a fortunate or
unfortunate ocurrence is of course debatable. If we had a "turn
sequence into bag" function somewhere (and it might be worth having it
for other reasons):

def bagit(seq):
import collections
d = collections.defaultdict(int)
for x in seq: d[x] += 1
return d

then one-linerness might be achieved in the "gimme nonduplicated items"
task via a dirty, rotten trick...:

list(x for bag in [bagit(myList)] for x in myList if bag[x] == 1)

...which I would of course never mention in polite company...

Alex

With a "context managed" set one could have, as a one-liner:

with cset() as s: retval = [ (x, s.add(x))[0] for x in myList if x not
in s ]

I've not looked at the underlying implementation of set(), but I
presume that checking set membership is also about O(1).

--
Cheers,
Steven

[waylan]

On Apr 3, 6:05 pm, Steven Bethard <[e-mail]:

bahoo wrote:

The larger problem is, I have a list of strings that I want to remove
from another list of strings.

If you don't care about the resulting order::

>>items = ['foo', 'bar', 'baz', 'bar', 'foo', 'frobble']
>>to_remove = ['foo', 'bar']
>>set(items) - set(to_remove)

set(['frobble', 'baz'])

I'm surprised no one has mentioned any of the methods of set. For
instance:

>>set.difference.__doc__

'Return the difference of two sets as a new set.\n\n(i.e. all
elements that are in this set but not in the other.)'

>>>set(items).difference(to_remove)

set(['frobble', 'baz'])

There are a few other cool methods of sets that come in handy for this
sort of thing. If only order could be preserved.

>
If you do care about the resulting order::

>>to_remove = set(to_remove)
>>[item for item in items if item not in to_remove]

['baz', 'frobble']

STeVe

[Steven D'Aprano]

On Wed, 04 Apr 2007 15:56:34 +0200, Hendrik van Rooyen wrote:

Now how would one do it and preserve the original order?
This apparently simple problem is surprisingly FOS...
But list comprehension to the rescue :

>>>>[x for x in duplist if duplist.count(x) == 1]

['haha', 5, 6]

>>>>

*shakes head* duh... why does it take so long?

Because you are using Shlemiel the painter's algorithm:

http://www.joelonsoftware.com/articl...000000319.html

Each time you call duplist.count(), you go back to the beginning
of the list and walk the entire list. You end up walking the list over
and over and over again.
--
Steven.

[Alex Martelli]

Ayaz Ahmed Khan <ay**@dev.slash.nullwrote:
...

I am getting varying results on my system on repeated runs. What about
itertools.ifilter()?

Calling itertools.ifilter returns an iterator; if you never iterate on
that iterator, that, of course, is going to be very fast (O(1), since it
does not matter how long the list you _don't_ iterate on is), but why
would you care how long it takes to do no useful work at all?

$ python -m timeit -s "L = ['0024', 'haha', '0024']; import itertools"

"itertools.ifilter(lambda i: i != '1024', L)"

100000 loops, best of 3: 5.37 usec per loop

$ python -m timeit -s "L = ['0024', 'haha', '0024']"
"[i for i in L if i != '0024']"
100000 loops, best of 3: 5.41 usec per loop

Here are the numbers I see:

brain:~ alex$ python -m timeit -s "L = ['0024', 'haha', '0024'];

import itertools" "itertools.ifilter(lambda i: i != '1024', L)"

1000000 loops, best of 3: 0.749 usec per loop

This is the "we're not doing any work" timing.

brain:~ alex$ python -m timeit -s "L = ['0024', 'haha',
'0024']" "[i for i in L if i != '1024']"
1000000 loops, best of 3: 1.37 usec per loop

This is "make a list in the obvious way, excluding an item that's never
there" (like in your code's first case, I'm comparing with 1024, which
ain't there, rather than with 0024, which is).

brain:~ alex$ python -m timeit -s "L = ['0024', 'haha',
'0024']; import itertools" "list(itertools.ifilter(lambda i: i !=
'1024', L))"
100000 loops, best of 3: 6.18 usec per loop

This is the "make it the hard way" (excluding a non-present item).
About 5/6 of the overhead comes from the list constructor.

When we exclude the item that IS there twice:

brain:~ alex$ python -m timeit -s "L = ['0024', 'haha',
'0024']" "[i for i in L if i != '0024']"
1000000 loops, best of 3: 0.918 usec per loop

this is the "obvious way to do it",

brain:~ alex$ python -m timeit -s "L = ['0024', 'haha',
'0024']; import itertools" "list(itertools.ifilter(lambda i: i !=
'0024', L))"
100000 loops, best of 3: 6.16 usec per loop

and this is the "hard and contorted way".
If you only want to loop, not to build a list, itertools.ifilter (or a
genexp) may be convenient (if the original list is long); but for making
lists, list comprehensions win hand-down.

Here are a few cases of "just looping" on lists of middling size:

brain:~ alex$ python -m timeit -s "L = 123*['0024', 'haha',
'0024']" "for j in [i for i in L if i != '0024']: pass"
10000 loops, best of 3: 70 usec per loop

brain:~ alex$ python -m timeit -s "L = 123*['0024', 'haha',
'0024']" "for j in (i for i in L if i != '0024'): pass"
10000 loops, best of 3: 70.9 usec per loop

brain:~ alex$ python -m timeit -s "L = 123*['0024', 'haha',
'0024']; import itertools" "for j in itertools.ifilter(lambda i: i !=
'0024', L
): pass"
10000 loops, best of 3: 151 usec per loop

Here, the overhead of itertools.ifilter is only about twice that of a
genexp (or list comprehension; but the LC should cost relatively more as
L's length keeps growing, due to memory-allocation issues).

BTW, sometimes simplest is still best:

brain:~ alex$ python -m timeit -s "L = 123*['0024', 'haha',
'0024']" "for i in L:

if i != '0024':
pass"

10000 loops, best of 3: 52.5 usec per loop

I.e., when you're just looping... just loop!-)
Alex

[Paul McGuire]

On Apr 3, 3:47 pm, [e-mail] wrote:

Beware that converting a list to set and then back to list won't
preserve the order of the items, because the set-type loses the order.

Also beware that this conversion will remove duplicates, so that if
'haha' is in the original list multiple times, there will only be one
after converting to a set and then back to a list. Only you can tell
us whether this behavior is desirable or not.

-- Paul