can list comprehensions replace map?

[David Isaac]

Newbie question:

I have been generally open to the proposal that list comprehensions
should replace 'map', but I ran into a need for something like
map(None,x,y)
when len(x)>len(y). I cannot it seems use 'zip' because I'll lose
info from x. How do I do this as a list comprehension? (Or,
more generally, what is the best way to do this without 'map'?)

Thanks,
Alan Isaac

[Michael Hoffman]

David Isaac wrote:

Newbie question:

I have been generally open to the proposal that list comprehensions
should replace 'map', but I ran into a need for something like
map(None,x,y)
when len(x)>len(y). I cannot it seems use 'zip' because I'll lose
info from x. How do I do this as a list comprehension? (Or,
more generally, what is the best way to do this without 'map'?)

It ain't broke so I'd stick with what you're doing. Even if map() is
removed as a builtin, it will surely stick around in a module.
--
Michael Hoffman

[Michael Hoffman]

Michael Hoffman wrote:

David Isaac wrote:

Newbie question:

I have been generally open to the proposal that list comprehensions
should replace 'map', but I ran into a need for something like
map(None,x,y)
when len(x)>len(y). I cannot it seems use 'zip' because I'll lose
info from x. How do I do this as a list comprehension? (Or,
more generally, what is the best way to do this without 'map'?)

It ain't broke so I'd stick with what you're doing. Even if map() is
removed as a builtin, it will surely stick around in a module.

Addendum: I know this doesn't answer your question, so if you were
asking out of purely academic interest, then someone else will probably
post another answer.
--
Michael Hoffman

[Larry Bates]

This isn't really a question about list
comprehensions as you are using a "feature"
of map by passing None as the function to be
executed over each list element:

This works when len(x) > len(y):

zip(x,y+(len(x)-len(y))*[None])

This works when len(y) >=0 len(x):

zip(x+(len(x)-len(y))*[None],y)

I would probably wrap into function:

def foo(x,y):
if len(x) > len(y):
return zip(x,y+(len(x)-len(y))*[None])

return zip(x+(len(x)-len(y))*[None],y)

Larry Bates

David Isaac wrote:

Newbie question:

I have been generally open to the proposal that list comprehensions
should replace 'map', but I ran into a need for something like
map(None,x,y)
when len(x)>len(y). I cannot it seems use 'zip' because I'll lose
info from x. How do I do this as a list comprehension? (Or,
more generally, what is the best way to do this without 'map'?)

Thanks,
Alan Isaac

[Paolino]

David Isaac wrote:

Newbie question:

I have been generally open to the proposal that list comprehensions
should replace 'map', but I ran into a need for something like
map(None,x,y)
when len(x)>len(y). I cannot it seems use 'zip' because I'll lose
info from x. How do I do this as a list comprehension? (Or,
more generally, what is the best way to do this without 'map'?)
Probably zip should change behaviour,and cover that case or at least
have another like 'tzip' in the __builtins__ .Dunno, I always thought
zip should not cut to the shortest list. Thanks,
Alan Isaac

[Andrew Dalke]

David Isaac wrote:

I have been generally open to the proposal that list comprehensions
should replace 'map', but I ran into a need for something like
map(None,x,y)
when len(x)>len(y). I cannot it seems use 'zip' because I'll lose
info from x. How do I do this as a list comprehension? (Or,
more generally, what is the best way to do this without 'map'?)

If you know that len(x)>=len(y) and you want the same behavior as
map() you can use itertools to synthesize a longer iterator

x = [1,2,3,4,5,6]
y = "Hi!"
from itertools import repeat, chain
zip(x, chain(y, repeat(None))) [(1, 'H'), (2, 'i'), (3, '!'), (4, None), (5, None), (6, None)]

This doesn't work if you want the result to be max(len(x), len(y))
in length - the result has length len(x).

As others suggested, if you want to use map, go ahead. It won't
disappear for a long time and even if it does it's easy to
retrofit if needed.

Andrew
da***@dalkescie ntific.com

[Raymond Hettinger]

[David Isaac]

I have been generally open to the proposal that list comprehensions
should replace 'map', but I ran into a need for something like
map(None,x,y)
when len(x)>len(y). I cannot it seems use 'zip' because I'll lose
info from x. How do I do this as a list comprehension? (Or,
more generally, what is the best way to do this without 'map'?)

[Paolino] Probably zip should change behaviour,and cover that case or at least
have another like 'tzip' in the __builtins__ .Dunno, I always thought
zip should not cut to the shortest list.

Heck no! For the core use case of lockstep iteration, it is almost
always a mistake to continue iterating beyond the length of the
shortest input sequence. Even for map(), the use cases are thin. How
many functions do something meaningful when one or more of their inputs
changes type and becomes a stream of Nones. Consider for example,
map(pow, seqa, seqb) -- what good can come of one sequence or the other
suddenly switching to a None mode?

As Andrew pointed out, if you really need that behavior, it can be
provided explicity. See the padNone() recipe in the itertools
documentation for an easy one-liner.

IMO, reliance on map's None fill-in feature should be taken as a code
smell indicating a design flaw (not always, but usually). There is a
reason that feature is missing from map() implementations in some other
languages.

In contrast, the existing behavior of zip() is quite useful. It allows
some of the input sequences to be infinite:

zip(itertools.c ount(1), open('myfile.tx t'))

Raymond

[Steven Bethard]

David Isaac wrote:

I ran into a need for something like map(None,x,y)
when len(x)>len(y). I cannot it seems use 'zip' because I'll lose
info from x.

I almost never run into this situation, so I'd be interested to know why
you need this. Here's one possible solution:

py> import itertools as it
py> def zipfill(*lists) :
.... max_len = max(len(lst) for lst in lists)
.... return zip(*[it.chain(lst, it.repeat(None, max_len - len(lst)))
.... for lst in lists])
....
py> zipfill(range(4 ), range(5), range(3))
[(0, 0, 0), (1, 1, 1), (2, 2, 2), (3, 3, None), (None, 4, None)]

If you prefer, you can replace the call to zip with it.zip and get an
iterator back instead of a list.

STeVe

[Paolino]

Raymond Hettinger wrote:

[David Isaac]

I have been generally open to the proposal that list comprehensions
should replace 'map', but I ran into a need for something like
map(None,x,y )
when len(x)>len(y). I cannot it seems use 'zip' because I'll lose
info from x. How do I do this as a list comprehension? (Or,
more generally, what is the best way to do this without 'map'?)

[Paolino]

Probably zip should change behaviour,and cover that case or at least
have another like 'tzip' in the __builtins__ .Dunno, I always thought
zip should not cut to the shortest list.

Heck no! For the core use case of lockstep iteration, it is almost
always a mistake to continue iterating beyond the length of the
shortest input sequence. Even for map(), the use cases are thin. How
many functions do something meaningful when one or more of their inputs
changes type and becomes a stream of Nones. Consider for example,
map(pow, seqa, seqb) -- what good can come of one sequence or the other
suddenly switching to a None mode?

As Andrew pointed out, if you really need that behavior, it can be
provided explicity. See the padNone() recipe in the itertools
documentation for an easy one-liner.

IMO, reliance on map's None fill-in feature should be taken as a code
smell indicating a design flaw (not always, but usually). There is a
reason that feature is missing from map() implementations in some other
languages.

In contrast, the existing behavior of zip() is quite useful. It allows
some of the input sequences to be infinite:

zip(itertools.c ount(1), open('myfile.tx t'))

Right point.
Well, for my little experiences use cases in which the lists have different
lengths are rare, but in those cases I don't see the reason of not being
able
to zip to the longest one.What is really strange is that I have to use
map(None,....) for that,instead of another zip-like function which ,at
least
would be intutitive for the average user.Also map(None,...) looks like a
super-hack
and it's not elegant or readable or logic (IMO)

I think zip comes to substitute the tuple.__new__ untolerant
implementation. A dumb like me wuold expect map(tuple,[1,2,3],[2,3,4]) to
work, so pretending map(None,....) would do it is like saying that None
and tuple are near concepts, which is obviously an absurdity.

Thanks anyway, for explanations.

Paolino

Raymond

[Andrew Dalke]

Steven Bethard wrote:

Here's one possible solution:

py> import itertools as it
py> def zipfill(*lists) :
... max_len = max(len(lst) for lst in lists)

A limitation to this is the need to iterate over the
lists twice, which might not be possible if one of them
is a file iterator.

Here's a clever, though not (in my opinion) elegant solution

import itertools

def zipfill(*seqs):
count = [len(seqs)]
def _forever(seq):
for item in seq: yield item
count[0] -= 1
while 1: yield None
seqs = [_forever(seq) for seq in seqs]
while 1:
x = [seq.next() for seq in seqs]
if count == [0]:
break
yield x

for x in zipfill("This", "is", "only", "a", "test."):
print x

This generates

['T', 'i', 'o', 'a', 't']
['h', 's', 'n', None, 'e']
['i', None, 'l', None, 's']
['s', None, 'y', None, 't']
[None, None, None, None, '.']

This seems a bit more elegant, though the "replace" dictionary is
still a bit of a hack

from itertools import repeat, chain, izip

sentinel = object()
end_of_stream = repeat(sentinel )

def zipfill(*seqs):
replace = {sentinel: None}.get
seqs = [chain(seq, end_of_stream) for seq in seqs]
for term in izip(*seqs):
for element in term:
if element is not sentinel:
break
else:
# All sentinels
break

yield [replace(element , element) for element in term]
(I originally had a "element == tuple([sentinel]*len(seqs))" check
but didn't like all the == tests incurred.)

Andrew
da***@dalkescie ntific.com