Fate of itertools.dropw hile() and itertools.takew hile()

[Raymond Hettinger]

I'm considering deprecating these two functions and would like some
feedback from the community or from people who have a background in
functional programming.

* I'm concerned that use cases for the two functions are uncommon and
can obscure code rather than clarify it.

* I originally added them to itertools because they were found in
other functional languages and because it seemed like they would serve
basic building blocks in combination with other itertools allow
construction of a variety of powerful, high-speed iterators. The
latter may have been a false hope -- to date, I've not seen good
recipes that depend on either function.

* If an always true or always false predicate is given, it can be hard
to break-out of the function once it is running.

* Both functions seem simple and basic until you try to explain them
to someone else. Likewise, when reading code containing dropwhile(),
I don't think it is self-evident that dropwhile() may have a lengthy
start-up time.

* Since itertools are meant to be combined together, the whole module
becomes easier to use if there are fewer tools to choose from.

These thoughts reflect my own experience with the itertools module.
It may be that your experience with them has been different. Please
let me know what you think.

Raymond

[Istvan Albert]

On Dec 29, 6:10 pm, Raymond Hettinger <[e-mail] wrote:

These thoughts reflect my own experience with the itertools module.
It may be that your experience with them has been different. Please
let me know what you think.

first off, the itertools module is amazing, thanks for creating it. It
changed the way I think about programming. In fact nowadays I start
all my programs with:

from itertools import *

which may not be the best form, but I got tired of importing every
single function individually or writing out the module name.

Now I never needed the dropwhile() and takewhile() functions, but that
may not mean much. For quite a while I never needed the repeat()
function either. It even looked nonsensical to have an iterator that
simply repeats the same thing over and over. One day I had to solve a
problem that needed repeat() and made me really understand what it was
for and got to marvel at a just how neat the solution was.

i.

[Steven D'Aprano]

On Sat, 29 Dec 2007 15:10:24 -0800, Raymond Hettinger wrote:

* Both functions seem simple and basic until you try to explain them to
someone else.

Oh I don't know about that. The doc strings seem to do an admirable job
to me. Compared to groupby(), the functions are simplicity themselves.

Likewise, when reading code containing dropwhile(), I
don't think it is self-evident that dropwhile() may have a lengthy
start-up time.

*scratches head in confusion*

It isn't? I can understand somebody *under*estimati ng the start-up time
(perhaps because they overestimate how quickly dropwhile() can iterate
through the items). But surely it is self-evident that a function which
drops items has to drop the items before it can start returning?

* Since itertools are meant to be combined together, the whole module
becomes easier to use if there are fewer tools to choose from.

True, but on the other hand a toolbox with too few tools is harder to use
than one with too many tools.

--
Steven

[bearophileHUGS]

Almost every day I write code that uses itertools, so I find it very
useful, and its functions fast.
Removing useless things and keeping things tidy is often positive. But
I can't tell you what to remove. Here are my usages (every sub-list is
sorted by inverted frequency usage):

I use often or very often:
groupby( iterable[, key])
imap( function, *iterables)
izip( *iterables)
ifilter( predicate, iterable)
islice( iterable, [start,] stop [, step])

I use once in while:
cycle( iterable)
chain( *iterables)
count( [n])
repeat( object[, times])

I have used probably one time or few times:
starmap( function, iterable)
tee( iterable[, n=2])
ifilterfalse( predicate, iterable)

Never used so far:
dropwhile( predicate, iterable)
takewhile( predicate, iterable)

Bye,
bearophile

[Michele Simionato]

On Dec 30, 12:10 am, Raymond Hettinger <[e-mail] wrote:

I'm considering deprecating these two functions and would like some
feedback from the community or from people who have a background in
functional programming.

I am with Steven D'Aprano when he says that takewhile and dropwhile
are clear enough. On the other hand, in my code
base I have exactly zero occurrences of takewhile and
dropwhile, even if I tend to use the itertools quite
often. That should be telling. If my situations is
common, that means that takewhile and dropwhile are
useless in practice and should be deprecated.
But I will wait for other respondents. It may just be
that I never needed them. I presume you did scans of
large code bases and you did not find occurrences of
takewhile and dropwhile, right?
Michele Simionato

[Marc 'BlackJack' Rintsch]

On Sat, 29 Dec 2007 15:10:24 -0800, Raymond Hettinger wrote:

These thoughts reflect my own experience with the itertools module.
It may be that your experience with them has been different. Please
let me know what you think.

I seem to be in a minority here as I use both functions from time to time.
One "recipe" is extracting blocks from text files that are delimited by a
special start and end line.

def iter_block(line s, start_marker, end_marker):
return takewhile(lambd a x: not x.startswith(en d_marker),
dropwhile(lambd a x: not x.startswith(st art_marker),
lines))

Maybe these functions usually don't turn up in code that can be called
"recipes" so often but are useful for themselves.

Ciao,
Marc 'BlackJack' Rintsch

[Istvan Albert]

On Dec 30, 3:29 am, Marc 'BlackJack' Rintsch <[e-mail] wrote:

One "recipe" is extracting blocks from text files that are delimited by a
special start and end line.

Neat solution!

I actually need such functionality every once in a while.

Takewhile + dropwhile to the rescue!

i.

[George Sakkis]

On Dec 30, 4:12 pm, Istvan Albert <istvan.alb...@ gmail.comwrote:

On Dec 30, 3:29 am, Marc 'BlackJack' Rintsch <[e-mail] wrote:

One "recipe" is extracting blocks from text files that are delimited by a
special start and end line.

Neat solution!

I actually need such functionality every once in a while.

Takewhile + dropwhile to the rescue!

i.

On at least one thread and a recipe for this task (http://
aspn.activestat e.com/ASPN/Cookbook/Python/Recipe/521877), the proposed
solutions involved groupby() with an appropriate key function. The
takewhile/dropwhile solution seems shorter and (maybe) easier to read
but perhaps not as flexible and general. Regardless, it's a good
example of takewhile/dropwhile.

George

[Raymond Hettinger]

[bearophile]

Here are my usages (every sub-list is
sorted by inverted frequency usage):

I use often or very often:
groupby( iterable[, key])
imap( function, *iterables)
izip( *iterables)
ifilter( predicate, iterable)
islice( iterable, [start,] stop [, step])

I use once in while:
cycle( iterable)
chain( *iterables)
count( [n])
repeat( object[, times])

I have used probably one time or few times:
starmap( function, iterable)
tee( iterable[, n=2])
ifilterfalse( predicate, iterable)

Never used so far:
dropwhile( predicate, iterable)
takewhile( predicate, iterable)

Thank you for the useful and informative response.
Raymond

[Raymond Hettinger]

[Michele Simionato]

in my code
base I have exactly zero occurrences of takewhile and
dropwhile, even if I tend to use the itertools quite
often. That should be telling.

Thanks for the additional empirical evidence.

I presume you did scans of
large code bases and you did not find occurrences of
takewhile and dropwhile, right?

Yes.
Raymond