Alternative to standard C "for"

[adomas.paltanavicius]

Hi there,

I am quite new to Python, and have a straight & simple question.
In C, there is for (init; cond; advance). We all know that.
In Python there are two ways to loop over i=A..B (numerical.):
1) i = A
while i<B:
...do something...
i+=STEP
2) for i in range(A, B, STEP):
...do something...

First case looks quite nasty, because it's for more complicated
things, not numerical loops. Second is very nice, but with there's
problem. If i do ..in range(1, 100000000).. (what I really need
sometimes), it takes few hundred megs of memory and slows
down. Are there other good ways for this simple problem? Generators?

Adomas

[Paul Rubin]

ad******************@gmail.com writes:

problem. If i do ..in range(1, 100000000).. (what I really need
sometimes), it takes few hundred megs of memory and slows
down. Are there other good ways for this simple problem? Generators?

use xrange instead of range.

[BJörn Lindqvist]

> I am quite new to Python, and have a straight & simple question.

In C, there is for (init; cond; advance). We all know that.
In Python there are two ways to loop over i=A..B (numerical.):
1) i = A
while i<B:
...do something...
i+=STEP

This is indeed quite ugly. You rarely need such loops in Python and
with some thinking you can often translate the C-equivalent to
something more pythonic. As you guessed, your second problem is best
solved with a generator function - xrange(). It is completely equal to
range() except that it returns a generator instead of a list.
--
mvh Björn

[Daniel Bickett]

ad******************@gmail.com wrote:

Are there other good ways for this simple problem? Generators?

Very interesting problem :) That never occured to me.

To prevent python from loading that entire list into memory, one
could, as you suggested, use a generator:

def genrange( start , stop , step = 1 ): while start < stop:
yield start
start += step
for x in range( 5 ): print "%s " % str( x ),

0 1 2 3 4
for x in genrange( 0 , 5 ):

print "%s " % str( x ),

0 1 2 3 4

--
Daniel Bickett
dbickett at gmail.com
http://heureusement.org/

[Daniel Bickett]

Paul Rubin wrote:

use xrange instead of range.

Woops ;) I wasn't aware such a function existed.

apologies-for-reinventing-the-wheel-ly y'rs,
--
Daniel Bickett
dbickett at gmail.com
http://heureusement.org/

[Georg Brandl]

BJörn Lindqvist wrote:

I am quite new to Python, and have a straight & simple question.
In C, there is for (init; cond; advance). We all know that.
In Python there are two ways to loop over i=A..B (numerical.):
1) i = A
while i<B:
...do something...
i+=STEP

This is indeed quite ugly. You rarely need such loops in Python and
with some thinking you can often translate the C-equivalent to
something more pythonic. As you guessed, your second problem is best
solved with a generator function - xrange(). It is completely equal to
range() except that it returns a generator instead of a list.

Slight terminology glitch -- it does return an iterator, not a
generator. Generators are functions that return iterators.

regards,
Georg

[Diez B. Roggisch]

> First case looks quite nasty, because it's for more complicated

things, not numerical loops. Second is very nice, but with there's
problem. If i do ..in range(1, 100000000).. (what I really need
sometimes), it takes few hundred megs of memory and slows
down. Are there other good ways for this simple problem? Generators?

Use xrange(). It computes the values as they are needed - not an entire
list.
--
Regards,

Diez B. Roggisch

[Alex Martelli]

Georg Brandl <g.**************@gmx.net> wrote:

Slight terminology glitch -- it does return an iterator, not a
generator. Generators are functions that return iterators.

xrange returns an ITERABLE, not an ITERATOR. Videat:

a = xrange(23, 43)
a.next() Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'xrange' object has no attribute 'next'

No next method -> not an iterator. iter(xrange(...)) DOES return an
iterator, btw.
Alex

[Georg Brandl]

Alex Martelli wrote:

Georg Brandl <g.**************@gmx.net> wrote:

Slight terminology glitch -- it does return an iterator, not a
generator. Generators are functions that return iterators.

xrange returns an ITERABLE, not an ITERATOR. Videat:

a = xrange(23, 43)
a.next() Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'xrange' object has no attribute 'next'

No next method -> not an iterator. iter(xrange(...)) DOES return an
iterator, btw.

*wince*

This was SO clear...

Georg

[Georg Brandl]

Alex Martelli wrote:

Georg Brandl <g.**************@gmx.net> wrote:

Slight terminology glitch -- it does return an iterator, not a
generator. Generators are functions that return iterators.

xrange returns an ITERABLE, not an ITERATOR. Videat:

a = xrange(23, 43)
a.next() Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'xrange' object has no attribute 'next'

No next method -> not an iterator. iter(xrange(...)) DOES return an
iterator, btw.

Thanks! Somehow it's always these little corrections that contain errors.

And, considering your other posts today, I got away quite well...

Georg

[Steve Holden]

ad******************@gmail.com wrote:

Hi there,

I am quite new to Python, and have a straight & simple question.
In C, there is for (init; cond; advance). We all know that.
In Python there are two ways to loop over i=A..B (numerical.):
1) i = A
while i<B:
...do something...
i+=STEP
2) for i in range(A, B, STEP):
...do something...

First case looks quite nasty, because it's for more complicated
things, not numerical loops. Second is very nice, but with there's
problem. If i do ..in range(1, 100000000).. (what I really need
sometimes), it takes few hundred megs of memory and slows
down. Are there other good ways for this simple problem? Generators?

Adomas

This question is so important to you you had to post it twelve times? :-)

Look at "xrange", which is similar to range but doesn't create the whole
list.

regards
Steve
--
Meet the Python developers and your c.l.py favorites March 23-25
Come to PyCon DC 2005 http://www.pycon.org/
Steve Holden http://www.holdenweb.com/

[James Stroud]

On Sat, 2005-02-05 at 10:49, BJörn Lindqvist wrote:

I am quite new to Python, and have a straight & simple question.
In C, there is for (init; cond; advance). We all know that.
In Python there are two ways to loop over i=A..B (numerical.):
1) i = A
while i<B:
...do something...
i+=STEP

This is indeed quite ugly. You rarely need such loops in Python and
with some thinking you can often translate the C-equivalent to
something more pythonic. As you guessed, your second problem is best
solved with a generator function - xrange(). It is completely equal to
range() except that it returns a generator instead of a list.

It seems I need constructs like this all of the time

i = 0
while i < len(somelist):
if oughta_pop_it(somelist[i]):
somelist.pop(i)
else:
i += 1

There has to be a better way...

Any thoughts?

James

-----------------
7. Obsession with National Security
Fear is used as a motivational tool by the government over the masses.

[Kent Johnson]

James Stroud wrote:

It seems I need constructs like this all of the time

i = 0
while i < len(somelist):
if oughta_pop_it(somelist[i]):
somelist.pop(i)
else:
i += 1

There has to be a better way...

somelist[:] = [ item for item in somelist if not oughta_pop_it(item) ]

Kent

[Michael Spencer]

James Stroud wrote:

It seems I need constructs like this all of the time

i = 0
while i < len(somelist):
if oughta_pop_it(somelist[i]):
somelist.pop(i)
else:
i += 1

There has to be a better way...

Do you have to modify your list in place?

If not, just create a copy with the filtered items:

somelist = [item for item in somelist if not oughta_pop_it(item)]

or you could use filter or itertools.ifilter to do much the same thing

Michael

[Jeremy Bowers]

On Thu, 17 Feb 2005 10:22:01 -0800, James Stroud wrote:

It seems I need constructs like this all of the time

i = 0
while i < len(somelist):
if oughta_pop_it(somelist[i]):
somelist.pop(i)
else:
i += 1

There has to be a better way...

Any thoughts?

Others have pointed out

somelist = [x for x in somelist if not oughta_pop_it(x)]

but I'd also point out what I haven't seen, that your algorithm is O(n^2)
making fairly reasonable assumptions, whereas the list comprehension is
O(n). For long lists, that adds up fast. In fact, if you pop two things
off in the first half of the list, you've already lost and everything
after that is a waste.

The only time you want to do that is when you're memory-constrained, since
it will take less memory... and I gotta say, I don't think Python is the
best "memory-constrained" language in the world :-) Besides, if you're
really dealing with that much data, it's probably time to use a database
of some sort and tell *it* what things to eliminate.

[John Machin]

James Stroud wrote:

It seems I need constructs like this all of the time

i = 0
while i < len(somelist):
if oughta_pop_it(somelist[i]):
somelist.pop(i)
else:
i += 1

There has to be a better way...

! for i in xrange(len(somelist)-1, -1, -1):
! if oughta_pop_it(somelist[i]):
! del somelist[i] # why use pop??

We don't need no stinking iterators; we're the
Python_1.5_or_earlier_istas!