using range() in for loops

[John Salerno]

I'm reading Text Processing in Python right now and I came across a
comment that is helping me to see for loops in a new light. I think
because I'm used to the C-style for loop where you create a counter
within the loop declaration, for loops have always seemed to me to be
about doing something a certain number of times, and not about iterating
over an object.

The reason for this distinction comes from the fact that I read a lot
how using range and for is somewhat discouraged, because it doesn't
really use a for loop for it's true purpose. So my question is, is this
just a Python-oriented opinion about for loops, or is it a general idea?

Also, what if you *do* need to just do something a set number of times.
Is this okay, or does it mean you are approaching the problem
incorrectly? Using for and range together seems to be a common idiom,
yet at the same time discouraged, so I'm wondering what is a good balance.

Thanks.

[Adam DePrince]

On Tue, 2006-04-04 at 21:54 -0400, John Salerno wrote:

I'm reading Text Processing in Python right now and I came across a
comment that is helping me to see for loops in a new light. I think
because I'm used to the C-style for loop where you create a counter
within the loop declaration, for loops have always seemed to me to be
about doing something a certain number of times, and not about iterating
over an object.
This is a normal reaction. Try to keep in mind that when you use any
"higher level" language your code will closer reflect what you want
rather than how to accomplish what you want. When you are creating a
loop over numbers, is your goal really to count, or to use the integer
that you are counting to do something else, like perhaps dereference
elements of a string.

Also, part of the problem is a preconceived restriction that you hold on
the use of forloops. They arn't just for counting, there are all sorts
of interesting things that can go in there. Remember, a for loop is
basically a while loop with a little bit of syntactic sugar. Look at
this:
for( a, b, c ) {
d;
e;
}

is the same as

a;
while (b)
{
d;
e;
c;
}
C for and while are the same creature. Python's while loop is Python's
version of for/while. If you wanted to mimick C you could write

i = 0
while( i<10 ):
print i
i+=1

but this is clumbsy and slower. for(i=0;i<10;i+ +) is a common enough
programing pattern, you arn't really interested in setting i,
incrementing, doing all of that housekeeping. You really want to repeat
10 times with with i set to 0, 1 ...

for ... xrange does this well and is somewhat the motivation for the
creation of xrange.
The reason for this distinction comes from the fact that I read a lot
how using range and for is somewhat discouraged, because it doesn't
really use a for loop for it's true purpose. So my question is, is this
Nothing in a well defined language has a true purpose. The true purpose
of an int in C isn't to be for loop fodder. One of the hallmarks of a
well designed language is orthogonality; most anything works with
anything else. A feature that has one specific use doesn't provide much
utility to justify the effort used to create it.

just a Python-oriented opinion about for loops, or is it a general idea?
Programming languages borrow from heavily from natural languages; yes,
for is a loaned word from English that exists in C, python ... lots of
languages. But just as when one human language borrows from another,
the resulting semantics are not always the same.

Each as a formal semantic. C and Python are somewhat different, and
yes, you could describe the philosophical difference as a matter of
opinions.

Also, what if you *do* need to just do something a set number of times.
Is this okay, or does it mean you are approaching the problem
incorrectly? Using for and range together seems to be a common idiom,
yet at the same time discouraged, so I'm wondering what is a good balance.

The correct idiom is for( xrange( foo )). This is preferred over range
for efficiency.

Historically there were no iters. If you wanted to do loop you would
say

for x in range( 10 ):
foo
bar

range created a list of 10 items and x marched over them. Creating this
list in advance doesn't need to take any more time; you have to create
the number objects anyway to assign to x at some point, so you might as
well get that done with upfront.

The problem was memory consumption. The memory requirements of the
list put a bound on large your iteration could be.

xrange was created, and soon followed general iters. Now, the range
object generates the numbers on the fly as the loop runs .. the loop
says "hey, whats next" and the next item is returned.

When people say "don't say for x in range" they are really saying "use
xrange instead of range."

[Sion Arrowsmith]

AndyL <ask@me> wrote:

Paul Rubin wrote:

Normally you'd use range or xrange. range builds a complete list in
memory so can be expensive if the number is large. xrange just counts
up to that number.

so when range would be used instead of xrange. if xrange is more
efficient, why range was not reimplemented?

If you actually want the list for some reason:

$ python2.4 -mtimeit 'list(xrange(10 0))'
100000 loops, best of 3: 4.54 usec per loop
$ python2.4 -mtimeit 'range(100)'
100000 loops, best of 3: 2.61 usec per loop

--
\S -- si***@chiark.gr eenend.org.uk -- http://www.chaos.org.uk/~sion/
___ | "Frankly I have no feelings towards penguins one way or the other"
\X/ | -- Arthur C. Clarke
her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump

[Steven D'Aprano]

On Wed, 05 Apr 2006 09:16:37 -0400, AndyL wrote:

Paul Rubin wrote:

Normally you'd use range or xrange. range builds a complete list in
memory so can be expensive if the number is large. xrange just counts
up to that number.

so when range would be used instead of xrange. if xrange is more
efficient, why range was not reimplemented?

For historical reasons.

Don't worry, in Python3000, range() will be an iterator, and xrange() will
disappear. Until then, I use range() for small loops (by small I mean
anything up to a few tens of thousands), and xrange() only when I
absolutely have to optimize my code to save a piddling few tens of
kilobytes of memory.
--
Steven.

[Steven D'Aprano]

On Wed, 05 Apr 2006 16:15:12 +0200, Georg Brandl wrote:

su************* *@gmail.com wrote:

hi John,
Python doesn't provide for loop like C / C++ but using Range() or
Xrange() you can achive all the functionalities of the C for loop.

Not quite.

Care to explain what the differences are, or shall we guess?

--
Steven.

[Steven D'Aprano]

On Wed, 05 Apr 2006 16:21:02 +0200, Georg Brandl wrote:

Because of backwards compatibility. range() returns a list, xrange() an
iterator: list(xrange(... )) will give the same results as range(...).

Georg is pretty much correct in his explanation, but just to dot all the
I's and cross all the T's, we should explain that xrange() doesn't return
an iterator, it returns a special xrange object:

x = xrange(1000)
type(x) <type 'xrange'>

xrange existed as a special bit of magic before Python supported
iterators. While xrange objects behave (sort of) like iterators, they
aren't quite the same. For instance, they don't have a next attribute:
x.next Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'xrange' object has no attribute 'next'

whereas iterators do:
i = iter(range(1000 ))
i.next <method-wrapper object at 0xf7054d2c>

Likewise, you can get random access to the items in an xrange object:
x[500] 500

but not in iterators:
i[500]

Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: unsubscriptable object

--
Steven.

[Georg Brandl]

Steven D'Aprano wrote:

On Wed, 05 Apr 2006 16:21:02 +0200, Georg Brandl wrote:

Because of backwards compatibility. range() returns a list, xrange() an
iterator: list(xrange(... )) will give the same results as range(...).

Georg is pretty much correct in his explanation, but just to dot all the
I's and cross all the T's, we should explain that xrange() doesn't return
an iterator, it returns a special xrange object:

Ah yes, the old iterator <-> iterable problem ;)

Georg

[Georg Brandl]

Steven D'Aprano wrote:

On Wed, 05 Apr 2006 16:15:12 +0200, Georg Brandl wrote:

su************* *@gmail.com wrote:

hi John,
Python doesn't provide for loop like C / C++ but using Range() or
Xrange() you can achive all the functionalities of the C for loop.

Not quite.

Care to explain what the differences are, or shall we guess?

C's for is much more powerful.

for(a; b; c) { d } translates to

a
while b
d
c

which can't be replaced by a simple

for i in range(...)

Georg

[Alex Martelli]

Georg Brandl <g.************ *@gmx.net> wrote:

Steven D'Aprano wrote:

On Wed, 05 Apr 2006 16:15:12 +0200, Georg Brandl wrote:

su************* *@gmail.com wrote:
hi John,
Python doesn't provide for loop like C / C++ but using Range() or
Xrange() you can achive all the functionalities of the C for loop.

Not quite.

Care to explain what the differences are, or shall we guess?

C's for is much more powerful.

for(a; b; c) { d } translates to

a
while b
d
c

which can't be replaced by a simple

for i in range(...)

No, but it can be easily replaced by:

# factoring out the loopstructure.. .
def gotcha():
a
while b:
yield c

# ...from the loopbody
for i in gotcha():
d

or several variations thereof. In C++, I almost half-heartedly replace
most of Python's generators' power with C++'s templated iterators, but
when I program in C I find myself really straightjackete d these days in
NOT being able to pull out the "loopstruct ure" as I can in Python (and,
about halfway, in C++).
Alex