why I don't like range/xrange

[stdazi]

Hello!

Many times I was suggested to use xrange and range instead of the
while constructs, and indeed, they are quite more elegant - but, after
calculating the overhead (and losen flexibility) when working with
range/xrange, and while loops, you get to the conclusion that it isn't
really worth using range/xrange loops.

I'd like to show some examples and I'll be glad if someone can suggest
some other fixes than while a loop :-)

a) range overfllow :
for i in range(0, 1 << len(S)) :
......
OverflowError: range() result has too many items

ok, so we fix this one with xrange !

b) xrange long int overflow :

for i in xrange(0, 1 << len(S)) :
.........
OverflowError: long int too large to convert to int

Next thing I miss is the flexibility as in C for loops :

for (i = 0; some_function() /* or other condition */ ; i++)

or,

for (i = 0 ; i < 10 ; i++)
i = 10;
I don't think range/xrange sucks, but I really think there should be
some other constructs to improve the looping flexibility. Other thing
may be, that I just miss an equally elegant alternative that's why I'd
like to hear some suggestions on how to fix the above issues.. (btw,
I've already browsed the archives related to my issue,but i don't see
any good solution)

Thanks

Jernej.

[Chris Mellon]

On 16 Feb 2007 07:30:15 -0800, stdazi <st****@gmail.c omwrote:

Hello!

Many times I was suggested to use xrange and range instead of the
while constructs, and indeed, they are quite more elegant - but, after
calculating the overhead (and losen flexibility) when working with
range/xrange, and while loops, you get to the conclusion that it isn't
really worth using range/xrange loops.

I'd like to show some examples and I'll be glad if someone can suggest
some other fixes than while a loop :-)

a) range overfllow :
for i in range(0, 1 << len(S)) :
.....
OverflowError: range() result has too many items

ok, so we fix this one with xrange !

b) xrange long int overflow :

for i in xrange(0, 1 << len(S)) :
........
OverflowError: long int too large to convert to int

xrange should be able to handle this. It's probably worth a bug report.

Next thing I miss is the flexibility as in C for loops :

for (i = 0; some_function() /* or other condition */ ; i++)

You'd use a regular for or a while loop here, without the loop index.

or,

for (i = 0 ; i < 10 ; i++)
i = 10;

This doesn't do anything as written. For all reasonable uses of it,
you can do it the same way in Python. Writing C code in Python is a
waste of time and an exercise in frustration.

>
I don't think range/xrange sucks, but I really think there should be
some other constructs to improve the looping flexibility. Other thing
may be, that I just miss an equally elegant alternative that's why I'd
like to hear some suggestions on how to fix the above issues.. (btw,
I've already browsed the archives related to my issue,but i don't see
any good solution)

Enumeration over range() and xrange() is rare in Python. loops like:

data = [...]
for ii in xrange(len(data )):
datum = data[ii]

are a wart. Enumerate over whatever you're going to work with.

In some cases, you want the index and the item. Use enumerate for that:

for index, datum in enumerate(data) :
....

[Paul McGuire]

On Feb 16, 9:30 am, "stdazi" <std...@gmail.c omwrote:

Hello!

Many times I was suggested to use xrange and range instead of the
while constructs, and indeed, they are quite more elegant - but, after
calculating the overhead (and losen flexibility) when working with
range/xrange, and while loops, you get to the conclusion that it isn't
really worth using range/xrange loops.

I'd like to show some examples and I'll be glad if someone can suggest
some other fixes than while a loop :-)

a) range overfllow :

for i in range(0, 1 << len(S)) :
.....
OverflowError: range() result has too many items

ok, so we fix this one with xrange !

b) xrange long int overflow :

for i in xrange(0, 1 << len(S)) :
........
OverflowError: long int too large to convert to int

Next thing I miss is the flexibility as in C for loops :

for (i = 0; some_function() /* or other condition */ ; i++)

or,

for (i = 0 ; i < 10 ; i++)
i = 10;

I don't think range/xrange sucks, but I really think there should be
some other constructs to improve the looping flexibility. Other thing
may be, that I just miss an equally elegant alternative that's why I'd
like to hear some suggestions on how to fix the above issues.. (btw,
I've already browsed the archives related to my issue,but i don't see
any good solution)

Thanks

Jernej.

Very little of my own code uses range or xrange, most of my for loops
iterate over a sequence or generator, as in "for item in blahList: do
something with item". I think range/xrange are common beginner's
constructs, since they reflect a more C-like looping method ("for i in
range(len(blahL ist)): do something with blahList[i]").

I really would *not* encourage use of range/xrange, but feel that
iteration over sequences and generators is the more elegant/Pythonic
way to go - who is suggesting you use range/xrange?

For that matter, just what is it you plan to do 2**len(S) times? If S
is of any significant length, the sun may be a lump of coal before you
are finished, regardless of what loop mechanism you use (although it
would make sense to avoid range's implementation of creating a list of
2**len(S) items - fortunately the implementation already resolves this
problem by raising a "that's too many items" exception, and thankfully
so).

Maybe instead of working around range/xrange, you should think whether
a 2**len(S) approach to your problem is feasible in the first place.

-- Paul

[Larry Bates]

stdazi wrote:

Hello!

Many times I was suggested to use xrange and range instead of the
while constructs, and indeed, they are quite more elegant - but, after
calculating the overhead (and losen flexibility) when working with
range/xrange, and while loops, you get to the conclusion that it isn't
really worth using range/xrange loops.

I'd like to show some examples and I'll be glad if someone can suggest
some other fixes than while a loop :-)

a) range overfllow :
for i in range(0, 1 << len(S)) :
.....
OverflowError: range() result has too many items

ok, so we fix this one with xrange !

b) xrange long int overflow :

for i in xrange(0, 1 << len(S)) :
........
OverflowError: long int too large to convert to int

Next thing I miss is the flexibility as in C for loops :

for (i = 0; some_function() /* or other condition */ ; i++)

or,

for (i = 0 ; i < 10 ; i++)
i = 10;
I don't think range/xrange sucks, but I really think there should be
some other constructs to improve the looping flexibility. Other thing
may be, that I just miss an equally elegant alternative that's why I'd
like to hear some suggestions on how to fix the above issues.. (btw,
I've already browsed the archives related to my issue,but i don't see
any good solution)

Thanks

Jernej.

Your example of for i in xrange(0, 1<<len(s)): must have resulted in
a number greater than 1 billion. Are you really doing this much or
are you just pointing out an edge case here?

You can always use while loop:

i=0
while i < (1<<len(s)):
i+=1
or

i=1
while 1:
if i == 10: break
i+=1

Personally I use a lot of for loops in my code where I iterate
over lists or with a generator as my target that returns instances
of my data until exhausted and find the code easy to read:

for line in myfile:
# do something with the line

No need to even "think" about numbers here unless I want to keep
track of the line numbers in which case I would do:

for linenumber, line in enumerate(myfil e):
# do something with each linenumber, line

I find that I use i, j, k pointers less and less as I write more code in
Python. I guess it is just what you get accustomed to using.

-Larry

[Steven D'Aprano]

On Fri, 16 Feb 2007 07:30:15 -0800, stdazi wrote:

Hello!

Many times I was suggested to use xrange and range instead of the
while constructs, and indeed, they are quite more elegant - but, after
calculating the overhead (and losen flexibility) when working with
range/xrange, and while loops, you get to the conclusion that it isn't
really worth using range/xrange loops.

I prefer to _measure_ the overhead instead of guessing.

import timeit
whileloop = """i = 0
while i < N:
i += 1
pass
"""
forloop = """for i in xrange(N):
pass
"""

Now let's see how fast the loops are.

>>timeit.Timer( whileloop, "N = 10000").repeat( 3, 1000)

[3.5716907978057 861, 3.5263650417327 881, 3.5975079536437 988]

>>timeit.Timer( forloop, "N = 10000").repeat( 3, 1000)

[1.3608510494232 178, 1.3419611454010 01, 1.3180010318756 104]

Looks to me that a for loop using xrange is more than twice as fast as a
while loop. The advantage is about the same for small N:

>>timeit.Timer( whileloop, "N = 100").repeat(3 , 1000)

[0.0522642135620 11719, 0.0493741035461 42578, 0.0419459342956 54297]

>>timeit.Timer( forloop, "N = 100").repeat(3 , 1000)

[0.0122590065002 44141, 0.0135121345520 01953, 0.0151968002319 33594]
What makes you think that a while loop has less overhead?

I'd like to show some examples and I'll be glad if someone can suggest
some other fixes than while a loop :-)

a) range overfllow :
for i in range(0, 1 << len(S)) :
.....
OverflowError: range() result has too many items

ok, so we fix this one with xrange !

By the way, you don't need to write range(0, N). You can just write
range(N).

Yes, you're correct, range(some_enor mous_number) will fail if
some_enormous_n umber is too big.

b) xrange long int overflow :

for i in xrange(0, 1 << len(S)) :
........
OverflowError: long int too large to convert to int

Are you really doing something at least 2147483647 times? I'm guessing
that you would be better off rethinking your algorithm.

Next thing I miss is the flexibility as in C for loops :

for (i = 0; some_function() /* or other condition */ ; i++)

This would be written in Python as:

i = 0
while some_function() :
i += 1

A more flexible way would be to re-write some_function() as an iterator,
then use it directly:

for item in some_function() :
# do something with item

or,

for (i = 0 ; i < 10 ; i++)
i = 10;

This would be written in Python as:

for i in xrange(10):
i = 10

I don't think range/xrange sucks, but I really think there should be
some other constructs to improve the looping flexibility.

Every loop can be turned into a while loop. If you have while, you don't
_need_ anything else.

But for elegance and ease of use, a small number of looping constructs is
good. Python has:

while condition:
block
else:
# runs if while exits *without* hitting break statement
block
for item in any_sequence:
block
else:
# runs if for exits *without* hitting break statement
block
Nice, clean syntax.

any_sequence isn't limited to mere arithmetic sequences -- it can be any
sequence of any objects. But if you need a C-style for loop, using
integers, range/xrange([start, ] stop [, step]) is provided.
--
Steven.

[Bart Ogryczak]

On Feb 16, 4:30 pm, "stdazi" <std...@gmail.c omwrote:

for (i = 0; some_function() /* or other condition */ ; i++)

C's "for(pre,cond,p ost) code" is nothing more, then shorthand form of
"pre; while(cond) {code; post;}"
Which translated to Python would be:

pre
while cond:
code
post

[Marc 'BlackJack' Rintsch]

In <pa************ *************** *@REMOVE.THIS.c ybersource.com. au>, Steven
D'Aprano wrote:

>for (i = 0 ; i < 10 ; i++)
i = 10;

This would be written in Python as:

for i in xrange(10):
i = 10

Nope, in Python it's:

for i in xrange(10):
break

I think his example should demonstrate that assigning to the loop variable
has no effect on the "loop test". Which IMHO is a good thing.

Ciao,
Marc 'BlackJack' Rintsch

[Bjoern Schliessmann]

stdazi wrote:

Many times I was suggested to use xrange and range instead of the
while constructs, and indeed, they are quite more elegant - but,
after calculating the overhead (and losen flexibility) when
working with range/xrange, and while loops, you get to the
conclusion that it isn't really worth using range/xrange loops.

How did you calculate that?

b) xrange long int overflow :

for i in xrange(0, 1 << len(S)) :
........
OverflowError: long int too large to convert to int

Why do you bit shift a len result here? Just to get a huge number?
Try the same with C. You'd get an integer overflow in the first
place.

But this long int =int issue should not exist in a future python
version any more, IIRC int and long int is scheduled to be merged
somehow. (Or isn't it?)

Regards,
Björn

--
BOFH excuse #203:

Write-only-memory subsystem too slow for this machine. Contact your
local dealer.

[Eduardo \EdCrypt\ O. Padoan]

But this long int =int issue should not exist in a future python

version any more, IIRC int and long int is scheduled to be merged
somehow. (Or isn't it?)

It is done.
http://mail.python.org/pipermail/pyt...ry/000251.html
--
EduardoOPadoan (eopadoan->altavix::com )
Bookmarks: http://del.icio.us/edcrypt

[Neil Cerutti]

On 2007-02-16, Bart Ogryczak <B.********@gma il.comwrote:

On Feb 16, 4:30 pm, "stdazi" <std...@gmail.c omwrote:

>for (i = 0; some_function() /* or other condition */ ; i++)

C's "for(pre,cond,p ost) code" is nothing more, then shorthand form of
"pre; while(cond) {code; post;}"
Which translated to Python would be:

pre
while cond:
code
post

No, when you consider the continue statement, which which Python
also supports.

for (pre; cond; post) {
continue;
}

That's not an infinite loop in C, but the Python while-loop
version would be.

--
Neil Cerutti