efficiency of range() and xrange() in for loops

[Steve R. Hastings]

When you compile the expression

for i in range(1000):
pass

does Python make an iterator for range(), and then generate the values
on the fly? Or does Python actually allocate the list [0, 1, 2, ..., 999]
and then step through it?

I was under the impression that recent releases of Python optimize this
case, but upon reflection, I have no idea where I got that impression.

If Python actually allocates the list, then clearly we should all use
"for i in xrange". But "for i in range" looks cleaner, and is potentially
more lightweight than xrange. Of course, if you want to write code that
runs well on older versions of Python, you need to use xrange().
If Python doesn't currently optimize this case, is there any chance this
optimization could be added?

P.S. It looks like all the cool people look at the generated bytecodes to
answer questions like this one. I want to be cool too. Where can I find
information about how to get a bytecodes listing for my compiled Python?

--
Steve R. Hastings "Vita est"
st***@hastings.org http://www.blarg.net/~steveha

[John Salerno]

Steve R. Hastings wrote:

If Python doesn't currently optimize this case, is there any chance this
optimization could be added?

I was just asking about this, and apparently it will be changed in 3.0:

Built-in Namespace
* Make built-ins return an iterator where appropriate (e.g.
range(),
zip(), etc.)

To be removed:
* xrange(): use range() instead [1]

[Georg Brandl]

Steve R. Hastings wrote:

When you compile the expression

for i in range(1000):
pass

does Python make an iterator for range(), and then generate the values
on the fly? Or does Python actually allocate the list [0, 1, 2, ..., 999]
and then step through it?
It does.
I was under the impression that recent releases of Python optimize this
case, but upon reflection, I have no idea where I got that impression.

If Python actually allocates the list, then clearly we should all use
"for i in xrange". But "for i in range" looks cleaner, and is potentially
more lightweight than xrange. Of course, if you want to write code that
runs well on older versions of Python, you need to use xrange().
If Python doesn't currently optimize this case, is there any chance this
optimization could be added?
As the alternative is so easy (adding a single character), I think this will
not make it into the 2.x line. A patch for the old compiler existed on SF,
but it will not work with the new AST compiler.
P.S. It looks like all the cool people look at the generated bytecodes to
answer questions like this one. I want to be cool too. Where can I find
information about how to get a bytecodes listing for my compiled Python?

The "dis" module.

Georg

[Fredrik Lundh]

Steve R. Hastings wrote:

If Python actually allocates the list, then clearly we should all use
"for i in xrange".
clearly we should all? speak for yourself. the difference isn't very
large, xrange is actually slower in some python versions, and you'll
need the integer objects sooner or later anyway...
If Python doesn't currently optimize this case, is there any chance this
optimization could be added?

in Python 2.X, range is defined to return a list. if you start returning
something else, you'll break stuff.

</F>

[Todd]

Steve R. Hastings wrote:

When you compile the expression

for i in range(1000):
pass

does Python make an iterator for range(), and then generate the values
on the fly? Or does Python actually allocate the list [0, 1, 2, ..., 999]
and then step through it?

I ran an experiment on this a while back. I thought it generated a
list. But the results turned out to be surprising. Take this with a
grain of salt. I'm not completely sure I did what I meant to do. But
it looks like however you do it, it works out about the same.

http://www.signalsguru.net/articles/...s/pyloops.html

[mensanator]

Todd wrote:

Steve R. Hastings wrote:

When you compile the expression

for i in range(1000):
pass

does Python make an iterator for range(), and then generate the values
on the fly? Or does Python actually allocate the list [0, 1, 2, ..., 999]
and then step through it?

I ran an experiment on this a while back. I thought it generated a
list. But the results turned out to be surprising. Take this with a
grain of salt. I'm not completely sure I did what I meant to do. But
it looks like however you do it, it works out about the same.

http://www.signalsguru.net/articles/...s/pyloops.html

Didn't it occur to you to also check memory usage?

I learned about xrange the hard way, when my range ate up all
available memory.

[Steve R. Hastings]

On Wed, 05 Apr 2006 22:24:24 +0200, Fredrik Lundh wrote:

If Python actually allocates the list, then clearly we should all use
"for i in xrange".
clearly we should all? speak for yourself.

I apologize for my pithy turn of phrase. Clearly what I should have
written was:

"If Python actually allocates the list, then clearly 'for i in xrange'
would be desirable in cases where memory allocation might possibly be a
problem; of course if you have lots of memory, or you are only looping a
small number of times, go ahead and use range() because why not."

I deeply apologize for inadvertently trying to tell you how to write a for
loop. I hope the offense isn't unforgivable.
I apologise for the fault in the newsgroup posting. Those responsible
have been sacked.

the difference isn't very
large, xrange is actually slower in some python versions, and you'll
need the integer objects sooner or later anyway...

Actually, for many uses of "for i in (range|xrange)", you only need the
value of i, and you aren't doing anything with the integer object. You
might not even be looking at the value of i:
start_time = time.time()
for i in xrange(10**6):
run_some_function()
stop_time = time.time()
secs = (stop_time - start_time) / 10**6
print "run_some_function() took on average, %f seconds." % secs
In the above, clearly we should all use xrange()... oops, I meant, if you
want to, you could use xrange() instead of allocating a list of a million
integers and then doing nothing with the list itself and then deallocating
the list of a million integers.

I apologise again for the fault in the newsgroup posting. Those
responsible for sacking the people who have just been sacked
have been sacked.

If Python doesn't currently optimize this case, is there any chance
this optimization could be added?

in Python 2.X, range is defined to return a list. if you start
returning something else, you'll break stuff.

Perhaps I'm mistaken here, but I don't see how this optimization could
possibly break anything. range() makes a list, and for consumes it, and
the list isn't seen anywhere else. If the Python compiler took this:

for i in range(10**6):
pass
and produced code equivalent to this:

for i in iter(range(10**6))
pass
How would this break stuff?
--
Steve R. Hastings "Vita est"
st***@hastings.org http://www.blarg.net/~steveha

[Giovanni Bajo]

Steve R. Hastings wrote:

in Python 2.X, range is defined to return a list. if you start
returning something else, you'll break stuff.

Perhaps I'm mistaken here, but I don't see how this optimization could
possibly break anything.

Because you assume that the only use-case of range() is within a for-loop.
range() is a builtin function that can be used in any Python expression. For
instance:

RED, GREEN, BLUE, WHITE, BLACK = range(5)
--
Giovanni Bajo

[Alan Morgan]

In article <te*********************@twister1.libero.it>,
Giovanni Bajo <no***@sorry.com> wrote:

Steve R. Hastings wrote:

in Python 2.X, range is defined to return a list. if you start
returning something else, you'll break stuff.

Perhaps I'm mistaken here, but I don't see how this optimization could
possibly break anything.

Because you assume that the only use-case of range() is within a for-loop.
range() is a builtin function that can be used in any Python expression. For
instance:

RED, GREEN, BLUE, WHITE, BLACK = range(5)

Hmmm, this worked fine when I used xrange as well. Am I missing something?
Obviously there *are* differences, viz:

a = range(5)
b = range(5)
a==b # True!
c = xrange(5)
d = xrange(5)
c==d # False!

and various other more arcane things, but do these actually happen
in real life?

Alan
--
Defendit numerus

[Steve R. Hastings]

On Wed, 05 Apr 2006 22:09:29 +0000, Giovanni Bajo wrote:

[...] you assume that the only use-case of range() is within a for-loop.

No, I do not assume any such thing. I have not suggested that range()
should be changed to always return an iterator. I wondered if the Python
compiler could, as a special case, turn:

for i in range(n)

into compiled code equivalent to

for i in itr

where "itr" is a lightweight iterator that returns the same values as
iter(range(n)).

iter(range(n)) will still allocate a list of n integers, and then produce
an iterator over that list, so I did not mean to suggest that for would
simply call iter() when you use "for i in range".

I apologize if my writing was unclear.
--
Steve R. Hastings "Vita est"
st***@hastings.org http://www.blarg.net/~steveha

[Erik Max Francis]

Alan Morgan wrote:

In article <te*********************@twister1.libero.it>,
Giovanni Bajo <no***@sorry.com> wrote:

Because you assume that the only use-case of range() is within a for-loop.
range() is a builtin function that can be used in any Python expression. For
instance:

RED, GREEN, BLUE, WHITE, BLACK = range(5)

Hmmm, this worked fine when I used xrange as well. Am I missing something?

Not in your use case. Tuple unpacking will iterate, and so it doesn't
matter whether it's an actual list or an iterator:

a, b, c = xrange(3)
a 0 b 1 c 2

There are certainly contexts where a sequence and its iterator are not
interchangeable. You missed an obvious one:
range(3) == xrange(3)

False
--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
I will always remember / This moment
-- Sade

[Georg Brandl]

Alan Morgan wrote:

In article <te*********************@twister1.libero.it>,
Giovanni Bajo <no***@sorry.com> wrote:

Steve R. Hastings wrote:

in Python 2.X, range is defined to return a list. if you start
returning something else, you'll break stuff.

Perhaps I'm mistaken here, but I don't see how this optimization could
possibly break anything.

Because you assume that the only use-case of range() is within a for-loop.
range() is a builtin function that can be used in any Python expression. For
instance:

RED, GREEN, BLUE, WHITE, BLACK = range(5)

Hmmm, this worked fine when I used xrange as well. Am I missing something?
Obviously there *are* differences, viz:

Just _look_ at the objects:

range(5) [0, 1, 2, 3, 4] xrange(5) xrange(5)

range is giving you a real list, while xrange is giving you an xrange object.
Have you tried to slice an xrange object? Or using .append on it?

Georg

[Steven D'Aprano]

On Wed, 05 Apr 2006 15:02:33 -0700, Steve R. Hastings wrote:

On Wed, 05 Apr 2006 22:24:24 +0200, Fredrik Lundh wrote:

If Python actually allocates the list, then clearly we should all use
"for i in xrange".
clearly we should all? speak for yourself.

I apologize for my pithy turn of phrase. Clearly what I should have
written was:

"If Python actually allocates the list, then clearly 'for i in xrange'
would be desirable in cases where memory allocation might possibly be a
problem; of course if you have lots of memory, or you are only looping a
small number of times, go ahead and use range() because why not."

How about this?

....because range() is simpler and actually an English word, it reads
better and won't confuse non-native English speakers who try looking
xrange up in a dictionary, it is aesthetically nicer, and at least some of
the time range() will actually be faster and maybe even consume less
memory (although if your list is that small, we're only talking about
trivial amounts of memory anyway).

But yes, for many purposes, the difference between xrange and range is
quite minimal. But please try to recall that many != all, and any
optimization put to Python 2.x must work in all cases, not just many.

I deeply apologize for inadvertently trying to tell you how to write a
for loop. I hope the offense isn't unforgivable.
Sarcasm aside, you will find many many people are deeply allergic to the
merest hint of premature optimization, especially when that premature
optimization is posed in such a way to hint that the person proposing it
hasn't really considered all the factors.

[snip]
Actually, for many uses of "for i in (range|xrange)", you only need the
value of i, and you aren't doing anything with the integer object. You
might not even be looking at the value of i:
start_time = time.time()
for i in xrange(10**6):
run_some_function()
stop_time = time.time()
secs = (stop_time - start_time) / 10**6 print "run_some_function() took
on average, %f seconds." % secs
In the above, clearly we should all use xrange()... oops, I meant, if
you want to, you could use xrange() instead of allocating a list of a
million integers and then doing nothing with the list itself and then
deallocating the list of a million integers.

Yes, the above example is a good use case for xrange. Did you think that
anyone denied that there were good cases for it?

Here is a bad usage case for xrange:

L = []
for i in xrange(10**6):
L.append(2*i+1)

This should likely be written as:

L = range(1, 2*10**6+1, 2)
I'll tell you what: I'll agree that the existence of bad usages for xrange
does NOT mean xrange should never be used, if you agree that the existence
of bad usages for range does NOT mean range should never be used.

In any case, if you look at the source code for the timeit module, you
will see that for cases where you genuinely don't need the values of i,
the optimal strategy is to not bother creating one million integers at all:

L = [None]*10**6 # don't time the creation of the list
start_time = time.time()
for _ in L:
run_some_function()
stop_time = time.time()

This strategy (I think) trades off a little extra memory for a little
extra speed; in other applications, speed may be less important and memory
more so, so xrange or some other iterator would be preferred.

If Python doesn't currently optimize this case, is there any chance
this optimization could be added?

in Python 2.X, range is defined to return a list. if you start
returning something else, you'll break stuff.

Perhaps I'm mistaken here, but I don't see how this optimization could
possibly break anything. range() makes a list, and for consumes it, and
the list isn't seen anywhere else.

But of course you are forgetting that range is just a function and can be
used anywhere, not just in for loops. Here is a toy example:

L = range(1000)
random.shuffle(L)
while L:
L.pop()

If range was optimized to return a xrange or iterator object, this code
would stop working.
--
Steven.

[Alan Morgan]

In article <e1**********@news.albasani.net>,
Georg Brandl <g.*************@gmx.net> wrote:

Alan Morgan wrote:

In article <te*********************@twister1.libero.it>,
Giovanni Bajo <no***@sorry.com> wrote:

Steve R. Hastings wrote:

> in Python 2.X, range is defined to return a list. if you start
> returning something else, you'll break stuff.

Perhaps I'm mistaken here, but I don't see how this optimization could
possibly break anything.

Because you assume that the only use-case of range() is within a for-loop.
range() is a builtin function that can be used in any Python expression. For
instance:

RED, GREEN, BLUE, WHITE, BLACK = range(5)
Hmmm, this worked fine when I used xrange as well. Am I missing something?
Obviously there *are* differences, viz:

Just _look_ at the objects:

range(5)[0, 1, 2, 3, 4] xrange(5)xrange(5)

Yes, I was well aware of this. I noted a difference in my original
post. I was just pointing out that the example given didn't, in
fact, behave differently for range() and xrange() even though range()
and xrange() *are* different.
range is giving you a real list, while xrange is giving you an xrange object.
Have you tried to slice an xrange object? Or using .append on it?

No, I hadn't. I presume these could all be defined.

Alan
--
Defendit numerus

[Steve R. Hastings]

On Thu, 06 Apr 2006 09:08:45 +1000, Steven D'Aprano wrote:

Yes, the above example is a good use case for xrange. Did you think that
anyone denied that there were good cases for it?

I am now officially sorry for starting this thread.

Please let me just summarize what I wanted to say:

* Question: Does Python do anything special with "for i in range"?
(And I got an answer: the answer is "no".)

* If Python does not do anything special with "for i in range", then you
are building a list and tearing it down again without using the list, and
for memory efficiency you might want to use xrange. (I used a more pithy
phrase, which I now regret.)

* It would be nice if the Python compiler checked for the special case of
"for i in range" and compiled it to "for i in itr" where itr is a
lightweight iterator that generates the same values as the range statement.
Potentially, I speculated, this special iterator might be more lightweight
than an xrange() object.
That is all. I did not mean to tell anyone how to write a for loop. I
did not mean to suggest that range() should be changed to always return an
iterator. I did not mean to insult anyone. I don't even know what I ever
wrote that sounded like "range should never be used".

And I hope everyone recognized that the bit about "those responsible...
have been sacked" was a reference to the opening credits of the movie
_Monty Python and the Holy Grail_. I was trying to be funny, not
sarcastic, bitter, etc.

Thank you for your patience and I am done with this thread, unless I have
written something unclear in *this* post and I have to post another post
to clarify it as well. :-(
--
Steve R. Hastings "Vita est"
st***@hastings.org http://www.blarg.net/~steveha

[Alan Morgan]

In article <F4******************************@speakeasy.net> ,
Erik Max Francis <ma*@alcyone.com> wrote:

Alan Morgan wrote:

In article <te*********************@twister1.libero.it>,
Giovanni Bajo <no***@sorry.com> wrote:

Because you assume that the only use-case of range() is within a for-loop.
range() is a builtin function that can be used in any Python expression. For
instance:

RED, GREEN, BLUE, WHITE, BLACK = range(5)

Hmmm, this worked fine when I used xrange as well. Am I missing something?

Not in your use case. Tuple unpacking will iterate, and so it doesn't
matter whether it's an actual list or an iterator:

a, b, c = xrange(3)
a0 b1 c2

There are certainly contexts where a sequence and its iterator are not
interchangeable. You missed an obvious one:
range(3) == xrange(3)

False

I thought that one was sufficiently obvious as not to need mentioning.
There was never any argument that range() and xrange() returned different
things; the question (as I understood it) was if you could generally
use the things they return in the same way and not *care* about the
difference (the answer being "Yes, except when you can't").

Alan
--
Defendit numerus

[Azolex]

Steve R. Hastings wrote:

On Thu, 06 Apr 2006 09:08:45 +1000, Steven D'Aprano wrote:

Yes, the above example is a good use case for xrange. Did you think that
anyone denied that there were good cases for it?

I am now officially sorry for starting this thread.

Don't. It's quite funny, thanks.

[Steve R. Hastings]

On Thu, 06 Apr 2006 02:33:16 +0200, Azolex wrote:

Don't. It's quite funny, thanks.

I guess I should laugh. :-/
When you read my original articles, did *you* think I was proposing that
range() be changed to always return an iterator? I thought what I wrote
was pretty clear...
--
Steve R. Hastings "Vita est"
st***@hastings.org http://www.blarg.net/~steveha

[Felipe Almeida Lessa]

Em Qua, 2006-04-05 Ã*s 22:24 +0200, Fredrik Lundh escreveu:

the difference isn't very
large, xrange is actually slower in some python versions, and you'll
need the integer objects sooner or later anyway...

Some code is worth a thousand words:
$ python2.3
Python 2.3.5 (#2, Mar 6 2006, 10:12:24)
[GCC 4.0.3 20060304 (prerelease) (Debian 4.0.2-10)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

from timeit import Timer
min(Timer('for i in range(1): pass').repeat(3, 100000))/100 0.00099303960800170891 min(Timer('for i in xrange(1): pass').repeat(3, 100000))/100 0.00077417135238647464 min(Timer('for i in range(10): pass').repeat(3, 1000)) 0.0016450881958007812 min(Timer('for i in xrange(10): pass').repeat(3, 1000)) 0.0013418197631835938 min(Timer('for i in range(100): pass').repeat(3, 1000)) 0.010170936584472656 min(Timer('for i in xrange(100): pass').repeat(3, 1000)) 0.0073339939117431641 min(Timer('for i in range(1000): pass').repeat(3, 1000)) 0.083337783813476562 min(Timer('for i in xrange(1000): pass').repeat(3, 1000)) 0.070460796356201172 min(Timer('for i in range(10000): pass').repeat(3, 1000)) 0.88027620315551758 min(Timer('for i in xrange(10000): pass').repeat(3, 1000)) 0.71050214767456055 min(Timer('for i in range(100000): pass').repeat(3, 1000)) 13.938336849212646 min(Timer('for i in xrange(100000): pass').repeat(3, 1000)) 7.0900959968566895 min(Timer('for i in range(1000000): pass').repeat(3, 100))*10 141.37096881866455 min(Timer('for i in xrange(1000000): pass').repeat(3, 100))*10 70.822579860687256
$ python2.4
Python 2.4.3 (#2, Mar 30 2006, 21:52:26)
[GCC 4.0.3 (Debian 4.0.3-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information. from timeit import Timer
min(Timer('for i in range(1): pass').repeat(3, 100000))/100 0.00084012031555175776 min(Timer('for i in xrange(1): pass').repeat(3, 100000))/100 0.00069013833999633791 min(Timer('for i in range(10): pass').repeat(3, 1000)) 0.0014748573303222656 min(Timer('for i in range(10): pass').repeat(3, 1000)) 0.0014920234680175781 min(Timer('for i in xrange(10): pass').repeat(3, 1000)) 0.0012860298156738281 min(Timer('for i in range(100): pass').repeat(3, 1000)) 0.0079219341278076172 min(Timer('for i in xrange(100): pass').repeat(3, 1000)) 0.0069670677185058594 min(Timer('for i in range(1000): pass').repeat(3, 1000)) 0.079225063323974609 min(Timer('for i in xrange(1000): pass').repeat(3, 1000)) 0.067707061767578125 min(Timer('for i in range(10000): pass').repeat(3, 1000)) 0.84912896156311035 min(Timer('for i in xrange(10000): pass').repeat(3, 1000)) 0.6791541576385498 min(Timer('for i in range(100000): pass').repeat(3, 1000)) 13.973598003387451 min(Timer('for i in xrange(100000): pass').repeat(3, 1000)) 6.8047640323638916 min(Timer('for i in range(1000000): pass').repeat(3, 100))*10 138.38916063308716 min(Timer('for i in xrange(1000000): pass').repeat(3, 100))*10

68.15216064453125
In sum, difference is as big as the function argument, but xrange was
*never* slower on Python 2.3 or Python 2.4 on x86 with Linux. I'd say
three things:

1) Probably xrange is faster than range on other architectures and
operational systems as well.
2) I can't say anything for Python < 2.3.
3) Run your own tests. If you see the same I saw, use xrange everywhere
you can.

The 3rd point is specially true if your code can be break in the middle,
for example (a really dumb example, I know):

for i in xrange(0, 10000000, 42):
if i**2 >= x:
break

This way you won't create all numbers, in this case just those less than
sqrt(x) plus one.

Cheers,

--
Felipe.

[Alex Martelli]

Steve R. Hastings <st***@hastings.org> wrote:
...

Actually, for many uses of "for i in (range|xrange)", you only need the
value of i, and you aren't doing anything with the integer object. You

Then, the speediest approach may be completely different:

import itertools

for i in itertools.repeat(None, N):
...
Remember, when you're thinking "blazing speed", think itertools.
Alex

[Felipe Almeida Lessa]

Em Qua, 2006-04-05 Ã*s 19:15 -0700, Alex Martelli escreveu:

Steve R. Hastings <st***@hastings.org> wrote:
...

Actually, for many uses of "for i in (range|xrange)", you only need the
value of i, and you aren't doing anything with the integer object. You

Then, the speediest approach may be completely different:

import itertools

for i in itertools.repeat(None, N):
...
Remember, when you're thinking "blazing speed", think itertools.

Hmmm...

min(Timer('for i in xrange(100000): pass').repeat(3, 1000)) 6.7740321159362793 min(Timer('for i in itertools.repeat(None, 100000): pass', setup='import itertools').repeat(3, 1000))
5.6378099918365479
min(Timer('for i in xrange(100): pass').repeat(3, 1000)) 0.0071630477905273438 min(Timer('for i in itertools.repeat(None, 100): pass',

setup='import itertools').repeat(3, 1000))
0.0065851211547851562

It *is* faster, but by a small margin. Considering that it is IMHO less
readable, I'd use it just for *very* big loops, or in functions in
hotspots.

Anyway, it's always good to remember about itertools, it's a great
module that some people don't even know.

Cheers,

--
Felipe.

[Azolex]

Steve R. Hastings wrote:

On Thu, 06 Apr 2006 02:33:16 +0200, Azolex wrote:

Don't. It's quite funny, thanks.

I guess I should laugh. :-/
When you read my original articles, did *you* think I was proposing that
range() be changed to always return an iterator? I thought what I wrote
was pretty clear...

I just re-read your original post, and in fact this appears to have been
your drift with "is there any chance this optimization could be added
?". Anyway that wasn't really controversial (since indeed, as was noted
by John Salerno who first replied to you, it is slated for py3k - and
with good reason).

[Carl Banks]

Steve R. Hastings wrote:

I wondered if the Python
compiler could, as a special case, turn:

for i in range(n)

into compiled code equivalent to

for i in itr

where "itr" is a lightweight iterator that returns the same values as
iter(range(n)).

Nope, out of the question for Python 2.x. Note that the the builtin
range could be rebound, or a global range could appear in the module,
at run time. There might even be a good reason to do so.... Point is,
optimizing the call to range can break compatibility even in the for
loop.
Carl Banks

[Fredrik Lundh]

Steve R. Hastings wrote:

the difference isn't very
large, xrange is actually slower in some python versions, and you'll
need the integer objects sooner or later anyway...
Actually, for many uses of "for i in (range|xrange)", you only need the
value of i, and you aren't doing anything with the integer object.

so you're saying that the value of i isn't an integer object?

if you don't need the index value at all, itertools.repeat is sometimes
faster than xrange.
for i in range(10**6):
pass

and produced code equivalent to this:

for i in iter(range(10**6))
pass

How would this break stuff?

how would that speed anything up?

</F>

[Georg Brandl]

Alan Morgan wrote:

range is giving you a real list, while xrange is giving you an xrange object.
Have you tried to slice an xrange object? Or using .append on it?

No, I hadn't. I presume these could all be defined.

How would xrange(100).remove(1) work?

Georg

[Paddy]

I wondered at the tone of some of the replies, re-read the repliess and
your original message. On first readings ithought that your original
message was OK and that the replies were a bit 'strong' . On second
reading I thought that the original could be interpreted a little less
nicely, but I had to go looking.

Please don't be put off using this news group. I guesss we all have to
be extra careful about tone when no one can see your facial expressions
:-)

- Pad.

P.S. I was having problems with the Google server - sorry if this is
replicated.

[Steve R. Hastings]

On Wed, 05 Apr 2006 22:08:29 -0700, Carl Banks wrote:

for i in range(n)

into compiled code equivalent to

for i in itr

where "itr" is a lightweight iterator that returns the same values as
iter(range(n)).

Nope, out of the question for Python 2.x. Note that the the builtin
range could be rebound, or a global range could appear in the module,
at run time.

Ah! Of course.

Thank you very much for explaining this.
--
Steve R. Hastings "Vita est"
st***@hastings.org http://www.blarg.net/~steveha

[Fredrik Lundh]

Carl Banks wrote:

I wondered if the Python compiler could, as a special case, turn:

for i in range(n)

into compiled code equivalent to

for i in itr

where "itr" is a lightweight iterator that returns the same values as
iter(range(n)).

Nope, out of the question for Python 2.x. Note that the the builtin
range could be rebound, or a global range could appear in the module,
at run time. There might even be a good reason to do so.... Point is,
optimizing the call to range can break compatibility even in the for
loop.

that could of course be addressed by turning for-in into a special method
on the range object... map

for variable in expression:
block

to

_value = expression
try:
_for = _value.__for__
except AttributeError:
_for = iter(_value).__for__
_for(variable, block)

and tweak as necessary.

</F>

[Alan Morgan]

In article <e1**********@news.albasani.net>,
Georg Brandl <g.*************@gmx.net> wrote:

Alan Morgan wrote:

range is giving you a real list, while xrange is giving you an xrange object.
Have you tried to slice an xrange object? Or using .append on it?

No, I hadn't. I presume these could all be defined.

How would xrange(100).remove(1) work?

One way is by first converting the xrange to a list. If we think of
the xrange as an efficient and space lean way to store certain types
of lists then it isn't unreasonable to return a regular list when
the conditions no longer hold.

Or extend the xrange object to allow multiple ranges and return the
union of xrange(0,1) and xrange(2,100). No reason why you can't
define the whole range of list operations over xrange and still leave
it as a nice, lazy list (other languages do). It's possible that no
one needs it enough to make it worthwhile (although having full fledged
lazy structures can really helpful when you need them) but it could
be done.

Alan
--
Defendit numerus

[Fredrik Lundh]

Alan Morgan wrote:

How would xrange(100).remove(1) work?

One way is by first converting the xrange to a list. If we think of
the xrange as an efficient and space lean way to store certain types
of lists then it isn't unreasonable to return a regular list when
the conditions no longer hold.

do you understand Python's object model to be able to suggest how such
a conversion could be done, or is that someone else's problem ?

</F>