I'm missing something here with range vs. xrange

[Joe Goldthwaite]

I've been playing with Python a bit. Doing little performance benchmarks and
working with Psyco. It's been fun and I've been learning a lot. For
example, in a previous post, I was looking for a way to dynamically add new
runtime function to a class. Martin told me to use a class instance
variable instead. It turns out that's faster than hard coding a list of
functions. Thanks Martin.

I read that the range function builds a list and that xrange returns an
iterator and is therefore more efficient. In my testing, they both come out
to almost exactly the same performance wise. Did something get changed in
Python 2.4 to make them identical? I searched the web but couldn't find
anything that would account for the similarities.

[Bjoern Schliessmann]

Joe Goldthwaite wrote:

I read that the range function builds a list and that xrange
returns an iterator and is therefore more efficient.

This is generally not true.

In my testing, they both come out to almost exactly the same
performance wise. Did something get changed in Python 2.4 to make
them identical?

No. Try again with a list of length 10000000.

Regards,
Björn

--
BOFH excuse #359:

YOU HAVE AN I/O ERROR -Incompetent Operator error

[Chris Mellon]

On Dec 7, 2007 3:08 PM, Joe Goldthwaite <jo*@goldthwaites.comwrote:

Here's the simple benchmark;

start = time.time()
for x in xrange(3):
for y in xrange(10000000):
pass
print 'xRange %s' % (time.time() - start)

start = time.time()
for x in range(3):
for y in range(10000000):
pass
print 'Range %s' % (time.time() - start)

Here's what I get;

xRange 92.5529999733
Range 95.2669999599

Not a lot of difference. Range is slower but not by much. I know that range
builds
a list then iterates through it. I thought that xrange just kept a counter
that was
incremented and returned with each call. No list was ever created. If that's
true
(and I guess it's not), xrange would be much faster than range. It seems
almost
identical. Given the amount of performance difference, I don't see why
xrange even
exists.

You can't imagine why someone might prefer an iterative solution over
a greedy one? Depending on the conditions, the cost of creating the
list can be a greater or a lesser part of the total time spent. Actual
iteration is essentially the same cost for both. Try looking at memory
usage while you're running these tests.

Here's my test results:
C:\>python -m timeit "for x in range(10000000):pass"
10 loops, best of 3: 593 msec per loop

Memory usage (extremely rough, only for comparison purposes: 163 MB

C:\>python -m timeit "for x in xrange(10000000):pass"
10 loops, best of 3: 320 msec per loop

Memory usage: just under 4MB

You mentioned psyco in your original post, which has specific
optimizations for range - I believe it allocates the entire list as an
empty memory block, and then creates the integer objects yielded from
the range lazily.

C:\>python -m timeit "range(10000000)"
10 loops, best of 3: 299 msec per loop

C:\>python -m timeit -s "import psyco;psyco.full()" "range(10000000)"
10 loops, best of 3: 0.0376 usec per loop