473,574 Members | 2,459 Online

# genexp performance problem?

Hello,

I found this strange:

python -mtimeit "sum(int(L) for L in xrange(3000))"
100 loops, best of 3: 5.04 msec per loop

python -mtimeit "import itertools; sum(itertools.i map(int, xrange(3000)))"
100 loops, best of 3: 3.6 msec per loop

I thought the two constructs could achieve the same speed.
--
Giovanni Bajo
May 31 '06 #1
4 1198
Giovanni Bajo wrote:
I found this strange:

python -mtimeit "sum(int(L) for L in xrange(3000))"
100 loops, best of 3: 5.04 msec per loop

python -mtimeit "import itertools; sum(itertools.i map(int, xrange(3000)))"
100 loops, best of 3: 3.6 msec per loop

I thought the two constructs could achieve the same speed.

hint: how many times to the interpreter have to look up the names "int"
and "L" in the two examples ?

</F>

May 31 '06 #2
Giovanni Bajo wrote:
I found this strange:

python -mtimeit "sum(int(L) for L in xrange(3000))"
100 loops, best of 3: 5.04 msec per loop

python -mtimeit "import itertools; sum(itertools.i map(int, xrange(3000)))" 100 loops, best of 3: 3.6 msec per loop

I thought the two constructs could achieve the same speed.

hint: how many times do the interpreter have to look up the names "int"
and "L" in the two examples ?

</F>

May 31 '06 #3
Giovanni Bajo wrote:
I found this strange:

python -mtimeit "sum(int(L) for L in xrange(3000))"
100 loops, best of 3: 5.04 msec per loop

python -mtimeit "import itertools; sum(itertools.i map(int, xrange(3000)))"
100 loops, best of 3: 3.6 msec per loop

I thought the two constructs could achieve the same speed.

I think early binding would have been preferable, but as Fredrik Lundh said,
int is looked up for every iteration which accounts for the slowdown.

For reference:
\$ python -m timeit "sum(int(i) for i in xrange(3000))"
1000 loops, best of 3: 1.92 msec per loop
\$ python -m timeit -s "from itertools import imap" "sum(imap(i nt,
xrange(3000)))"
1000 loops, best of 3: 1.17 msec per loop

You can shave off a few percent by turning int into a local variable:
\$ python -m timeit -s "int_ = int" "sum(int_(i ) for i in xrange(3000))"
1000 loops, best of 3: 1.74 msec per loop

On the other hand the function call overhead that imap() sometimes enforces
is larger than the cost of a symbol lookup:
\$ python -m timeit -s"def square(i): return i*i" -s"from itertools import
imap" "sum(imap(squar e, xrange(3000)))"
100 loops, best of 3: 2.25 msec per loop
\$ python -m timeit "sum(i*i for i in xrange(3000))"
1000 loops, best of 3: 1.29 msec per loop

Peter
May 31 '06 #4
Fredrik Lundh wrote:
I found this strange:

python -mtimeit "sum(int(L) for L in xrange(3000))"
100 loops, best of 3: 5.04 msec per loop

python -mtimeit "import itertools; sum(itertools.i map(int,
xrange(3000)))" 100 loops, best of 3: 3.6 msec per loop

I thought the two constructs could achieve the same speed.

hint: how many times to the interpreter have to look up the names
"int"
and "L" in the two examples ?

Ah right, thanks!
--
Giovanni Bajo
May 31 '06 #5

This thread has been closed and replies have been disabled. Please start a new discussion.