473,574 Members | 2,459 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

genexp performance problem?

Hello,

I found this strange:

python -mtimeit "sum(int(L) for L in xrange(3000))"
100 loops, best of 3: 5.04 msec per loop

python -mtimeit "import itertools; sum(itertools.i map(int, xrange(3000)))"
100 loops, best of 3: 3.6 msec per loop

I thought the two constructs could achieve the same speed.
--
Giovanni Bajo
May 31 '06 #1
4 1198
Giovanni Bajo wrote:
I found this strange:

python -mtimeit "sum(int(L) for L in xrange(3000))"
100 loops, best of 3: 5.04 msec per loop

python -mtimeit "import itertools; sum(itertools.i map(int, xrange(3000)))"
100 loops, best of 3: 3.6 msec per loop

I thought the two constructs could achieve the same speed.


hint: how many times to the interpreter have to look up the names "int"
and "L" in the two examples ?

</F>

May 31 '06 #2
Giovanni Bajo wrote:
I found this strange:

python -mtimeit "sum(int(L) for L in xrange(3000))"
100 loops, best of 3: 5.04 msec per loop

python -mtimeit "import itertools; sum(itertools.i map(int, xrange(3000)))" 100 loops, best of 3: 3.6 msec per loop

I thought the two constructs could achieve the same speed.


hint: how many times do the interpreter have to look up the names "int"
and "L" in the two examples ?

</F>

May 31 '06 #3
Giovanni Bajo wrote:
I found this strange:

python -mtimeit "sum(int(L) for L in xrange(3000))"
100 loops, best of 3: 5.04 msec per loop

python -mtimeit "import itertools; sum(itertools.i map(int, xrange(3000)))"
100 loops, best of 3: 3.6 msec per loop

I thought the two constructs could achieve the same speed.


I think early binding would have been preferable, but as Fredrik Lundh said,
int is looked up for every iteration which accounts for the slowdown.

For reference:
$ python -m timeit "sum(int(i) for i in xrange(3000))"
1000 loops, best of 3: 1.92 msec per loop
$ python -m timeit -s "from itertools import imap" "sum(imap(i nt,
xrange(3000)))"
1000 loops, best of 3: 1.17 msec per loop

You can shave off a few percent by turning int into a local variable:
$ python -m timeit -s "int_ = int" "sum(int_(i ) for i in xrange(3000))"
1000 loops, best of 3: 1.74 msec per loop

On the other hand the function call overhead that imap() sometimes enforces
is larger than the cost of a symbol lookup:
$ python -m timeit -s"def square(i): return i*i" -s"from itertools import
imap" "sum(imap(squar e, xrange(3000)))"
100 loops, best of 3: 2.25 msec per loop
$ python -m timeit "sum(i*i for i in xrange(3000))"
1000 loops, best of 3: 1.29 msec per loop

Peter
May 31 '06 #4
Fredrik Lundh wrote:
I found this strange:

python -mtimeit "sum(int(L) for L in xrange(3000))"
100 loops, best of 3: 5.04 msec per loop

python -mtimeit "import itertools; sum(itertools.i map(int,
xrange(3000)))" 100 loops, best of 3: 3.6 msec per loop

I thought the two constructs could achieve the same speed.


hint: how many times to the interpreter have to look up the names
"int"
and "L" in the two examples ?


Ah right, thanks!
--
Giovanni Bajo
May 31 '06 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
1668
by: Bryan | last post by:
i thought that LC and genexp were supposed to be faster than map. i also thought i read somewhere in this group that the slowest looping mechanism in 2.4 is 20% faster than the fastest looping mechanism in 2.3. if i'm wrong about this, remembered wrong or did something wrong, please let me know. here are some timeit tests i did on my machine....
6
1506
by: Bengt Richter | last post by:
E.g., so we could write for x in seq if x is not None: print repr(x), "isn't None ;-)" instead of for x in (x for x in seq if x is not None): print repr(x), "isn't None ;-)"
6
2312
by: teedilo | last post by:
We have an application with a SQL Server 2000 back end that is fairly database intensive -- lots of fairly frequent queries, inserts, updates -- the gamut. The application does not make use of performance hogs like cursors, but I know there are lots of ways the application could be made more efficient database-wise. The server code is...
13
2742
by: bjarne | last post by:
Willy Denoyette wrote; > ... it > was not the intention of StrousTrup to the achieve the level of efficiency > of C when he invented C++, ... Ahmmm. It was my aim to match the performance of C and I achieved that aim very early on. See, for example "The Design and Evolution of C++". -- Bjarne Stroustrup;...
7
1235
by: Paul Rubin | last post by:
I tried to code the Sieve of Erastosthenes with generators: def sieve_all(n = 100): # yield all primes up to n stream = iter(xrange(2, n)) while True: p = stream.next() yield p # filter out all multiples of p from stream stream = (q for q in stream if q%p != 0)
0
7797
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main...
0
7715
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
8041
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
0
8093
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
6440
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5611
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
5296
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
1
2233
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
1
1333
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.