I have a doubt regarding list comprehensions:
According to Mark Lutz in his book Learning Pyhon:
"...there is currently a substantial performance advantage to the
extra complexity in this case: based on tests run under Python 2.2,
map calls are roughly twice as fast as equivalent for loops, and list
comprehensions are usually very slightly faster than map. This speed
difference owes to the fact that map and list comprehensions run at C
language speed inside the interpreter, rather than stepping through
Python for loop code within the PVM."
but I also read in Python Performance Tips by Skip Montanaro that
Lists comprehensions are not generally faster than the for loop
version.
What I'd like to know is if using list comprehensions would give me a
performance advantage over traditional for loops or not.
I'm getting fond of list comprehensions, but I wonder if it's wise to
abuse of them...
9  2354 
Neuruss> What I'd like to know is if using list comprehensions would
Neuruss> give me a performance advantage over traditional for loops or
Neuruss> not.
You can always run tests to see. <wink> There is some data dependency so it
makes sense to test map, listcomps and for loops using data that's typical
of the application you intend to use them in.
Skip
Yup, use the standard timeit module for testing.
Made for these purposes. . http://epydoc.sourceforge.net/stdlib...it-module.html
Cheers!
Remco
Skip Montanaro wrote: Neuruss> What I'd like to know is if using list comprehensions would
Neuruss> give me a performance advantage over traditional for loops or
Neuruss> not.
You can always run tests to see. <wink> There is some data dependency so it
makes sense to test map, listcomps and for loops using data that's typical
of the application you intend to use them in.
Skip
[Neuruss] What I'd like to know is if using list comprehensions would give me a performance advantage over traditional for loops or not.
For Py2.4, list comprehensions are much faster than equivalent for-loops.
I'm getting fond of list comprehensions, but I wonder if it's wise to
abuse of them...
Use whatever is clearest.
Don't abuse anything.
Raymond Hettinger
Neuruss <lu****@gmx.net > wrote:
... What I'd like to know is if using list comprehensions would give me a
performance advantage over traditional for loops or not.
Probably, a little, depending on the exact Python release you're using,
and on exactly what you're doing -- as others have suggested, timeit.py
can help. Don't expect _dramatic_ differences.
I'm getting fond of list comprehensions, but I wonder if it's wise to
abuse of them...
No, by definition of "abuse". Use list comprehensions to make lists,
not instead of perfectly normal loops. Consider for example:
kallisti:~/cb alex$ python2.4 timeit.py -s'def f():pass' 'for x in
xrange(999): f()'
1000 loops, best of 3: 1.25e+03 usec per loop
kallisti:~/cb alex$ python2.4 timeit.py -s'def f():pass' 'for x in
xrange(999): f()'
1000 loops, best of 3: 1.29e+03 usec per loop
kallisti:~/cb alex$ python2.4 timeit.py -s'def f():pass' '[f() for x in
xrange(999)]'
1000 loops, best of 3: 1.45e+03 usec per loop
kallisti:~/cb alex$ python2.4 timeit.py -s'def f():pass' '[f() for x in
xrange(999)]'
1000 loops, best of 3: 1.44e+03 usec per loop
So, in this case, abusing list comprehensions slows you down by over
10%.
More generally, _who cares_ about this kind of speedups?! Premature
optimization is the root of all evil in programming. Make your programs
clear, simple, readable -- that's ALWAYS important! So is using general
algorithms and data structures with good O() characteristics if the
input size is liable to grow a lot. But squeezing out 10% or 20% here
or there is going to matter in a TINY minority of cases. Don't distort
your coding for such purposes!
Alex
Raymond Hettinger <vz******@veriz on.net> wrote: [Neuruss] What I'd like to know is if using list comprehensions would give
me a > performance advantage over traditional for loops or not.
For Py2.4, list comprehensions are much faster than equivalent for-loops.
....but if the for loop is NOT equivalent (it doesn't accumulate results
into a resulting list), it's still faster. As I posted:
kallisti:~/cb alex$ python2.4 timeit.py -s'def f():pass' 'for x in
xrange(999): f()'
1000 loops, best of 3: 1.29e+03 usec per loop
kallisti:~/cb alex$ python2.4 timeit.py -s'def f():pass' '[f() for x in
xrange(999)]'
1000 loops, best of 3: 1.45e+03 usec per loop
the LC is paying for the building of a list of 999 references to None,
which the for loop easily avoids, so the for loop is much faster here.
Of course, it WAS way more pronounced in 2.3:
kallisti:~/cb alex$ python2.3 timeit.py -s'def f():pass' 'for x in
xrange(999): f()'
1000 loops, best of 3: 1.76e+03 usec per loop
kallisti:~/cb alex$ python2.3 timeit.py -s'def f():pass' '[f() for x in
xrange(999)]'
100 loops, best of 3: 2.43e+03 usec per loop I'm getting fond of list comprehensions, but I wonder if it's wise to
abuse of them...
Use whatever is clearest.
Don't abuse anything.
Absolutely. And: use 2.4 -- note that the SLOWER solution in 2.4 still
gains a good 20% over the FASTER one in 2.3... anybody who cares about
performance should be giving 2.4 an intense workout already, rarin' for
the day in which it will be officially released so it can be sensibly
used in customer-delivered software... (and, for those not in the know,
I should point out that Raymond was the one most responsible for the
overall impressive speed-up 2.2->2.3->2.4...!!!-).
Alex
Thank you guys!
I will investigate the timeit module as suggested for these kind of tests...
On Thu, 30 Sep 2004 10:55:52 +0200, al*****@yahoo.c om (Alex Martelli) wrote: Raymond Hettinger <vz******@veriz on.net> wrote:
[Neuruss] What I'd like to know is if using list comprehensions would give
me a > performance advantage over traditional for loops or not.
For Py2.4, list comprehensions are much faster than equivalent for-loops.
...but if the for loop is NOT equivalent (it doesn't accumulate results
into a resulting list), it's still faster. As I posted:
kallisti:~/cb alex$ python2.4 timeit.py -s'def f():pass' 'for x in
xrange(999): f()'
1000 loops, best of 3: 1.29e+03 usec per loop
kallisti:~/cb alex$ python2.4 timeit.py -s'def f():pass' '[f() for x in
xrange(999)]'
1000 loops, best of 3: 1.45e+03 usec per loop
the LC is paying for the building of a list of 999 references to None,
which the for loop easily avoids, so the for loop is much faster here.
What if you abuse the LC so it makes an empty list? E.g.,
[None for x in xrange(999) if f() and False]
Not that I'm trying to promote LC abuse ;-)
Regards,
Bengt Richter
Bengt Richter <bo**@oz.net> wrote:
... ...but if the for loop is NOT equivalent (it doesn't accumulate results
into a resulting list), it's still faster. As I posted:
kallisti:~/cb alex$ python2.4 timeit.py -s'def f():pass' 'for x in
xrange(999): f()'
1000 loops, best of 3: 1.29e+03 usec per loop
kallisti:~/cb alex$ python2.4 timeit.py -s'def f():pass' '[f() for x in
xrange(999)]'
1000 loops, best of 3: 1.45e+03 usec per loop
the LC is paying for the building of a list of 999 references to None,
which the for loop easily avoids, so the for loop is much faster here.
What if you abuse the LC so it makes an empty list? E.g.,
[None for x in xrange(999) if f() and False]
Now tell me, is timing so HARD...?! This obfuscation times (on 2.4 and
the same old iBook as above) to 1.38e+03 usec per loop, still slower
than the plain old loop -- the if/and rigmarole costs...
Alex
On Thu, 30 Sep 2004 20:13:04 +0200, al*****@yahoo.c om (Alex Martelli) wrote: Bengt Richter <bo**@oz.net> wrote:
... >...but if the for loop is NOT equivalent (it doesn't accumulate results
>into a resulting list), it's still faster. As I posted:
>
>kallisti:~/cb alex$ python2.4 timeit.py -s'def f():pass' 'for x in
>xrange(999): f()'
>1000 loops, best of 3: 1.29e+03 usec per loop
>kallisti:~/cb alex$ python2.4 timeit.py -s'def f():pass' '[f() for x in
>xrange(999)]'
>1000 loops, best of 3: 1.45e+03 usec per loop
>
>the LC is paying for the building of a list of 999 references to None,
>which the for loop easily avoids, so the for loop is much faster here.
What if you abuse the LC so it makes an empty list? E.g.,
[None for x in xrange(999) if f() and False]
Now tell me, is timing so HARD...?! This obfuscation times (on 2.4 and
the same old iBook as above) to 1.38e+03 usec per loop, still slower
than the plain old loop -- the if/and rigmarole costs...
Sure, I was just wondering what would happen if you traded that for list growth overhead.
Anyway, thanks. It _is_ hard to compare with someone else's previous timing numbers ;-)
Maybe timeit.py should have an option for bogomips normalization or returning
time*bogomips. My system needs an upgrade ;-)
Regards,
Bengt Richter This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: Elaine Jackson |
last post by:
List comprehensions don't work the way you intuitively expect them to work. I
realize many people have no intuitions about how list comprehensions 'should'
work, so if you recognize yourself in this description, feel free to go back to
whatever you were doing before. If you're still here, though, I invite you to
consider the following definition:
partitions = lambda n: ]++x for x in partitions(n-k) for k in
range(1,n)]
As defined...
|
by: Mahesh Padmanabhan |
last post by:
Hi,
When list comprehension was added to the language, I had a lot of
trouble understanding it but now that I am familiar with it, I am not
sure how I programmed in Python without it.
Now I see that generator expressions have been added to the language
with 2.4 and I question the need for it. I know that it allows for lazy
evaluation which speeds things up for larger lists but why was it
necessary to add it instead of improving list...
|
by: Mike Meyer |
last post by:
Ok, we've added list comprehensions to the language, and seen that
they were good. We've added generator expressions to the language, and
seen that they were good as well.
I'm left a bit confused, though - when would I use a list comp instead
of a generator expression if I'm going to require 2.4 anyway?
Thanks,
<mike
--
|
by: Steven Bethard |
last post by:
George Sakkis wrote:
> "Steven Bethard" <steven.bethard@gmail.com> wrote:
>> Dict comprehensions were recently rejected:
>> http://www.python.org/peps/pep-0274.html
>> The reason, of course, is that dict comprehensions don't gain you
>> much at all over the dict() constructor plus a generator expression,
>> e.g.:
>> dict((i, chr(65+i)) for i in range(4))
>
> Sure, but the same holds for list comprehensions: list(i*i for i in
|
by: Steven Bethard |
last post by:
Tom Anderson <twic@urchin.earth.li> wrote:
> Sounds good. More generally, i'd be more than happy to get rid of list
> comprehensions, letting people use list(genexp) instead. That would
> obviously be a Py3k thing, though.
Alex Martelli wrote:
> I fully agree, but the BDFL has already (tentatively, I hope)
> Pronounced that the form will stay in Py3K as syntax sugar for
> list(...). I find that to be a truly hateful prospect, but...
| | |
by: Lonnie Princehouse |
last post by:
List comprehensions appear to store their temporary result in a
variable named "_" (or presumably "_", "_" etc for nested
comprehensions)
In other words, there are variables being put into the namespace with
illegal names (names can't contain brackets). Can't someone come up
with a better hack than this? How about using "_1", "_2", etc, or
actually making "_" a list of lists and using the real first, second,
third elements? This is...
|
by: John Salerno |
last post by:
I posted this code last night in response to another thread, and after I
posted it I got to wondering if I had misused the list comprehension. Here's
the two examples:
Example 1:
--------------------
def compress(s):
new =
for c in s:
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
| | |
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |
