432,474 Members | 966 Online + Ask a Question
Need help? Post your question and get tips & solutions from a community of 432,474 IT Pros & Developers. It's quick & easy.

# Profiling, sum-comprehension vs reduce

 P: n/a This must be because of implementation right? Shouldn't reduce be faster since it iterates once over the list? doesnt sum first construct the list then sum it? ----------------------- >>================================ RESTART ================================ reduce with named function: 37.9864357062 reduce with nested, named function: 39.4710288598 reduce with lambda: 39.2463927678 sum comprehension: 25.9530121845 >>================================ RESTART ================================ reduce with named function: 36.4529584067 reduce with nested, named function: 37.6278529813 reduce with lambda: 38.2629448715 sum comprehension: 26.0197561422 >>> from timeit import Timer def add(x,y): return x+y def rednamed(lst): return reduce(add, lst) def rednn(lst): def add2(x,y): return x+y return reduce(add2, lst) def redlambda(lst): return reduce(lambda x,y:x+y, lst) def com(lst): return sum(x for x in lst) s = xrange(101) t1 = Timer('rednamed(s)', 'from __main__ import rednamed, s') t2 = Timer('rednn(s)', 'from __main__ import rednn, s') t3 = Timer('redlambda(s)', 'from __main__ import redlambda, s') t4 = Timer('com(s)', 'from __main__ import com, s') print "reduce with named function: ", t1.timeit() print "reduce with nested, named function: ", t2.timeit() print "reduce with lambda: ", t3.timeit() print "sum comprehension: ", t4.timeit() --------------------------------------- also, using range instead of xrange doesnt seem to generate a performance-penalty: >>> reduce with named function: 36.7560729087 reduce with nested, named function: 38.5393266463 reduce with lambda: 38.3852953378 sum comprehension: 27.9001007111 >>> Sep 13 '08 #1
7 Replies

 P: n/a On Sat, 13 Sep 2008 01:06:22 -0700, cnb wrote: This must be because of implementation right? Shouldn't reduce be faster since it iterates once over the list? doesnt sum first construct the list then sum it? No it doesn't. Why should it? also, using range instead of xrange doesnt seem to generate a performance-penalty: (De)Allocating a list of length 100 isn't very slow. Try some million elements. And watch the memory consumption too. Ciao, Marc 'BlackJack' Rintsch Sep 13 '08 #2

 P: n/a doesnt sum first construct the list then sum it? def com(lst): return sum(x for x in lst) You construct a generator over an existing list in your code. Try sum([x for x in lst]) to see the effect of additional list construction. And while you're at it, try the simple sum(lst). Cheers, - harold - Sep 13 '08 #3

 P: n/a On Sep 13, 10:06*am, cnb

 P: n/a On Sat, 13 Sep 2008 01:06:22 -0700, cnb wrote: This must be because of implementation right? Shouldn't reduce be faster since it iterates once over the list? doesnt sum first construct the list then sum it? What makes you think that? Given the speed of sum(), it sure doesn't look like it's generating a full list before summing. Why would it? reduce with named function: 37.9864357062 reduce with nested, named function: 39.4710288598 reduce with lambda: 39.2463927678 sum comprehension: 25.9530121845 If you want to see reduce really shine, time it with a C-based function rather than one written in pure Python: >>Timer('reduce(add, xrange(10000))', .... 'from operator import add').repeat(number=10000) [19.724750995635986, 19.410486936569214, 19.614511013031006] >>>Timer('reduce(add, xrange(10000))', .... 'def add(x, y): return x+y').repeat(number=10000) [45.210143089294434, 44.814558982849121, 46.906874895095825] You probably won't see much (if any) benefit for small lists, so make sure your test is on a significantly-sized input list. Of course, sum() is even faster than reduce: >>Timer('sum(xrange(10000))').repeat(number=1000 0) [9.814924955368042, 8.7169640064239502, 9.5062401294708252] -- Steven Sep 13 '08 #5

 P: n/a Steven D'Aprano wrote: If you want to see reduce really shine, time it with a C-based function rather than one written in pure Python: >>>Timer('reduce(add, xrange(10000))', ... 'from operator import add').repeat(number=10000) [19.724750995635986, 19.410486936569214, 19.614511013031006] >>>Timer('reduce(add, xrange(10000))', ... 'def add(x, y): return x+y').repeat(number=10000) [45.210143089294434, 44.814558982849121, 46.906874895095825] .... Of course, sum() is even faster than reduce: >>>Timer('sum(xrange(10000))').repeat(number=10000 ) [9.814924955368042, 8.7169640064239502, 9.5062401294708252] 'Of course', because the irreducible difference between reduce(add.seq) and sum(seq) is that reduce has to call an add function while sum has the operation built-in in place of a call. Sep 13 '08 #6 