Initialize lists; here's why

[bartonc]

Most other programming languages, as well as Python extension packages require arrays to be initialized to a particular size. Well, it turns out that there is an advantage to doing this with native python lists, too (that is, of course, if you know the size of the list beforehand).

I wrote two tests in a module (call it module1):

def listAppendTest(n):
    aList = []
    for i in range(n):
        aList.append(i)
 
def listInitTest(n):
    aList = [0] * n
    for i in range(n):
        aList[i] = 1
 

In a second module I use timeit to check this theory:

import timeit
 
test1ExecStr = \
"""from module1 import listAppendTest
listAppendTest(5)"""
 
t = timeit.Timer(test2ExecStr)
print t.timeit()
 
# 22.7464275488
 
test2ExecStr = \
"""from module1 import listInitTest
listInitTest(5)"""
 
t = timeit.Timer(test2ExecStr)
print t.timeit()
 
# 17.7503707873

Interesting results, no?

[robin746]

Your testing only five times? I doubt that is statistically sound. What happens when you try 1000?

In any case trying to micromanage performance like this is really the wrong way to programme. First, find if your app is not fast enough, then profile, then fix the bottleneck.

Remember the first rule of optimisation: Don't.

[bvdet]

Your testing only five times? I doubt that is statistically sound. What happens when you try 1000?

In any case trying to micromanage performance like this is really the wrong way to programme. First, find if your app is not fast enough, then profile, then fix the bottleneck.

Remember the first rule of optimisation: Don't.

By my recollection, timeit() defaults to 1000000 iterations.

I routinely optimize as I program where possible. If I know a one method is more efficient than another, I use the most efficient, unless additional functionality is required. Occasionally I use timeit() just so I will know.

Would you use string concatenation repeatedly to build long strings or the string join method? Here's another example:

Method 1:

def __add__(self, other):
        return Point(*[a+b for a,b in zip(self,other)])

Method 2:

def __add__(self, other):
        return Point(self.x+other[0], self.y+other[1], self.z+other[2])

Which one should I use?

[bartonc]

Your testing only five times? I doubt that is statistically sound. What happens when you try 1000?

In any case trying to micromanage performance like this is really the wrong way to programme. First, find if your app is not fast enough, then profile, then fix the bottleneck.

Remember the first rule of optimisation: Don't.

How 'bout 5 million times?

[bartonc]

Most other programming languages, as well as Python extension packages require arrays to be initialized to a particular size. Well, it turns out that there is an advantage to doing this with native python lists, too (that is, of course, if you know the size of the list beforehand).

I wrote two tests in a module (call it module1):

def listAppendTest(n):
    aList = []
    for i in range(n):
        aList.append(i)
 
def listInitTest(n):
    aList = [0] * n
    for i in range(n):
        aList[i] = 1
 

In a second module I use timeit to check this theory:

import timeit
 
test1ExecStr = \
"""from module1 import listAppendTest
listAppendTest(5)"""
 
t = timeit.Timer(test2ExecStr)
print t.timeit()
 
# 22.7464275488
 
test2ExecStr = \
"""from module1 import listInitTest
listInitTest(5)"""
 
t = timeit.Timer(test2ExecStr)
print t.timeit()
 
# 17.7503707873

Interesting results, no?

Interestingly, (and this is something that I've been meaning to do for a long time) the import is taking a significant amount of time.

import timeit
 
test1ExecStr = \
"""from test3 import listAppendTest"""
 
t = timeit.Timer(test1ExecStr)
print t.timeit()
 
# 8.945650355
 
test2ExecStr = \
"""from test3 import listInitTest"""
 
t = timeit.Timer(test2ExecStr)
print t.timeit()
 
# 8.87637339383