In article <88*************************@posting.google.com> ,
kl*******@home.com (Kamilche) wrote:
I have a routine that I really need, but it slows down processing
significantly. Can you spot any ineffeciencies in the code?
This code makes a critical function of mine run about 7x slower than
using a prebuilt format string. For maximum flexibility, it would be
best to calculate the format string using this method, so I'd dearly
love to keep it.
def fmtstring(args):
delim = '\0'
fmt = []
fmt.append('<')
for arg in args:
t = type(arg)
if t == types.StringType:
l = len(arg)
fmt.append(str(l) + 's')
elif t == types.IntType:
fmt.append('i')
elif t == types.LongType:
fmt.append('q')
elif t == types.BooleanType:
fmt.append('c')
elif t == types.FloatType:
fmt.append('d')
else:
raise Exception("Can't pack argument of type %s!" % t)
s = ''.join(fmt)
s = s + '\0'
return s
The first thing is to profile your code and make sure this really is
significant in the overall scheme of things. From your description, it
sounds like you've already done that, so I'll go with that assumption.
What's the most likely value for type(arg)? If 90% of your args are
floats, for example, most of the time you work your way all the way down
the if/else tree before getting to the right spot. Putting the tests in
order of probability of occurance will help you a little.
Doing "from types import *" and then being able to just say "IntType"
instead of "types.IntType" will be a little faster since there will be
one less name lookup.
Another thing you can try is doing dictionary lookups instead of an
if/else tree. Something like:
map = {IntType: 'i',
LongType: 'q',
BooleanType: 'c',
FloatType: 'd'}
[...]
try:
fmt.append (map[t])
except KeyError:
raise Exception ("Can't pack %s" % t)
You still need to special-case StringType, however.
Lastly, is an idea which is really quite gross. If you're desperate for
speed, the majority of your args are strings, and you have no pride, it
might be worth trying, however. Instead of saying:
t = type(arg)
if t == types.StringType:
l = len(arg)
you could do something like:
try:
l = len (arg)
fmt.append(str(l) + 's')
except TypeError:
process all the non-string types
The idea here is to not waste time getting and testing the type, just
assume it's a string and deal with the consequences later if your
assumption was wrong. In real life, this plays out as, "It's easier to
ask forgiveness than permission".
Of the types you mentioned, string is the only one which which has a
length; all the others will raise TypeError when passed to len().
Exception processing is relatively slow, but if it only happens a small
percentage of the time, you might be ahead of the game this way.
Some people would call this elegant, others would call it a gross hack.
I guess it depends on your perspective, and how desperate you are to
make your code run faster :-)
Of course, you could put several of these ideas together. Try the
exception-catching idea for strings first, and then sort out the other 4
types using a dictionary lookup.
My gut feeling is that even with all these tricks, the best you'll be
seeing is a factor of 2 or 3 speedup. I'd be amazed if you could get
anywhere near the 7x you're looking for.
