473,323 Members | 1,622 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,323 software developers and data experts.

Need Psyco profiling help...

Hey...as for what I'm doing with Python, look at my post "Strange Hotshot
problem". To make a long story short, I'm embedding Python in order to write a
plugin to a freeware music program; the plugin interfaces to the program with
C++, but the plugin calls Python routines to do much (but not all) of the work.

Which brings me to the problem. It's SLOW! Well, that's hardly surprising, but
anyway...I'm using Psyco, and it's definitely at least twice as fast with it
than it is without it. However this is not enough: my program produces crackles
and such as a result of hogging the CPU so much that there's latency. So
obviously I need to optimize.

But to optimize, I need to find a bottleneck, using a profiler. An obvious way
to do this with Psyco is to just use psyco.log('somefile.log') and then
psyco.profile()...however most of the time seems to be spent in Psyco's
active_start function! I decided the profiler may well be screwy so I tries
Hotshot. Well, Hotshot and Psyco don't work well together, I found out. But my
buddy CyanPhase suggested I write a do-nothing function in place of my Work
function, bind it with Psyco, and see how much CPU it takes. This function
simply filled an array of floats with zeroes and returned it (because the
caller needed some data to copy to another buffer). This only took 2.5% of the
CPU! Therefore the overhead of using Psyco certainly can't be eating up the
unaccounted-for 97.5% of the CPU, it has to be my code. But Psyco's profiler
says it isn't.

What's going on here? How can I find where the real bottleneck is so I can fix
it?

- Kef

Jul 18 '05 #1
4 2128
On Sat, Oct 25, 2003 at 01:49:40AM +0000, KefX wrote:

But to optimize, I need to find a bottleneck, using a profiler. An obvious way
to do this with Psyco is to just use psyco.log('somefile.log') and then
psyco.profile()...however most of the time seems to be spent in Psyco's
active_start function! I decided the profiler may well be screwy so I tries


As far as I'm aware, there's no way to use the standard python profiling
tools to profile non-python functions, like C extension functions or
psyco'd functions. Your best bet is to probably profile without psyco, and
optimise the bottlenecks you find in pure python. When you're done, try
using psyco again and see if performance is good enough. Very roughly
speaking, two psyco'd functions are likely to have roughly the same relative
performance as they did before psycoing, so hopefully this approach is good
enough to get the improvements you need.

-Andrew.
Jul 18 '05 #2
Can you run any test code *outside* of hotshot? This would certainly
tell you whether it was worth the effort. I.e. if you can't get it fast
outside of hotshot, it won't work inside ;)


Jul 18 '05 #3
KefX wrote:
Hotshot. Well, Hotshot and Psyco don't work well together, I found out.


You seem not to have thought about running Hotshot only (without Psyco)
to find out where your code spends most of its time. Psyco not only
isn't happy with Hotshot, but whatever results Hotshot could get are
wrong because it doesn't know about machine-code-compiled functions.

Also note that if the Python code has no long loop, e.g. if it is only
functions that are repeatedly called by C/C++ code (with the loop in
C/C++), then Psyco won't help at all, it will probably even slow things
down. It can only accelerate functions that run for some time (I mean,
doing at least 100 or 1000 iterations to fill some array should be ok,
but if C/C++ calls your function 100 or 1000 times to fill the same
array it is not).
A bientot,

Armin.

Jul 18 '05 #4
>Also note that if the Python code has no long loop, e.g. if it is only
functions that are repeatedly called by C/C++ code (with the loop in
C/C++), then Psyco won't help at all, it will probably even slow things
down. It can only accelerate functions that run for some time (I mean,
doing at least 100 or 1000 iterations to fill some array should be ok,
but if C/C++ calls your function 100 or 1000 times to fill the same
array it is not).


Psyco does speed it up considerably. I don't have any profiling data to back it
up, but the latency I hear in my synthesized instrument is greatly reduced when
I use Psyco. Or to put it in a sillier fashion, if the sound my synth should
make is "PIIIING!", with Psyco it goes "PI-I-I-ING!", and without it, it goes
"PI...I...I...I...N...G..." (in other words, the difference is VERY audible.
:))

The reason I don't really want to profile without Psyco is that I'm worried
that the data may not be representative of the conditions with Psyco. However,
it's becoming increasingly apparent that I may have no other choice.

- Kef

Jul 18 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

10
by: William S. Perrin | last post by:
I'm a python rookie, anyone have and suggestions to streamline this function? Thanks in advance..... def getdata(myurl): sock = urllib.urlopen(myurl) xmlSrc = sock.read() sock.close() ...
0
by: Jeremy Sanders | last post by:
Hi - I'm trying to build a Psyco rpm on Fedora 1. I try the command xpc5:~/psyco-1.1.1> python setup.py bdist_rpm this fails with: .... copying dist/psyco-1.1.1.tar.gz ->...
3
by: Dick Moores | last post by:
psyco is acting a bit psycho for me. Please see my spinForWeb.py at <http://www.rcblue.com/Python/spinForWeb.py> When psyco is in use, entering an integer somewhere between 2000 and 2500...
5
by: Fausto Arinos Barbuto | last post by:
Hi All; I have Psyco (on Windows XP) and now I want to install it on Linux, too. I FTP'd the tarball (tar.gz) from Psyco's site but can't get it compiled. First, I tried the usual "python...
6
by: danmcleran | last post by:
I'm not seeing much benefit from psyco (only 5-10% faster). Maybe this example is too trivial? Can someone give me some pointers as to what kind of code would see a dramatic benefit? Here's the...
3
by: a | last post by:
hi i tried psyco+webpy here is the error that i got please let me know if any of you has success run psyco+webpy thanks import web, psyco urls = ( '/', 'view', '/add','add'
0
by: L'eau Prosper Research | last post by:
Press Release: L'eau Prosper Research (Website: http://www.leauprosper.com) releases new TradeStation 8 Add-on - L'eau Prosper Market Manipulation Profiling Tools Set. L'eau Prosper Market...
0
by: L'eau Prosper Research | last post by:
NEW TradeStation 8 Add-on - L'eau Prosper Market Manipulation Profiling Tools Set By L'eau Prosper Research Press Release: L'eau Prosper Research (Website: http://www.leauprosper.com) releases...
1
by: Arash Arfaee | last post by:
Hello All, I am trying to use psyco with wingide on mac. when I open Mac Python shell I can import psyco, but not inside the wingide. Even python shell on wingide cannot import psyco. Can...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.