473,221 Members | 1,806 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,221 software developers and data experts.

Need Psyco profiling help...

Hey...as for what I'm doing with Python, look at my post "Strange Hotshot
problem". To make a long story short, I'm embedding Python in order to write a
plugin to a freeware music program; the plugin interfaces to the program with
C++, but the plugin calls Python routines to do much (but not all) of the work.

Which brings me to the problem. It's SLOW! Well, that's hardly surprising, but
anyway...I'm using Psyco, and it's definitely at least twice as fast with it
than it is without it. However this is not enough: my program produces crackles
and such as a result of hogging the CPU so much that there's latency. So
obviously I need to optimize.

But to optimize, I need to find a bottleneck, using a profiler. An obvious way
to do this with Psyco is to just use psyco.log('somefile.log') and then
psyco.profile()...however most of the time seems to be spent in Psyco's
active_start function! I decided the profiler may well be screwy so I tries
Hotshot. Well, Hotshot and Psyco don't work well together, I found out. But my
buddy CyanPhase suggested I write a do-nothing function in place of my Work
function, bind it with Psyco, and see how much CPU it takes. This function
simply filled an array of floats with zeroes and returned it (because the
caller needed some data to copy to another buffer). This only took 2.5% of the
CPU! Therefore the overhead of using Psyco certainly can't be eating up the
unaccounted-for 97.5% of the CPU, it has to be my code. But Psyco's profiler
says it isn't.

What's going on here? How can I find where the real bottleneck is so I can fix
it?

- Kef

Jul 18 '05 #1
4 2119
On Sat, Oct 25, 2003 at 01:49:40AM +0000, KefX wrote:

But to optimize, I need to find a bottleneck, using a profiler. An obvious way
to do this with Psyco is to just use psyco.log('somefile.log') and then
psyco.profile()...however most of the time seems to be spent in Psyco's
active_start function! I decided the profiler may well be screwy so I tries


As far as I'm aware, there's no way to use the standard python profiling
tools to profile non-python functions, like C extension functions or
psyco'd functions. Your best bet is to probably profile without psyco, and
optimise the bottlenecks you find in pure python. When you're done, try
using psyco again and see if performance is good enough. Very roughly
speaking, two psyco'd functions are likely to have roughly the same relative
performance as they did before psycoing, so hopefully this approach is good
enough to get the improvements you need.

-Andrew.
Jul 18 '05 #2
Can you run any test code *outside* of hotshot? This would certainly
tell you whether it was worth the effort. I.e. if you can't get it fast
outside of hotshot, it won't work inside ;)


Jul 18 '05 #3
KefX wrote:
Hotshot. Well, Hotshot and Psyco don't work well together, I found out.


You seem not to have thought about running Hotshot only (without Psyco)
to find out where your code spends most of its time. Psyco not only
isn't happy with Hotshot, but whatever results Hotshot could get are
wrong because it doesn't know about machine-code-compiled functions.

Also note that if the Python code has no long loop, e.g. if it is only
functions that are repeatedly called by C/C++ code (with the loop in
C/C++), then Psyco won't help at all, it will probably even slow things
down. It can only accelerate functions that run for some time (I mean,
doing at least 100 or 1000 iterations to fill some array should be ok,
but if C/C++ calls your function 100 or 1000 times to fill the same
array it is not).
A bientot,

Armin.

Jul 18 '05 #4
>Also note that if the Python code has no long loop, e.g. if it is only
functions that are repeatedly called by C/C++ code (with the loop in
C/C++), then Psyco won't help at all, it will probably even slow things
down. It can only accelerate functions that run for some time (I mean,
doing at least 100 or 1000 iterations to fill some array should be ok,
but if C/C++ calls your function 100 or 1000 times to fill the same
array it is not).


Psyco does speed it up considerably. I don't have any profiling data to back it
up, but the latency I hear in my synthesized instrument is greatly reduced when
I use Psyco. Or to put it in a sillier fashion, if the sound my synth should
make is "PIIIING!", with Psyco it goes "PI-I-I-ING!", and without it, it goes
"PI...I...I...I...N...G..." (in other words, the difference is VERY audible.
:))

The reason I don't really want to profile without Psyco is that I'm worried
that the data may not be representative of the conditions with Psyco. However,
it's becoming increasingly apparent that I may have no other choice.

- Kef

Jul 18 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

10
by: William S. Perrin | last post by:
I'm a python rookie, anyone have and suggestions to streamline this function? Thanks in advance..... def getdata(myurl): sock = urllib.urlopen(myurl) xmlSrc = sock.read() sock.close() ...
0
by: Jeremy Sanders | last post by:
Hi - I'm trying to build a Psyco rpm on Fedora 1. I try the command xpc5:~/psyco-1.1.1> python setup.py bdist_rpm this fails with: .... copying dist/psyco-1.1.1.tar.gz ->...
3
by: Dick Moores | last post by:
psyco is acting a bit psycho for me. Please see my spinForWeb.py at <http://www.rcblue.com/Python/spinForWeb.py> When psyco is in use, entering an integer somewhere between 2000 and 2500...
5
by: Fausto Arinos Barbuto | last post by:
Hi All; I have Psyco (on Windows XP) and now I want to install it on Linux, too. I FTP'd the tarball (tar.gz) from Psyco's site but can't get it compiled. First, I tried the usual "python...
6
by: danmcleran | last post by:
I'm not seeing much benefit from psyco (only 5-10% faster). Maybe this example is too trivial? Can someone give me some pointers as to what kind of code would see a dramatic benefit? Here's the...
3
by: a | last post by:
hi i tried psyco+webpy here is the error that i got please let me know if any of you has success run psyco+webpy thanks import web, psyco urls = ( '/', 'view', '/add','add'
0
by: L'eau Prosper Research | last post by:
Press Release: L'eau Prosper Research (Website: http://www.leauprosper.com) releases new TradeStation 8 Add-on - L'eau Prosper Market Manipulation Profiling Tools Set. L'eau Prosper Market...
0
by: L'eau Prosper Research | last post by:
NEW TradeStation 8 Add-on - L'eau Prosper Market Manipulation Profiling Tools Set By L'eau Prosper Research Press Release: L'eau Prosper Research (Website: http://www.leauprosper.com) releases...
1
by: Arash Arfaee | last post by:
Hello All, I am trying to use psyco with wingide on mac. when I open Mac Python shell I can import psyco, but not inside the wingide. Even python shell on wingide cannot import psyco. Can...
0
by: veera ravala | last post by:
ServiceNow is a powerful cloud-based platform that offers a wide range of services to help organizations manage their workflows, operations, and IT services more efficiently. At its core, ServiceNow...
0
by: VivesProcSPL | last post by:
Obviously, one of the original purposes of SQL is to make data query processing easy. The language uses many English-like terms and syntax in an effort to make it easy to learn, particularly for...
3
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 3 Jan 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). For other local times, please check World Time Buddy In...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: mar23 | last post by:
Here's the situation. I have a form called frmDiceInventory with subform called subfrmDice. The subform's control source is linked to a query called qryDiceInventory. I've been trying to pick up the...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.