473,324 Members | 2,313 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,324 software developers and data experts.

hotspot profiler experience and accuracy?

I have a parser I need to optimize. It has some disk IO and a lot of
looping over characters.

I used the hotspot profiler to gain insight on optimization options. The
methods show up on on the top of this list seems fairly trivial and does
not look like CPU hogger. Nevertheless I optimized it and have 25%
performance gain according to hotspot's number.

But the numbers look skeptical. Hotspot claim 71.166 CPU seconds but the
actual elapsed time is only 54s. When measuring elapsed time instead of
CPU time the performance gain is only 13% with the profiler running and
down to 10% when not using the profiler.

Is there something I misunderstood in reading the numbers?
Jul 18 '05 #1
3 2050
On Tue, 01 Feb 2005 19:24:20 -0800, aurora wrote:
I have a parser I need to optimize. It has some disk IO and a lot of
looping over characters.

I used the hotspot profiler to gain insight on optimization options. The
methods show up on on the top of this list seems fairly trivial and does
not look like CPU hogger. Nevertheless I optimized it and have 25%
performance gain according to hotspot's number.


I can't answer your other question, but general optimization advice, since
I've been in this situation a couple of times: Generally, you're not
going to win in an interpreted language like Python* looping over all
chars. You should either eliminate those loops if possible, or move the
guts of the looping into regular expressions, which are designed to do
that sort of thing as optimally as possible. (I've never looked at the
internals, but I believe the "compile()" function in the re module isn't
just a way of conveniently sticking a regex into a variable; you are
actually creating a compiled and reasonably optimized (as long as you
don't get crazy) character scanner that you simply Can Not beat in pure
Python.)

As a last-ditch scenario, you could go to a C extension, but regexs should
be good enough, and only beatable in the simplest of cases.

Write good REs (you could ask for help, but you should probably just test
it yourself; the key thing to try for, I think, is to put as much into one
RE as possible and ask the match object which thing matched but I may be
wrong; more experienced comments welcomed), and then run the profiler
again.

In general though, the precise numbers coming out of the profiler are less
important than their relationships; as long as the relationships are
maintained the data is still good.

*: At current technology levels. Yes, someday optimization will make
looping over characters in Python even faster than C, or so the theory
goes. That's not today, or even tomorrow, though.
Jul 18 '05 #2
aurora wrote:
But the numbers look skeptical. Hotspot claim 71.166 CPU seconds but
the actual elapsed time is only 54s. When measuring elapsed time
instead of CPU time the performance gain is only 13% with the profiler
running and down to 10% when not using the profiler.

Is there something I misunderstood in reading the numbers?


Well, I'm confused too. Look at my post from a few months ago:
http://tinyurl.com/6awzj
(note that my code contained a few errors and that you need
to use the fixed code that I posted a few replies later).

Perhaps somebody can explain a bit more about this this time? :-)

At the moment, frankly, hotspot seems rather useless.

--Irmen
Jul 18 '05 #3
Thanks for pointing me to your analysis. Now I know it wasn't me doing
something wrong.

hotspot did lead me to knock down a major performance bottleneck one time.
I found that zipfile.ZipFile() basically read the entire zip file in
instantiation time, even though you may only need one file from it
subsequencely.

In anycase the number of function call seems to make sense and it should
give some insight to the runtime behaviour. The CPU time is just so
misleading.
aurora wrote:
But the numbers look skeptical. Hotspot claim 71.166 CPU seconds but
the actual elapsed time is only 54s. When measuring elapsed time
instead of CPU time the performance gain is only 13% with the profiler
running and down to 10% when not using the profiler.
Is there something I misunderstood in reading the numbers?


Well, I'm confused too. Look at my post from a few months ago:
http://tinyurl.com/6awzj
(note that my code contained a few errors and that you need
to use the fixed code that I posted a few replies later).

Perhaps somebody can explain a bit more about this this time? :-)

At the moment, frankly, hotspot seems rather useless.

--Irmen


Jul 18 '05 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: FSJOSEPH | last post by:
To the Group: Hi all, I am using a java/pdf report engine namely Elixir Report Server. The report server uses JDK 1.4.1 with hotspot server JVM support. My application runs on AIX and the...
4
by: Naresh Agarwal | last post by:
Hi What are the different kinds of JVMs exist and in what respect do they differ. What is the difference between client, server, classic and hotspot jvms? thanks, Naresh
1
by: Mario_5 | last post by:
I made a small application which print some JPanel content: .... disableDoubleBuffering(componentToBePrinted); componentToBePrinted.paint(graphics); enableDoubleBuffering(componentToBePrinted);...
2
by: Cindy | last post by:
hi, Can someone tell me what is a hotspot button? Is it done using javascript? Thanks. Regards, Cindy
6
by: cournape | last post by:
Hi there, I have some scientific application written in python. There is a good deal of list processing, but also some "simple" computation such as basic linear algebra involved. I would like to...
1
by: John J. Hughes II | last post by:
Using the Visual C#.Net editor I have created a cursor.cur file as a resource. I would like to be able to set the hotspot. The problem is the "Set Hotspot Tool" is deselected. Anyone have a clue...
2
by: Perspolis | last post by:
Hi all I want to have hotSpot on my image. I put some lables on it and when I click image I can know where I select. it works but if my image zoom in/out I lose positions of lables.. what's your...
1
by: Lespaul36 | last post by:
I have a couple of custom cursors that I loaded as embeded resource and then create a customcursor array. Is there a way to change the hotspot on the cursors. I need one of them to be closer to...
3
by: Robert | last post by:
Hi, I'm interested in two applications for javascript. A profiler and a minimizer. I found several, but I am hoping to hear with which applications some of you guys have a good experience. I...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.