Profiling evaluation tools?

[Jens Theisen]

Hello,

I want to apologise in advance for this being off topic. It's not neither
A C nor a C++ question, but to profiling in general, though I my chances
are best to find the answer in the C/C++ community.

I have a C++ program to profile and went about it by producing large
history files of calling dependencies with associated times. It is
presumably similar to gprof's data format and could be converted.

I'm now looking for utilities that can evaluate such a tree. I only
looked at the gprof's manual page and that seemed too basic to me.

I already wrote some Python scripts that can extract stuff like:

- Break the time down into that spent in each (libraries, files,
functions, ...), exclusive to that spent on their behalf in other
such entities. Print it in order.

- Give a tree of (libraries, files, functions) according to their
calling dependencies, but don't recurse into those subentities below
a certain threshhold of consumed time. Give the number of calls and
time (including the time spent in subentities).

All this is already quite useful, but I also dearly want to have:

- My script is already "merging together" subtrees on the same
level. For instance, if f calls g a number of times, and g h a
number of times, you only get f->g->h once in the tree view of the
data, the number at the line of h being the total calls to h through
this callchain. However, if a call to f occurs at a number of
different places in the code, those occurences will not be merged
together: I can't say, view any call to f as a the "whole program"
and forget other calls, merging all these subtrees into one.

- I have a general structure of attributes to each node (ie library, file
and function names), but I also want to have additional attributes like
the value of a parameter the function was called with, which may only
apply to some functions. I need to filter out certain values of these
attributes or break down to their values.

- Reverse tree view: Given a (library, file, function), how often was
it called and how much time did it spend on the behalf of all it's
calling parent, grandparents, etc., again breaking recursion on a
certain time threshold. A special case of this is the ability to
extract a backtrace from the profiling data, which is useful on it's
own.

I have a number of vague ideas how to go about this.

My scripts are in Python, but they are messy and slow. An exotic
approach would be xslt, since it's designed to operate on tree-data. I
have zero experience in this regard - is this possible? how does it
perform? Another one is trying to use databases as a storage for
trees, but I'm not sure if all of these operations will be efficient.

The most realistic option I would go about in the absence of further
advice and existing utilities is to rewrite my scripts in C++.

All of this needs properly thinking through, so I'd really much rather
use an exisiting solution. Is there one?

Cheers,

Jens

[usenet]

In comp.lang.c Flash Gordon <sp**@flash-gordon.me.uk> wrote:

Jens Theisen wrote:

Flash wrote:

Since you are using GNU tools, try asking in GNU mailing lists.
Alternatively try asking in a more general group or an OS specific
group. Don't ask in what you *know* is the wrong group.

An OS specific group would be just as wrong, since it's not an OS specific
problem.

Would you go in to a Hi-fi shop and ask for advice on how to repair a
printer just because you can't find a printer specialist, or would you
go in to a general computer repair centre? They both have a lot of
electronics in, after all.

If I can't find a computer repair centre in my town, I'd go into the
Hi-fi shop and friendly ask something like:

"Good morning, I want to apologise in advance for this being in the
wrong shop. It's not neither A receiver nor CD-player question, but
electronics in general, though I my chances are best to find the answer
in the local hi-fi shop"

They might just be able to answer my question (did you plug the power
cord in?), or kindly tell me that there *is* a general computer repair
center down the road, that I didn't know about.

Whatever the outcome, everybody would be kind, friendly and happy.

[Neil Cerutti]

["Followup-To:" header set to comp.lang.c++.]
On 2006-02-16, us****@zevv.nl <us****@zevv.nl > wrote:

Would you go in to a Hi-fi shop and ask for advice on how to repair
a printer just because you can't find a printer specialist, or
would you go in to a general computer repair centre? They both have
a lot of electronics in, after all.
If I can't find a computer repair centre in my town, I'd go into the
Hi-fi shop and friendly ask something like:

"Good morning, I want to apologise in advance for this being in the
wrong shop. It's not neither A receiver nor CD-player question, but
electronics in general, though I my chances are best to find the
answer in the local hi-fi shop"

Hi-fi shops are used to incredibly strange questions, so while they
probably would be no help, they might get a laugh. Like the time time
my brother was asked for "a new needle for my 8-track player". The guy
was extremely disappointed in the answer. ;-)
They might just be able to answer my question (did you plug the power
cord in?), or kindly tell me that there *is* a general computer repair
center down the road, that I didn't know about.

Whatever the outcome, everybody would be kind, friendly and happy.

That depends on several unknown factors. Rudeness in electronics shops
is quite possible.

--
Neil Cerutti

[Michael Wojcik]

In article <43************ ***@yahoo.com>, CBFalconer <cb********@yah oo.com> writes:

Jens Theisen wrote:

Mark wrote:

The thing is, there is NO generic C answer to this as its a toolset
thing, not a C thing.

I think my original posting was very clear in that it's not at all
specific to a certain toolset.

Maybe not, but the solution necessarily must be. Either the
compiler/runtime system has special provisions, or you must distort
the original source code to inject something or other.

While I agree that the query was OT for c.l.c (and presumably for
c.l.c++), both Mark's response and yours are incorrect. There are
profiling tools which are specific to language implementations (which
it appears is what "toolset" refers to here), but there are also
those that are not.

Of the three profiling techniques I'm familiar with, only one -
instrumentation - is specific to a language implementation. Your
post describes an instrumenting profiler. That is not the only
mechanism for profiling.

Sampling and virtualizing profilers need not be tied to a particular
implementation, and indeed often are not. They are generally created
for a particular OS, since they require fairly intimate interaction
with the execution environment. (Obviously hardware-based sampling
profilers can be at least somewhat OS-independent.)

Given all of that, OS-specific groups do indeed appear to be the
correct place to ask.

--
Michael Wojcik mi************@ microfocus.com

Even 300 years later, you should plan it in detail, when it comes to your
summer vacation. -- Pizzicato Five

[Jens Theisen]

Michael wrote:

While I agree that the query was OT for c.l.c (and presumably for
c.l.c++), both Mark's response and yours are incorrect. There are
profiling tools which are specific to language implementations (which
it appears is what "toolset" refers to here), but there are also
those that are not.

Why I also agree that my OP was off-topic, I do so for opposite reasons.

I don't need a Profiler, I need something to convert the data of an
already profiled application into information.

Such a solution could be based on xslt as well as sql or something
completely different.

That said, you're all right that this is not the place for that
discussion.

Jens