Profiling evaluation tools?

[Jens Theisen]

Hello,

I want to apologise in advance for this being off topic. It's not neither
A C nor a C++ question, but to profiling in general, though I my chances
are best to find the answer in the C/C++ community.

I have a C++ program to profile and went about it by producing large
history files of calling dependencies with associated times. It is
presumably similar to gprof's data format and could be converted.

I'm now looking for utilities that can evaluate such a tree. I only
looked at the gprof's manual page and that seemed too basic to me.

I already wrote some Python scripts that can extract stuff like:

- Break the time down into that spent in each (libraries, files,
functions, ...), exclusive to that spent on their behalf in other
such entities. Print it in order.

- Give a tree of (libraries, files, functions) according to their
calling dependencies, but don't recurse into those subentities below
a certain threshhold of consumed time. Give the number of calls and
time (including the time spent in subentities).

All this is already quite useful, but I also dearly want to have:

- My script is already "merging together" subtrees on the same
level. For instance, if f calls g a number of times, and g h a
number of times, you only get f->g->h once in the tree view of the
data, the number at the line of h being the total calls to h through
this callchain. However, if a call to f occurs at a number of
different places in the code, those occurences will not be merged
together: I can't say, view any call to f as a the "whole program"
and forget other calls, merging all these subtrees into one.

- I have a general structure of attributes to each node (ie library, file
and function names), but I also want to have additional attributes like
the value of a parameter the function was called with, which may only
apply to some functions. I need to filter out certain values of these
attributes or break down to their values.

- Reverse tree view: Given a (library, file, function), how often was
it called and how much time did it spend on the behalf of all it's
calling parent, grandparents, etc., again breaking recursion on a
certain time threshold. A special case of this is the ability to
extract a backtrace from the profiling data, which is useful on it's
own.

I have a number of vague ideas how to go about this.

My scripts are in Python, but they are messy and slow. An exotic
approach would be xslt, since it's designed to operate on tree-data. I
have zero experience in this regard - is this possible? how does it
perform? Another one is trying to use databases as a storage for
trees, but I'm not sure if all of these operations will be efficient.

The most realistic option I would go about in the absence of further
advice and existing utilities is to rewrite my scripts in C++.

All of this needs properly thinking through, so I'd really much rather
use an exisiting solution. Is there one?

Cheers,

Jens

[Flash Gordon]

Jens Theisen wrote:

Hello,

I want to apologise in advance for this being off topic. It's not neither
A C nor a C++ question,
If you know it is not a C or C++ question and know it is therefor off
topic here, don't ask it here.
but to profiling in general, though I my chances
are best to find the answer in the C/C++ community.
No, you are most likely to get complained at for posting in what you
know is the wrong place.
I have a C++ program to profile and went about it by producing large
history files of calling dependencies with associated times. It is
presumably similar to gprof's data format and could be converted.

<snip>

Since you are using GNU tools, try asking in GNU mailing lists.
Alternatively try asking in a more general group or an OS specific
group. Don't ask in what you *know* is the wrong group.
--
Flash Gordon
Living in interesting times.
Web site - http://home.flash-gordon.me.uk/
comp.lang.c posting guidlines and intro -
http://clc-wiki.net/wiki/Intro_to_clc

[Ico]

In comp.lang.c Jens Theisen <jt***@arcor.de > wrote:

I have a C++ program to profile and went about it by producing large
history files of calling dependencies with associated times. It is
presumably similar to gprof's data format and could be converted.

[... snipped a lot of requested features ...]

<offtopic answer following>

If you happen to work on linux, take a look at the excellent callgrind
and kcachegrind tools. Callgrind is a tool that runs your code in a
virtual machine. It can do profiling, call tracing, cache profiling and
a lot more. Kcachegrind is a gui for showing the profiling results in
various ways.

http://kcachegrind.sourceforge.net/cgi-bin/show.cgi
--
:wq
^X^Cy^K^X^C^C^C ^C

[Jens Theisen]

Flash wrote:

Since you are using GNU tools, try asking in GNU mailing lists.
Alternatively try asking in a more general group or an OS specific
group. Don't ask in what you *know* is the wrong group.

An OS specific group would be just as wrong, since it's not an OS specific
problem.

I searched for newsgroups or mailinglists for profiling related talks but
failed in finding them.

I don't know what the right group is to ask, and obviously you don't
either.

Jens

[Jens Theisen]

Ico wrote:

http://kcachegrind.sourceforge.net/cgi-bin/show.cgi

That's indeed an interesting start, thank you very much!

Jens

[Mark McIntyre]

On 15 Feb 2006 13:27:00 +0000, in comp.lang.c , jt***@arcor.de (Jens
Theisen) wrote:

Flash wrote:

Since you are using GNU tools, try asking in GNU mailing lists.
Alternatively try asking in a more general group or an OS specific
group. Don't ask in what you *know* is the wrong group.

An OS specific group would be just as wrong, since it's not an OS specific
problem.

The thing is, there is NO generic C answer to this as its a toolset
thing, not a C thing. You need to ask in a group specialising in each
OS or toolset you're interested in. From your original post, you are
interested in only one OS and toolset, so ask over there.
Mark McIntyre
--
"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan

----== Posted via Newsfeeds.Com - Unlimited-Unrestricted-Secure Usenet News==----
http://www.newsfeeds.com The #1 Newsgroup Service in the World! 120,000+ Newsgroups
----= East and West-Coast Server Farms - Total Privacy via Encryption =----

[Jens Theisen]

Mark wrote:

The thing is, there is NO generic C answer to this as its a toolset
thing, not a C thing. You need to ask in a group specialising in each
OS or toolset you're interested in. From your original post, you are
interested in only one OS and toolset, so ask over there.
Mark McIntyre

I think my original posting was very clear in that it's not at all
specific to a certain toolset.

Jens

[CBFalconer]

Jens Theisen wrote:

Mark wrote:

The thing is, there is NO generic C answer to this as its a toolset
thing, not a C thing. You need to ask in a group specialising in each
OS or toolset you're interested in. From your original post, you are
interested in only one OS and toolset, so ask over there.
Mark McIntyre

I think my original posting was very clear in that it's not at all
specific to a certain toolset.

Maybe not, but the solution necessarily must be. Either the
compiler/runtime system has special provisions, or you must distort
the original source code to inject something or other. You don't
have a namespace to play in, since things are either reserved for
the implementation or for the user.

--
"The power of the Executive to cast a man into prison without
formulating any charge known to the law, and particularly to
deny him the judgement of his peers, is in the highest degree
odious and is the foundation of all totalitarian government
whether Nazi or Communist." -- W. Churchill, Nov 21, 1943

[Keith Thompson]

jt***@arcor.de (Jens Theisen) writes:

Flash wrote:

Since you are using GNU tools, try asking in GNU mailing lists.
Alternatively try asking in a more general group or an OS specific
group. Don't ask in what you *know* is the wrong group.

An OS specific group would be just as wrong, since it's not an OS specific
problem.

I searched for newsgroups or mailinglists for profiling related talks but
failed in finding them.

I don't know what the right group is to ask, and obviously you don't
either.

That doesn't imply that *this* is the right group.

You might try comp.programmin g.

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[Flash Gordon]

Jens Theisen wrote:

Flash wrote:

Since you are using GNU tools, try asking in GNU mailing lists.
Alternatively try asking in a more general group or an OS specific
group. Don't ask in what you *know* is the wrong group.
An OS specific group would be just as wrong, since it's not an OS specific
problem.

Many OS specific groups deal with tool chains.
I searched for newsgroups or mailinglists for profiling related talks but
failed in finding them.

I don't know what the right group is to ask, and obviously you don't
either.

There are a number of possible general groups including alt.comp.misc
which from the name is a very general group to comp.programmer which is
another very general group amongst others. Is it really so hard to try a
generalist group when you can't find a specific group?

Would you go in to a Hi-fi shop and ask for advice on how to repair a
printer just because you can't find a printer specialist, or would you
go in to a general computer repair centre? They both have a lot of
electronics in, after all.
--
Flash Gordon
Living in interesting times.
Web site - http://home.flash-gordon.me.uk/
comp.lang.c posting guidlines and intro -
http://clc-wiki.net/wiki/Intro_to_clc