473,394 Members | 1,755 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,394 software developers and data experts.

Tracing the execution of scripts?

Alright, I seem to be at a loss for what I am looking for, and I am not
even really all that sure if it is possible or not. I found the 'pdb'
debugger, but I was wondering if there was something that would trace or
log the order of line execution for a multi-module Python program. I am
having a little bit of a problem tracking down a problem that I
mentioned earlier
(http://groups.google.com/group/comp....59fc888b365be),
and I am hoping that maybe seeing how the statements are executed will
shed some light on the entire thing for me.

The debugger isn't working, though, because I have a timer set up to
fire off every 20ms to check for incoming network traffic from the
server, and that timer firing off makes stepping through with the
debugger seemingly impossible to get any useful data.

Basically, is there something that will log every line of Python code
executed, in its order of execution, to a text file so that I can see
what is (or isn't) happening that I am expecting? I know that the
server is sending and receiving properly, because I can connect to it
with a regular telnet client and it works -- but at the same time,
everything I can think of to check my program's functionality checks out
just fine (e.g., it reports that it is successfully sending what I am
typing to the server, and it says that the server is not sending
anything back to be read from the socket).

If it makes any difference, I am using wxWidgets' Python bindings for
the UI code (and the timer), and using Python's socket library for the
network stuff.

-- Mike
Oct 27 '06 #1
10 2416
Jean-Paul Calderone <ex*****@divmod.comwrites:
1) Write unit tests for your code. Keep writing unit tests until
you have some that _don't pass_. Then fix your code so that they
do. When you do further development, write the tests first, then
implement the code that makes them pass.
Hear hear.

Be advised, though, that attempting to apply unit tests to code that
wasn't designed with testing in mind may very quickly reveal a poor
design. [0] If you can't easily test pieces of the code independently,
you probably haven't written those pieces to be loosely coupled and
well-defined.

The moral? Writing unit tests *along with* the functional code will
result in a design that is loosely coupled, and probably
better-defined. Also, hopefully, easy to test :-)
[0] I have no idea whether this is the case for the OP. It's a very
common symptom that arises from people who are first advised to
introduce tests to their code, though.

--
\ "If you ever catch on fire, try to avoid seeing yourself in the |
`\ mirror, because I bet that's what REALLY throws you into a |
_o__) panic." -- Jack Handey |
Ben Finney

Oct 27 '06 #2
"Michael B. Trausch" <"mike$#at^&nospam!%trauschus"wrote:
Basically, is there something that will log every line of Python code
executed, in its order of execution, to a text file so that I can see
what is (or isn't) happening that I am expecting?
Python itself can do this for you. A __VERY__ simple approach:

|def myCallTrace (frame, event, arg):
| tracefile.write('localTrace\n frame: '+str(frame)+'\n event:
|'+str(event)+'\n arg: '+str(arg)+'\n')
| tracefile.write(' '+str(inspect.getframeinfo(frame))+'\n')
|
|def myTrace (frame, event, arg):
| tracefile.write('globalTrace\n frame: '+str(frame)+'\n event:
|'+str(event)+'\n arg: '+str(arg)+'\n')
| if event == 'call':
| return myCallTrace
|
|import inspect
|import sys
|
|sys.settrace(myTrace)
|tracefile = file('trace.txt', 'w')

Insert this in you program and you get a trace of every line of Python-Code
executed in the file trace.txt. You must read the documentation of the
module inspect and of sys.settrace() to understand, what happens and what
it means. Additionally, if it should work with threads, you must take care
that every thread gets its own output file. But as a first step to get a
trace of execution, this should do it.

HTH, Regards
Stephan

Oct 27 '06 #3
pydb (http://bashdb.sf.net/pydb) has a both the ability to trace lines
as they are executed as well as an --output option to have this sent
to a file rather than stdout. If your program has threads it would be
good to use the --threading option. (The best threading support is if
your program uses the threading module for threads rather than the
lower-level thread module.)

A demo of the debugger including showing line tracing and output to a file is
here:
http://showmedo.com/videos/video?nam...romSeriesID=28
"Michael B. Trausch" <"mike$#at^&nospam!%trauschus"writes:
Alright, I seem to be at a loss for what I am looking for, and I am not
even really all that sure if it is possible or not. I found the 'pdb'
debugger, but I was wondering if there was something that would trace or
log the order of line execution for a multi-module Python program. I am
having a little bit of a problem tracking down a problem that I
mentioned earlier
(http://groups.google.com/group/comp....59fc888b365be),
and I am hoping that maybe seeing how the statements are executed will
shed some light on the entire thing for me.

The debugger isn't working, though, because I have a timer set up to
fire off every 20ms to check for incoming network traffic from the
server, and that timer firing off makes stepping through with the
debugger seemingly impossible to get any useful data.

Basically, is there something that will log every line of Python code
executed, in its order of execution, to a text file so that I can see
what is (or isn't) happening that I am expecting? I know that the
server is sending and receiving properly, because I can connect to it
with a regular telnet client and it works -- but at the same time,
everything I can think of to check my program's functionality checks out
just fine (e.g., it reports that it is successfully sending what I am
typing to the server, and it says that the server is not sending
anything back to be read from the socket).

If it makes any difference, I am using wxWidgets' Python bindings for
the UI code (and the timer), and using Python's socket library for the
network stuff.

-- Mike
Oct 27 '06 #4
Ben Finney wrote:
Jean-Paul Calderone <ex*****@divmod.comwrites:
>1) Write unit tests for your code. Keep writing unit tests until
you have some that _don't pass_. Then fix your code so that they
do. When you do further development, write the tests first, then
implement the code that makes them pass.

Hear hear.

Be advised, though, that attempting to apply unit tests to code that
wasn't designed with testing in mind may very quickly reveal a poor
design. [0] If you can't easily test pieces of the code independently,
you probably haven't written those pieces to be loosely coupled and
well-defined.
I will whole-heartedly admit that my code is probably poorly designed.
*shrug* Though, I *do* try to write everything in such a way as to be
able to easily re-use it later. When I wrote little things just to help
me sysadmin, I learned that was really key to making my life easier.

Besides, I am lazy... reuse certainly serves my needs there! :-)
>
The moral? Writing unit tests *along with* the functional code will
result in a design that is loosely coupled, and probably
better-defined. Also, hopefully, easy to test :-)
I think I have more to learn on the concept of unit-testing. I have
never seen a group push the idea so hard. I have read about unit
testing before, and even written tests (in other languages) to test
modules of library code that I put together once, but that was about it.
I need to, I think, get more "into" the concept, though. It isn't
something that I am able to just whip out and go "Hey, a widget unit
test! And it works!" probably because of my own lack of practice.
>
[0] I have no idea whether this is the case for the OP. It's a very
common symptom that arises from people who are first advised to
introduce tests to their code, though.
It is very likely that it is the case. I am, by all means, what I could
consider to be a novice programmer when it comes to anything outside of
my little world of PHP. I can write web applications in PHP until the
cows come home with little to no problem, and I can play with SQL the
same way. But, I seem to have problems when I step out of that little
box, and so, that tells me that I need to work harder at it. :-)

-- Mike
Oct 27 '06 #5
Stephan Kuhagen wrote:
"Michael B. Trausch" <"mike$#at^&nospam!%trauschus"wrote:
>Basically, is there something that will log every line of Python code
executed, in its order of execution, to a text file so that I can see
what is (or isn't) happening that I am expecting?

Python itself can do this for you. A __VERY__ simple approach:
[snip]
>
Insert this in you program and you get a trace of every line of Python-Code
executed in the file trace.txt. You must read the documentation of the
module inspect and of sys.settrace() to understand, what happens and what
it means. Additionally, if it should work with threads, you must take care
that every thread gets its own output file. But as a first step to get a
trace of execution, this should do it.
I will need to keep this around to look at a little later. It looks
like it would be something particularly useful when line-tracing by
itself fails. I am constantly finding myself amazed at Python's
capabilities.

-- Mike
Oct 27 '06 #6
***********************
Your mail has been scanned by InterScan MSS.
***********************
On Friday 27 October 2006 17:31, R. Bernstein wrote:
pydb (http://bashdb.sf.net/pydb) has a both the ability to trace lines
I faced several time that pydb stuck without sign of errors. In the other hand
Pdb doesn't appear that problem.
Mostly pydb freeze on long loops.
It might be some problem on my setup, I'll check it up...

F

Oct 27 '06 #7
I just thought I would put my 2 cents in on this issue. Others
have suggested that unit tests are an excellent way of debugging
your code and I agree. I also find that writing code from the
outset using a logging class (there is one in the standard
library) that allows you to create log files of information as
you run your application is a good idea. Using increasingly
detailed logs (I use a debug mode of 1,2,3,4 for more detail)
that dump where you are and intermediate variable values works
EXTREMELY well for me. I leave this code in the application
so that I can have customers (or myself) run the application
in debug mode should I have a hard to find problem. This is
especially true for long running or lights-out batch apps
that have no UI making debugging even more difficult. I find
that the overhead of testing if I'm in debug mode and logging
results is almost non-existent to the overall execution speed
of my scripts, but then I don't have very many really speed
sensitive scripts so your mileage might vary.

Hope the information helps.

-Larry
Oct 27 '06 #8
Stephan Kuhagen wrote:
"Michael B. Trausch" <"mike$#at^&nospam!%trauschus"wrote:
Basically, is there something that will log every line of Python code
executed, in its order of execution, to a text file so that I can see
what is (or isn't) happening that I am expecting?

Python itself can do this for you. A __VERY__ simple approach:
...
Additionally, if it should work with threads, you must take care
that every thread gets its own output file.
Or be differentiated *somehow*. My pyconquer module tabs each thread's
activity into swimlanes, so you can even see when each thread starts,
obtains/releases the GIL, and stops. See the longer example here:
http://projects.amor.org/misc/wiki/PyConquer
Robert Brewer
System Architect
Amor Ministries
fu******@amor.org

Oct 27 '06 #9
Fulvio <fu****@tm.net.mywrites:
***********************
Your mail has been scanned by InterScan MSS.
***********************
Delighted to know that.
>
On Friday 27 October 2006 17:31, R. Bernstein wrote:
pydb (http://bashdb.sf.net/pydb) has a both the ability to trace lines

I faced several time that pydb stuck without sign of errors. In the other hand
Pdb doesn't appear that problem.
Mostly pydb freeze on long loops.
In version 1.19 (released today), I extended signal handling so that
you can send the program a signal and it will print a stack trace of
where it is and continue. But even before this release, you could
arrange to enter the debugger by sending it a signal. It's possible
something like this might be used help track down the problem in
either pydb or another Python program that seems to be not
responding. On the other hand, use at your own risk - I don't
guarentee this will work for you.

And.. before you ask for more details, I'll repeat what someone else
posted in response to another of your questions:

I guess you've been told to read this here, but just in case it
wasn't, or you didn't bother to read it:

http://catb.org/esr/faqs/smart-questions.html
It might be some problem on my setup, I'll check it up...
Given your other posts, that's quite possible. If it's not, submit a
bug report. (Posting to c.l.r isn't the same as submitting a bug
report). Thanks.
Oct 27 '06 #10
"Michael B. Trausch" <mi*************************@bag.python.orgwrite s:
Ben Finney wrote:
Jean-Paul Calderone <ex*****@divmod.comwrites:
1) Write unit tests for your code. Keep writing unit tests until
you have some that _don't pass_. Then fix your code so that they
do. When you do further development, write the tests first, then
implement the code that makes them pass.
Hear hear.

Be advised, though, that attempting to apply unit tests to code
that wasn't designed with testing in mind may very quickly reveal
a poor design. [0] If you can't easily test pieces of the code
independently, you probably haven't written those pieces to be
loosely coupled and well-defined.

I will whole-heartedly admit that my code is probably poorly
designed. *shrug* Though, I *do* try to write everything in such a
way as to be able to easily re-use it later.
The trick is to write pieces that are small and loosely-coupled, so
that small pieces of code (not just entire modules) can be re-used
without being re-written.

Test-driven development helps with this, because a small,
loosely-coupled piece of code is easy to test, so you will gravitate
toward that style.
Besides, I am lazy... reuse certainly serves my needs there! :-)
The programming virtues espoused by Larry Wall continue to hold true :-)
I think I have more to learn on the concept of unit-testing.
I have never seen a group push the idea so hard.
Writing each test before the functional code is not a new idea, but
Python makes it trivially easy to do, so the excuses for not doing it
are much weaker and easier to ridicule :-)
I have read about unit testing before, and even written tests (in
other languages) to test modules of library code that I put together
once, but that was about it.
Here's an article that goes through the concepts, and happens to use
Python.

<URL:http://www.onlamp.com/pub/a/python/2004/12/02/tdd_pyunit.html>
I need to, I think, get more "into" the concept, though. It isn't
something that I am able to just whip out and go "Hey, a widget unit
test! And it works!" probably because of my own lack of practice.
One thing to note is that UI widgets shouldn't be the main thing
you're testing. You need to write code that can be easily tested
independent of everything else, because that way you can know that it
works and move on to other things quickly, building on it as a
foundation.

UI widgets should be as dumb as possible, because they don't make a
whole lot of sense by themselves except in the dumbest sense. Every
part of your program that actually does something interesting with
data or resources should be utterly independent of any UI
widgets. *That* is the code that you need to get right, so that should
be riddled with tests.

Your widgets should be plugging into code that dumbly passes the data
to the well-tested, UI-independent code that does work with that data;
and the widgest should be getting their data from such UI-independent
outputs. This is one aspect of "loose coupling": the code talks along
a very narrow, well-defined, simple interface to other parts of the
code.

UI code especially is prone to change entirely independent of the
desired functionality. for this reason, it's vital that you *can*
change the functionality and UI code each without needing to change
the other. The same principle applies along any interface between
parts of code that can be expected to change independent of each
other: loosely couple them together, with narrow well-defined
interfaces.

You may have heard of the "Model, View, Controller" pattern. This is a
design pattern that emphasises a loose coupling between the UI (the
"controller") and the data model, by abstracting each to the other via
a simple "view" interface module. That way, each of the Controller and
the Model can be oblivious to the most common changes in the other,
even when those changes make for complex code, because the View they
interact with stays simple and doesn't need to change.

<URL:http://en.wikipedia.org/wiki/Model-view-controller>

The reason test-driven development is so useful here is that, in order
to write a test for something *before* it exists, you must think about
how that piece of code will interact with the rest of the code. You
must, in other words, design its interface before writing its
code. This is excellent, because it means you're focussing on exactly
the thing that will help to keep your overall design loosely coupled.

Then, after you have a test that interacts with this interface, all
you need to do is write the simplest code to satisfy that
interface. The temptation to write clever, complex code is reduced,
because you're now interested in satisfying the external test, not in
treating this piece of code as an awesome complex machine ignoring the
rest of the code.

Good luck with getting test infected :-)

<URL:http://c2.com/cgi/wiki?TestInfected>

--
\ "I bought a self learning record to learn Spanish. I turned it |
`\ on and went to sleep; the record got stuck. The next day I |
_o__) could only stutter in Spanish." -- Steven Wright |
Ben Finney

Oct 28 '06 #11

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: karoshi | last post by:
Hello, I want to gain insight into the access and running of scripts by the php engine on my production box. Most scripts are executed via cronjobs and they invoke a lot of other php scripts....
6
by: Paolo Losi | last post by:
Hi all, I'm pretty new to the python language so please excuse me if this is FAQ... I'm very glad to be part of the list! :-) I'm looking into a way to implement a generic workflow framework...
10
by: Sorawit Sharp | last post by:
My app executes C# user scripts via dynamic code execution. Is it possible to execute the scripts single step wise like a debugger ?
5
by: Patrick | last post by:
I understand it is built in behaviour that if an ASP.NET's web.config is set to: <customErrors mode="RemoteOnly" /> then I only get a detailed error message on screen when the ASP.NET...
6
by: serge calderara | last post by:
Dear all, I have an applicatin that generate a querry to an SQL server, then display results on a second webform. I try to see how tracing works, then I have notice that as soon as I...
2
by: deepukutty | last post by:
Hi all, I know tht we can do tracing in two ways.one in application level and the other is at Page level. I am able to see the details of trace either on the page itself or .../trace.axd page....
0
by: cnys | last post by:
We have an ASP.NET 2.0 (C#) app and we're trying to add tracing into it. The tracing functionality within .NET is great, but when we output this to a file, it's kind of sparse. So, we're looking...
1
by: Sabiyur | last post by:
Hi All, I don't know whether this is the right forum to post this question. If not, Please forgive me. I want to write my own tool which can trace the code execution (print the sequence of...
4
by: Michel Esber | last post by:
Environment: DB2 v8 LUW FP 15 running on Linux. For some reason that I canīt explain, a simple insert statement on a table may run very quickly, or may take forever (20-30mins) to finish....
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.