473,597 Members | 2,045 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Getting better traceback info on exec and execfile - introspection?

In doing the extension to the python debugger which I have here:
http://sourceforge.net/project/showf...kage_id=175827
I came across one little thing that it would be nice to get done better.

I notice on stack traces and tracebacks, an exec or execfile command
appears as a stack entry -- probably as it should since it creates a
new environment.

However the frame information for exec or execfile looks like this:
File "<string>", line 1, in ?

Okay, the "line 1" probably changes, but most of the time it's going
to be one. And in code that's in CVS (cvsview:
http://cvs.sourceforge.net/viewcvs.py/bashdb/pydb/) you'll see that
instead of this I now put some thing like:

## 1 in exec '<string>' at line 1

which is perhaps is a little more honest since one is not really in a
file called <string>. However the way the debugger gets this *is*
still a little hoaky in that it looks for something in the frame's
f_code.co_filen ame *called* <string>. And from that it *assumes* this
is an exec, so it can't really tell the difference between an exec
command an execfile command, or a file called <string>. But I suppose
a little more hoakiness *could* be added to look at the outer frame
and parse the source line for "exec" or "execfile".

And suppose instead of '<string>' I'd like to give the value or the
leading prefix of the value instead of the unhelpful word '<string>'?
How would one do that? Again, one way is to go into the outer frame
get the source line (if that exists), parse that and interpolate
argument to exec(file). Is there are better way?

Another and perhaps more direct way to distinguish exec/execfile might
be to some how look at the byte code of the frame entry and decode
that. It also could be used for example in another place in the
debugger where one wants to skip over executing "def" statements (not
the function invocation, but the statement which adds the
function/method so it can get called). But depending on how things are
done unless there are nice version-independent and
implementation-independent symbolic definitions for exec_opcode,
execfile_opcode and def_opcode, although straight-forward and reliable
for a given version/implementation it might not be a good thing to do.

Any help, thoughts or pointers?
Jan 14 '06 #1
8 3312
I suggested:
And suppose instead of '<string>' I'd like to give the value or the
leading prefix of the value instead of the unhelpful word '<string>'?
How would one do that? Again, one way is to go into the outer frame
get the source line (if that exists), parse that and interpolate
argument to exec(file). Is there are better way?
I've hacked this approach into the CVS for pydb, but it is not all
that accurate (although in practice I'm pleased with the result).

My current regular expression is (^|\s+)exec\s+( .*) which does match
simple things, but it would fail on say by *not* matching:
x=1;exec "y=2";exec "z=3"
and fail by *wrongly* matching on:
x=" exec meeting in 5 mins"

Even if I knew how to call the python parser on the line, that still
might not be good enough in the presence of continuation lines.

This probably should be addressed at a lower level in gathering
traceback data inside Python.
Another and perhaps more direct way to distinguish exec/execfile might

^^^^^^^^
execfile is not a problem, just exec.
Jan 15 '06 #2
R. Bernstein wrote:
In doing the extension to the python debugger which I have here:
http://sourceforge.net/project/showf...kage_id=175827 I came across one little thing that it would be nice to get done better.

I notice on stack traces and tracebacks, an exec or execfile command
appears as a stack entry -- probably as it should since it creates a
new environment.

However the frame information for exec or execfile looks like this:
File "<string>", line 1, in ?


That comes from how the code object was compiled:
code = compile('error' ,'Anything you want goes here','single')
exec code

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "Anything you want goes here", line 1, in ?
NameError: name 'error' is not defined
For example, in ipython:

In [1]: error
---------------------------------------------------------------------------
exceptions.Name Error Traceback (most recent
call last)

/home/fperez/<ipython console>

NameError: name 'error' is not defined
So any regexp-matching based approach here is likely to be fairly brittle,
unless you restrict your tool to the standard python interpreter, and you
get some guarantee that it will always tag interactive code with
'<string>'.

Regards,

f

Jan 15 '06 #3
Fernando Perez <fp********@gma il.com> writes:
R. Bernstein wrote: ....
However the frame information for exec or execfile looks like this:
File "<string>", line 1, in ?


That comes from how the code object was compiled:

.... So any regexp-matching based approach here is likely to be fairly brittle,
unless you restrict your tool to the standard python interpreter, and you
get some guarantee that it will always tag interactive code with
'<string>'.
Thanks for the information! Alas, that's bad news. The question then
remains as to how to accurately determine if a frame is running an
exec operation and accurately get the string associated with the exec.

Given all the introspection about python's introspection, it should be
possible.

I also tried pydb.py using IPython 0.7.0 (using an automatically
updated Fedora Core 5 RPM) and although I was expecting pydb.py breakage
from what you report, I didn't find any difference.
% ipython
Python 2.4.2 (#1, Dec 17 2005, 13:02:22)
Type "copyright" , "credits" or "license" for more information.

IPython 0.7.0 -- An enhanced Interactive Python.
? -> Introduction to IPython's features.
%magic -> Information about IPython's 'magic' % functions.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: %run /usr/lib/python2.4/site-packages/pydb.py test1.py /home/rocky/python/test1.py(2)?()

-> import sys
(Pydb) where
-> 0 in file '/home/rocky/python/test1.py' at line 2
## 1 in exec cmd in globals, locals at line 1
## 2 run() called from file '/usr/lib/python2.4/bdb.py' at line 366
(Pydb)

Note entry ##1 is *not*
File "<string>", line 1, in ?

So somehow I guess ipython is not intercepting or changing how compile
was run here.
Jan 15 '06 #4
R. Bernstein wrote:
Fernando Perez <fp********@gma il.com> writes:
R. Bernstein wrote: ...
> However the frame information for exec or execfile looks like this:
> File "<string>", line 1, in ?


That comes from how the code object was compiled:

...
So any regexp-matching based approach here is likely to be fairly
brittle, unless you restrict your tool to the standard python
interpreter, and you get some guarantee that it will always tag
interactive code with '<string>'.


Thanks for the information! Alas, that's bad news. The question then
remains as to how to accurately determine if a frame is running an
exec operation and accurately get the string associated with the exec.

Given all the introspection about python's introspection, it should be
possible.


I thought a little about this. One possibility would be to check whether
the argument given in the string exists as a file; if not, assume it's a
string. I can't think of another way, since the string given is completely
arbitrary. But someone who knows more about the internal structure of code
objects may give you better tips.
I also tried pydb.py using IPython 0.7.0 (using an automatically
updated Fedora Core 5 RPM) and although I was expecting pydb.py breakage
from what you report, I didn't find any difference.
% ipython
Python 2.4.2 (#1, Dec 17 2005, 13:02:22)
Type "copyright" , "credits" or "license" for more information.

IPython 0.7.0 -- An enhanced Interactive Python.
? -> Introduction to IPython's features.
%magic -> Information about IPython's 'magic' % functions.
help -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]: %run /usr/lib/python2.4/site-packages/pydb.py test1.py
/home/rocky/python/test1.py(2)?()

-> import sys
(Pydb) where
-> 0 in file '/home/rocky/python/test1.py' at line 2
## 1 in exec cmd in globals, locals at line 1
## 2 run() called from file '/usr/lib/python2.4/bdb.py' at line 366
(Pydb)

Note entry ##1 is *not*
File "<string>", line 1, in ?

So somehow I guess ipython is not intercepting or changing how compile
was run here.


Oh, that's because you're using %run, so your code is in complete control.
What I meant about a restriction was thiking about using your pydb as the
debugger called by ipython when exceptions occur.

Try

%pdb

and then %run any script with an error in it, to be dropped in post-mortem
mode inside the stack. That uses an ipython-enhanced pdb, but not your
pydb. As your code matures and improves, it would be nice to, for example,
make it possible to plug it into ipython as a 'better debugger'. But
there, you'll see '<ipython console>' as the filename markers.

Anyway, good luck with this. If you are interested in ipython integration,
I suggest the ipython-dev list as a better place for discussion: I only
monitor c.l.py on occasion, so I may well miss things here.

Regards,

f

Jan 15 '06 #5
R. Bernstein wrote:
..
..
..
which is perhaps is a little more honest since one is not really in a
file called <string>. However the way the debugger gets this *is*
still a little hoaky in that it looks for something in the frame's
f_code.co_filen ame *called* <string>. And from that it *assumes* this
is an exec, so it can't really tell the difference between an exec
command an execfile command, or a file called <string>. But I suppose
a little more hoakiness *could* be added to look at the outer frame
and parse the source line for "exec" or "execfile".
You should check the getFrameInfo function in zope.interface package:
http://svn.zope.org/Zope3/trunk/src/...77&view=markup
And suppose instead of '<string>' I'd like to give the value or the
leading prefix of the value instead of the unhelpful word '<string>'?
How would one do that? Again, one way is to go into the outer frame
get the source line (if that exists), parse that and interpolate
argument to exec(file). Is there are better way?


Py library (http://codespeak.net/py/current/doc/index.html) has some
similar functionality in the code subpackage.

Jan 15 '06 #6
"Ziga Seilnacht" <zi************ @gmail.com> writes:
You should check the getFrameInfo function in zope.interface package:
http://svn.zope.org/Zope3/trunk/src/...77&view=markup


Thanks! Just looked at that. The logic in the relevant part (if I've extracted
this correctly):

f_globals = frame.f_globals
hasName = '__name__' in f_globals
module = hasName and sys.modules.get (f_globals['__name__']) or None
namespaceIsModu le = module and module.__dict__ is f_globals
if not namespaceIsModu le:
# some kind of funky exec
kind = "exec"

is definitely not obvious to me. exec's don't have module namespace
(or something or other)? Okay. And that the way to determine the
module-namespace thingy is whatever that logic is? Are the assumptions
here be likely to be valid in the future?

Another problem I have with that code is that it uses the Zope Public
License. But the code is adapted from the Python Enterprise
Application Kit (PEAK) which doesn't seem to use the Zope Public
License. I'm not sure it's right to adopt a Zope Public License just
for this.

So instead, I followed the other avenue I suggested which is
dissembling the statement around the frame. Talk about the kettle
calling the pot black! Yes, I don't know if the assumptions in this
method are likely to be valid in the future either.

But I can use op-code examination to also help me determine if we are
stopped at a def statement which I want to skip over. So here's the
code I've currently settled on:

from opcode import opname
def op_at_frame(fra me):
code = frame.f_code.co _code
pos = frame.f_lasti
op = ord(code[pos])
return opname[op]

def is_exec_stmt(fr ame):
"""Return True if we are looking at an exec statement"""
return frame.f_back is not None and op_at_frame(fra me.f_back)=='EX EC_STMT'

re_def = re.compile(r'^\ s*def\s+')
def is_def_stmt(lin e, frame):
"""Return True if we are looking at a def statement"""
# Should really also check that the operand is a code object
return re_def.match(li ne) and op_at_frame(fra me)=='LOAD_CONS T'

But it may just be because I wrote it that I find it easier to
understand and more straightforward to fathom.
And suppose instead of '<string>' I'd like to give the value or the
leading prefix of the value instead of the unhelpful word '<string>'?
How would one do that? Again, one way is to go into the outer frame
get the source line (if that exists), parse that and interpolate
argument to exec(file). Is there are better way?


Py library (http://codespeak.net/py/current/doc/index.html) has some
similar functionality in the code subpackage.


Again, many thanks. I'll have to study this further. It may be that
exec and so on are wrapped so that it's possible to squirrel away the
string before calling exec. Again, dunno. But thanks for the pointer.
Jan 16 '06 #7
Fernando Perez <fp********@gma il.com> writes:
I thought a little about this. One possibility ...
Thanks. A sibling thread has the code I'm currently using.
Oh, that's because you're using %run, so your code is in complete control.
What I meant about a restriction ... Okay.
If you are interested in ipython integration,
Yes I am.
I suggest the ipython-dev list as a better place for discussion: I only
monitor c.l.py on occasion, so I may well miss things here.


Okay now subscribed. But interestingly I looked in the IPython PDF and
didn't see mention of it when looking for contact info. I do however
see it (now) listed on http://ipython.scipy.org/. Thanks again.
Jan 16 '06 #8
Fernando Perez <fp********@gma il.com> writes:
So any regexp-matching based approach here is likely to be fairly brittle,
unless you restrict your tool to the standard python interpreter, and you
get some guarantee that it will always tag interactive code with
'<string>'.


Meant to mention for what it's worth. Looks like I'm not the first to use the
filename == '<string>' test. I note this in the stock pdb.py:

# To be overridden in derived debuggers
def defaultFile(sel f):
"""Produce a reasonable default."""
filename = self.curframe.f _code.co_filena me
if filename == '<string>' and self.mainpyfile :
filename = self.mainpyfile
return filename

I'm not sure under what conditions this equality test normally occurs
though.
Jan 16 '06 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
17367
by: Jonathan | last post by:
I'm puzzled by Python's behavior when binding local variables which are introduced within exec() or execfile() statements. First, consider this simple Python program: # main.py def f() : x = 1 print "x:", x f()
3
3991
by: | last post by:
Hi everybody, I'm sure there's a way to do this, but I can't find it. How can I execute another .py file from my first .py file without using an exec* command? They're both in the same directory, and it would be nice to have some run("another.py") type statement as opposed to a big exec with absolute pathnames and garbage like that. It works the way I have it, but it just seems like a bad way in general to do it. I'm runnning Python...
2
6705
by: Chris S. | last post by:
I'd like to dynamically execute multiple lines of indented code from within a script, but I can't seem to find a suitable function. Exec only works with unindented code, and execfile only works with files. I suppose I could write my string to a temporary file and then use execfile, but that seems like a hack. Is there an easier way? Any help is appreciated.
2
4180
by: tedsuzman | last post by:
----- def f(): ret = 2 exec "ret += 10" return ret print f() ----- The above prints '12', as expected. However,
5
14791
by: David Rasmussen | last post by:
If I have a string that contains the name of a function, can I call it? As in: def someFunction(): print "Hello" s = "someFunction" s() # I know this is wrong, but you get the idea... /David
1
8061
by: tkpmep | last post by:
I installed SciPy and NumPy (0.9.5, because 0.9.6 does not work with the current version of SciPy), and had some teething troubles. I looked around for help and observed that the tutorial is dated October 2004, and is not as thorough as Python's documentation. Is there an alternative source of information that lists all the functions and their usage? I tried using scipy.info to get information on the std function in the stats libary,...
5
2629
by: Edward K Ream | last post by:
It looks like both exec and execfile are converting "\n" to an actual newline in docstrings! Start idle: Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) on win32
1
2257
by: Fernando Perez | last post by:
Hi all, I'm finding the following behavior truly puzzling, but before I post a bug report on the site, I'd rather be corrected if I'm just missing somethin obvious. Consider the following trivial script: # Simple script that imports something from the stdlib from math import sin, pi
8
1992
by: gregpinero | last post by:
I'm running code via the "exec in context" statement within a much larger program. What I would like to do is capture any possible errors and show a pretty traceback just like the Python interactive interpreter does, but only show the part of the traceback relating to the code sent to exec. For example here is the code I'm using: try: exec code
0
7965
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8380
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8031
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
6686
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
5847
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5426
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
3881
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
3923
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
2399
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.