[Howto] Compiling debug Python extensions for non-debug Python

[Mike C. Fletcher]

Every few months I get to working on a Python C extension (built with
distutils) and discover a pointer error or the like where I'd like to be
able to step into my DLL and see where the extension is going wonky.

Now, the "proper" way to debug a DLL is apparently to build Python in
debug mode (i.e. with the _d form dll), then build each and every
extension I depend on (in debug mode), and then build each of my own
extensions in debug mode. That takes forever when there's 8 or 9
complex-to-build extensions involved in the system generating the error,
and is pointless when I could just print-debug in 1/100th of the time to
fix the particular error.

Now, it so happens that there's no reason I know of to do all that
rebuilding when all you want to do is to generate a program database and
see what's going on *in your own extension*. You can quite readily
create a debuggable DLL linked against the non-debug Python and all its
extensions with a 2 or 3 line change to msvccompiler.py in distutils. I
discovered that quite a while ago, but as the change is to a core Python
file, I tend to forget about it and overwrite it when I upgrade Python
and then spend a few hours re-discovering the exact switches, so, for my
own memory, and for any other extension writers who want this kind of
down-and-dirty debugging, here's the changes required:

In lib/distutils/msvccompiler.py:

self.compile_options = [
'/nologo',
'/Od', # turn off optimisations (allow debug)
'/MD', '/W3', '/GX',
'/Zi', # create a pdb file for the object files we create,
/Z7 works fine too
]
self.ldflags_shared = [
'/DLL', '/nologo', '/INCREMENTAL:YES',
'/DEBUG', # do debug-mode linking
]

then rebuild, forcing a recompile to make sure everything's been built
with the new flags.

There's probably some perfectly good reason not to do this as the
default for distutils "debug" mode for building a PYD, but I find it
*very* helpful, and hopefully others will as well.

Keywords: Visual C MSVC++ VC++ Win32 debug extension dll pyd distutils

Enjoy all,
Mike

_______________________________________
Mike C. Fletcher
Designer, VR Plumber, Coder
http://members.rogers.com/mcfletch/

[Edward C. Jones]

Mike C. Fletcher wrote:

Every few months I get to working on a Python C extension (built with
distutils) and discover a pointer error or the like where I'd like to be
able to step into my DLL and see where the extension is going wonky.

....

I have had the same problem in Linux. In the old DOS Borland Debugger, I
could tell the debugger to display a particular source code file. Then I
could set a breakpoint in it. Is this possible in gdb or some other UNIX
debugger? Has anyone ported the Borland Debugger to Linux?

[Neil Hodgson]

Edward C. Jones:

I have had the same problem in Linux. In the old DOS Borland Debugger, I
could tell the debugger to display a particular source code file. Then I
could set a breakpoint in it. Is this possible in gdb or some other UNIX
debugger? Has anyone ported the Borland Debugger to Linux?

Many of them do although it can be less than obvious how to do it.
KDevelop can do this although IIRC you first need to set up a dummy project
before being able to run an arbitrary executable. I have had problems with
Linux debuggers using inaccurate line number information although some of
this had been caused by using files from Windows with their \r\n line
endings. My current favourite Linux debugger is KDbg as it isn't an IDE and
so doesn't try to make you create project files.

Neil

[Martin v. Löwis]

Edward C. Jones wrote:

I have had the same problem in Linux. In the old DOS Borland Debugger, I
could tell the debugger to display a particular source code file. Then I
could set a breakpoint in it. Is this possible in gdb or some other UNIX
debugger? Has anyone ported the Borland Debugger to Linux?

In gdb, type

b <source>:<line>

Regards,
Martin