Problems with adding a new built-in data type

[Geoff Biggs]

Evening all,

I'm trying to add a new built-in number data type to Python with its own
syntax, so I'm working directly with the interpreter rather than
creating my own extension module (side note: I've appended something
extra to the version thing in the Makefile - I doubt this is relevant to
the problem but it's probably best you have all the info). The complex
data type is similar to what I'm trying to do so I've been following
that as an example. I've successfully extended the tokenizer and the
parsenumber() function in compile.c to do what I want and written the
data type in my two new files Objects/mynewobject.c and
Include/mynewobject.h. I've included mynewobject.h in Python.h and added
the two files to the Makefile.

However, when I tried to run the code by typing in the syntax to produce
my data type, Python suffers a segmentation fault. I traced this to an
attempt to get the hash of a null pointer when adding the new instance
of my type after parsenumber() was called. For completeness, here's the
backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x08080e9a in PyObject_Hash (v=0x81486c0) at Objects/object.c:1162
1162 if (tp->tp_hash != NULL)
(gdb) bt
#0 0x08080e9a in PyObject_Hash (v=0x81486c0) at Objects/object.c:1162
#1 0x0808e199 in tuplehash (v=0x40324574) at Objects/tupleobject.c:244
#2 0x08080eae in PyObject_Hash (v=0x40324574) at Objects/object.c:1163
#3 0x0807b46c in PyDict_GetItem (op=0x40319c14, key=0x40324574) at
Objects/dictobject.c:492
#4 0x080c72c7 in com_add (c=0xbfffed60, list=0x40326114,
dict=0x40319c14, v=0x40326290) at Python/compile.c:975
#5 0x080c74ac in com_addconst (c=0xbfffed60, v=0x40326290) at
Python/compile.c:1001
#6 0x080c90e3 in com_atom (c=0xbfffed60, n=0x4028d500) at
Python/compile.c:1689

The crash appears to be caused because while v exists, v's members are
all 0 and so tp becomes 0 when it is made equal to v->ob_type and you
get a NULL pointer exception. As far as I can tell, this v is the second
object in the tuple created in com_add() and is supposed to be the type
of the object being added to the dictionary in a tuple. Not knowing why
it was coming out as zero, I did some more poking around (been doing a
lot of that over the past 5 days...) and found that there is a file
called bltinmodule.c with a bunch of lines, one of which mentions the
complex data type that I am using as a guide:

SETBUILTIN("complex", &PyComplex_Type);

So I made one of these for mynewobject and added it. I figured from this
bit of code that the reason I was getting NULL pointer exceptions was
because my type hadn't been initialised in some way and that this would
do it. But here's the problem: despite trying for longer than I've slept
in the past week, I can't get it to work. With that line in the Python
interpreter segfaults as soon as it starts (while trying to import the
os module) and so it can't even finish compiling the extension modules.
If I comment out the SETBUILTIN line I added it will compile fine but
then I go back to the first problem.

I have structured my object identically to the complex object as far as
I can tell, which works. So the question is, what's broken? What have I
missed when adding a new builtin?

(This mess is all happening on Linux tillinshir 2.6.7-gentoo-r11-gb #1
Wed Aug 4 11:13:14 NZST 2004 i686 mobile AMD Athlon(tm) XP 2000+
AuthenticAMD GNU/Linux, if that matters.)

Thanks in advance,
Geoff Biggs

[Geoffrey Biggs]

No need to worry anymore, I've fixed the problem. If you're making a
built-in type, remember to define your PyTypeObject with something other
than NULL in the PyObject_HEAD_INIT. Non-built-ins don't need to worry
about this because it gets filled in later for them (as in the example
in the docs), but built-ins appear to need to fill it in themselves.

Geoff Biggs

Geoff Biggs wrote:

Evening all,

I'm trying to add a new built-in number data type to Python with its own
syntax, so I'm working directly with the interpreter rather than
creating my own extension module (side note: I've appended something
extra to the version thing in the Makefile - I doubt this is relevant to
the problem but it's probably best you have all the info). The complex
data type is similar to what I'm trying to do so I've been following
that as an example. I've successfully extended the tokenizer and the
parsenumber() function in compile.c to do what I want and written the
data type in my two new files Objects/mynewobject.c and
Include/mynewobject.h. I've included mynewobject.h in Python.h and added
the two files to the Makefile.

However, when I tried to run the code by typing in the syntax to produce
my data type, Python suffers a segmentation fault. I traced this to an
attempt to get the hash of a null pointer when adding the new instance
of my type after parsenumber() was called. For completeness, here's the
backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x08080e9a in PyObject_Hash (v=0x81486c0) at Objects/object.c:1162
1162 if (tp->tp_hash != NULL)
(gdb) bt
#0 0x08080e9a in PyObject_Hash (v=0x81486c0) at Objects/object.c:1162
#1 0x0808e199 in tuplehash (v=0x40324574) at Objects/tupleobject.c:244
#2 0x08080eae in PyObject_Hash (v=0x40324574) at Objects/object.c:1163
#3 0x0807b46c in PyDict_GetItem (op=0x40319c14, key=0x40324574) at
Objects/dictobject.c:492
#4 0x080c72c7 in com_add (c=0xbfffed60, list=0x40326114,
dict=0x40319c14, v=0x40326290) at Python/compile.c:975
#5 0x080c74ac in com_addconst (c=0xbfffed60, v=0x40326290) at
Python/compile.c:1001
#6 0x080c90e3 in com_atom (c=0xbfffed60, n=0x4028d500) at
Python/compile.c:1689

The crash appears to be caused because while v exists, v's members are
all 0 and so tp becomes 0 when it is made equal to v->ob_type and you
get a NULL pointer exception. As far as I can tell, this v is the second
object in the tuple created in com_add() and is supposed to be the type
of the object being added to the dictionary in a tuple. Not knowing why
it was coming out as zero, I did some more poking around (been doing a
lot of that over the past 5 days...) and found that there is a file
called bltinmodule.c with a bunch of lines, one of which mentions the
complex data type that I am using as a guide:

SETBUILTIN("complex", &PyComplex_Type);

So I made one of these for mynewobject and added it. I figured from this
bit of code that the reason I was getting NULL pointer exceptions was
because my type hadn't been initialised in some way and that this would
do it. But here's the problem: despite trying for longer than I've slept
in the past week, I can't get it to work. With that line in the Python
interpreter segfaults as soon as it starts (while trying to import the
os module) and so it can't even finish compiling the extension modules.
If I comment out the SETBUILTIN line I added it will compile fine but
then I go back to the first problem.

I have structured my object identically to the complex object as far as
I can tell, which works. So the question is, what's broken? What have I
missed when adding a new builtin?

(This mess is all happening on Linux tillinshir 2.6.7-gentoo-r11-gb #1
Wed Aug 4 11:13:14 NZST 2004 i686 mobile AMD Athlon(tm) XP 2000+
AuthenticAMD GNU/Linux, if that matters.)

Thanks in advance,
Geoff Biggs