Using alternative PyObject* in dynamically created classes

[Jacek Generowicz]

I would like some of my _dynamically_ created new-style classes to
have instances which are not represented by the standard PyObject, but
one which has an extra void* slot which I want to use for my nefarious
purposes, in extension functions. The pointer need not be initialized
(for now) I just want it to be there in case I want to stuff something
in there later on in the run.

What is the laziest way of achieving this?

As far as I can tell, PyTypeObject (and its allocators, deallocators,
methods in general) only refer to the type of the instance struct via
tp_basicsize (where only the struct's size is held).

So my first attempt merely resets tp_basicsize to the size of my
alternative instance struct, as soon as the class is created. This
goes horribly wrong when instances of the class are deallocated, but I
can't see why.

Where else must I provide information about the instance struct type ?
[I will be away for a few days, from a few hours from now; please
don't take a long response time as a sign of rudeness.]

[Jacek Generowicz]

Jacek Generowicz <ja**************@cern.ch> writes:

I would like some of my _dynamically_ created new-style classes to
have instances which are not represented by the standard PyObject, but
one which has an extra void* slot which I want to use for my nefarious
purposes, in extension functions. The pointer need not be initialized
(for now) I just want it to be there in case I want to stuff something
in there later on in the run.

What is the laziest way of achieving this?

As far as I can tell, PyTypeObject (and its allocators, deallocators,
methods in general) only refer to the type of the instance struct via
tp_basicsize (where only the struct's size is held).

So my first attempt merely resets tp_basicsize to the size of my
alternative instance struct, as soon as the class is created. This
goes horribly wrong when instances of the class are deallocated, but I
can't see why.

Where else must I provide information about the instance struct type ?
[I will be away for a few days, from a few hours from now; please
don't take a long response time as a sign of rudeness.]

Hmm ... no takers ?

:-(

[Thomas Heller]

Jacek Generowicz <ja**************@cern.ch> writes:

Jacek Generowicz <ja**************@cern.ch> writes:

I would like some of my _dynamically_ created new-style classes to
have instances which are not represented by the standard PyObject, but
one which has an extra void* slot which I want to use for my nefarious
purposes, in extension functions. The pointer need not be initialized
(for now) I just want it to be there in case I want to stuff something
in there later on in the run.

What is the laziest way of achieving this?

As far as I can tell, PyTypeObject (and its allocators, deallocators,
methods in general) only refer to the type of the instance struct via
tp_basicsize (where only the struct's size is held).

So my first attempt merely resets tp_basicsize to the size of my
alternative instance struct, as soon as the class is created. This
goes horribly wrong when instances of the class are deallocated, but I
can't see why.

Where else must I provide information about the instance struct type ?
[I will be away for a few days, from a few hours from now; please
don't take a long response time as a sign of rudeness.]

Hmm ... no takers ?

:-(

Can't you simply create a new, subclassable Python type with an
additonal void* slot, and derive your classes from it? Where's the
problem?

Thomas

[Jacek Generowicz]

Thomas Heller <th*****@python.net> writes:

Jacek Generowicz <ja**************@cern.ch> writes:

Jacek Generowicz <ja**************@cern.ch> writes:

So my first attempt merely resets tp_basicsize to the size of my
alternative instance struct, as soon as the class is created. This
goes horribly wrong when instances of the class are deallocated, but I
can't see why.

OK ... type_new augments the size it finds in tp_basicsize with space
for extra slots ... so resetting tp_basicsize once the type has been
created is indeed too late.
Can't you simply create a new, subclassable Python type with an
additonal void* slot, and derive your classes from it? Where's the
problem?

You mean statically create an extension type with the appropriate size
in the tp_basicsize, and dynamically create types which inherit from
it?

Sounds like it should work.

I'll give it a go, as I now see that my initial approach is guaranteed
to be wrong.

Thanks.

[Ronald Oussoren]

On 23-nov-04, at 14:07, Jacek Generowicz wrote:

Jacek Generowicz <ja**************@cern.ch> writes:

I would like some of my _dynamically_ created new-style classes to
have instances which are not represented by the standard PyObject, but
one which has an extra void* slot which I want to use for my nefarious
purposes, in extension functions. The pointer need not be initialized
(for now) I just want it to be there in case I want to stuff something
in there later on in the run.

What is the laziest way of achieving this?

As far as I can tell, PyTypeObject (and its allocators, deallocators,
methods in general) only refer to the type of the instance struct via
tp_basicsize (where only the struct's size is held).

So my first attempt merely resets tp_basicsize to the size of my
alternative instance struct, as soon as the class is created. This
goes horribly wrong when instances of the class are deallocated, but I
can't see why.

Where else must I provide information about the instance struct type ?
[I will be away for a few days, from a few hours from now; please
don't take a long response time as a sign of rudeness.]

Hmm ... no takers ?

:-(

Appearently not :-).

Do you want to create those classes in Python, or from your extension
module? If you create the class in C its easy to add a hidden slot. You
can even add a field to the type object that contains the location of
that hidden slot.

Ronald

[Jacek Generowicz]

Jacek Generowicz <ja**************@cern.ch> writes:

Thomas Heller <th*****@python.net> writes:

Jacek Generowicz <ja**************@cern.ch> writes:
....
Can't you simply create a new, subclassable Python type with an
additonal void* slot, and derive your classes from it? Where's the
problem?

You mean statically create an extension type with the appropriate size
in the tp_basicsize, and dynamically create types which inherit from
it?

Sounds like it should work.

I'll give it a go, as I now see that my initial approach is guaranteed
to be wrong.

Given that type_new builds the contents of the tp_basicsize slot on
the basis of the supreclass's tp_basicsize, it seems that setting the
size in a superclass is the only way of getting some non-standard size
in there, without re-writing the whole of type_new.

So, I tried this approach. It works.

[Jacek Generowicz]

Ronald Oussoren <ro************@mac.com> writes:

Do you want to create those classes in Python, or from your extension
module?
Both actually.
If you create the class in C its easy to add a hidden slot.

If you create the class statically. I need to be able to create them
dynamically, by calling type.__new__.

Anyway, the way forward seems to be to create a base class with the
appropriate slot, and dynamically create subclasses of it, as Thomas
suggested. type_new adds some slots to whatever it finds in the
superclass' tp_basicsize.