C extension - new and init functions

[Paul Moore]

I have a C extension type, "Pattern", where my type manages a
dynamically allocated memory block (of "instructions", the details
aren't important).

The basic constructor builds a pattern with a single instruction
("End") as follows:

static PyObject *
Pattern_new(PyTypeObject *type, PyObject *args, PyObject *kwds)
{
Pattern *self;

self = (Pattern *)type->tp_alloc(type, 0);
self->prog = NULL;
return (PyObject *)self;
}

static int
Pattern_init(Pattern *self, PyObject *args, PyObject *kwds)
{
self->prog = newpatt(0);
if (self->prog == NULL)
return -1;

return 0;
}

This is fine, but in a lot of cases I want to allocate a Pattern with
a larger buffer (which will be filled in C before being passed back to
Python code). I want to allow the type to be subclassed, so I believe
that any allocation I do in C should be by calling the type
constructor - so my code to build instances should have the form (for
a Pattern classmethod):

static PyObject *
Pattern_XXX(PyObject *cls, PyObject *arg)
{
...

result = PyObject_CallFunction(cls, "");
if (result == NULL)
return NULL;

...
}

OK, but that allocates the initial buffer, which I will then throw
away. In practice, this isn't a significant issue (the overhead of one
allocation and one deallocation isn't going to wreck things!) but it
feels somehow clumsy. However, I can't see an obvious way to refactor
the code so that building the object and assigning the buffer are
separate - so that I can write a function like Pattern_init, which
allocates a differently sized buffer (well, I could, but if I did, I'd
have no way of using it that respects subclassing...

Can anybody offer me any suggestions on how to code this (or
confirmation that there's no way, and I should just live with the
unwanted allocation)?

Thanks,
Paul.

[Hrvoje Niksic]

Paul Moore <p.*******@gmail.comwrites:

OK, but that allocates the initial buffer, which I will then throw
away.

Why don't you use a constructor argument that allows specifying the
initial buffer size?

In practice, this isn't a significant issue (the overhead of one
allocation and one deallocation isn't going to wreck things!) but it
feels somehow clumsy. However, I can't see an obvious way to
refactor the code so that building the object and assigning the
buffer are separate - so that I can write a function like
Pattern_init, which allocates a differently sized buffer (well, I
could, but if I did, I'd have no way of using it that respects
subclassing...

Another way would be to avoid allocating the buffer at all until it is
actually needed.

[Paul Moore]

On 17 Nov, 09:40, Hrvoje Niksic <[e-mail]:

Paul *Moore <[e-mail]:

OK, but that allocates the initial buffer, which I will then
throw away.

Why don't you use a constructor argument that allows specifying
the initial buffer size?

That's an option, but I don't want to expose that argument to Python
code (as it would allow the user to create an object which wasn't
fully initialised - I only use the extra argument myself when there's
additional initialisation to do in C).

In practice, this isn't a significant issue (the overhead of
one allocation and one deallocation isn't going to wreck
things!) but it feels somehow clumsy. However, I can't see an
obvious way to refactor the code so that building the object
and assigning the buffer are separate - so that I can write a
function like Pattern_init, which allocates a differently
sized buffer (well, I could, but if I did, I'd have no way of
using it that respects subclassing...

Another way would be to avoid allocating the buffer at all
until it is actually needed.

The problem then is that I have to include null checks in a lot of
code that can otherwise assume that objects passed in have a valid
buffer.

Thinking it through, I think the extra dealloc/alloc and minor code
duplication probably isn't enough of an issue to compromise safety
over.

Thanks,
Paul.