problem with Py_BuildValue

[Christian Meesters]

Hi,

currently I have a problem understanding Py_BuildValue. I have this code:

static PyObject *function(PyObject *self, PyObject *args) {
PyObject * python_return_value = NULL;
PyObject * dummy = NULL;
double * internal_list;
<snip and forget the rest>

/* converting to python representation */
for (i=0; i < limit; i++) {
dummy = Py_BuildValue("d", internal_list[i]);
if (!dummy) return NULL;
PyList_Append(python_return_value, dummy);
Py_DECREF(dummy); dummy = NULL;
}
return python_return_value
}

This doesn't work. What I see, when invoking the function "function()" in
Python is a list of refcounts, like: [<refcnt 0 at 0x94a29d4>, <refcnt 0 at
0x94a29e4>, ...]. However, if I change the Py_BuildValue-line to be
dummy = Py_BuildValue("i", (int)internal_list[i]);
I do get the 'right' integer return values. Point is that I really would
like to work with Python-floats afterwards.

Any idea where a pitfall might be here?

TIA
Christian

PS Oh, and I tried casting to float and explicitly to double, too. Same
result as without the casts.

[=?iso-8859-1?q?C=E9dric_Lucantis?=]

Hi,

Hi,

currently I have a problem understanding Py_BuildValue. I have this code:

static PyObject *function(PyObject *self, PyObject *args) {
PyObject * python_return_value = NULL;
PyObject * dummy = NULL;
double * internal_list;
<snip and forget the rest>

/* converting to python representation */
for (i=0; i < limit; i++) {
dummy = Py_BuildValue("d", internal_list[i]);
if (!dummy) return NULL;
PyList_Append(python_return_value, dummy);
Py_DECREF(dummy); dummy = NULL;
}
return python_return_value
}

This doesn't work. What I see, when invoking the function "function()" in
Python is a list of refcounts, like: [<refcnt 0 at 0x94a29d4>, <refcnt 0 at
0x94a29e4>, ...]. However, if I change the Py_BuildValue-line to be
dummy = Py_BuildValue("i", (int)internal_list[i]);
I do get the 'right' integer return values. Point is that I really would
like to work with Python-floats afterwards.

Any idea where a pitfall might be here?

I see nothing wrong with your code so I'd say it is somewhere else (did you
snip any code between the end of the loop and the return?). I've never seen
those 'refcnt' objects but a refcount of 0 sounds like you unrefed your
objects one extra time by mistake. This would produce a segfault on unix, but
maybe not on all platforms ? You should check the return value of
PyList_Append() and if it doesn't help trace the content of your list after
each iteration to see when the bad things happen (you can check the reference
count of an object with obj->ob_refcnt).

Finally note that in your case it would be much simpler and more efficient to
use the float constructor directly:

dummy = PyFloat_FromDouble(internal_list([i]))

PS: always use Py_CLEAR(dummy) instead of Py_DECREF(dummy); dummy=NULL;
(though it doesn't really matter in this simple case - see
http://docs.python.org/api/countingRefs.html)

--
Cédric Lucantis

[Christian Meesters]

Thank you. At least I can exclude another few error sources, now.

Cédric Lucantis wrote:

I see nothing wrong with your code so I'd say it is somewhere else (did
you snip any code between the end of the loop and the return?).

No. (Apart from freeing allocated memory.)

I've never
seen those 'refcnt' objects but a refcount of 0 sounds like you unrefed
your objects one extra time by mistake. This would produce a segfault on
unix, but maybe not on all platforms ?

Well, I am working on Linux. Python 2.5.1, gcc 4.1.3 . And I do not see
segfaults until I start working in Python with the return value of that
function, of course.

You should check the return value
of PyList_Append()

It is always 0, regardless of what I do.

and if it doesn't help trace the content of your list
after each iteration to see when the bad things happen (you can check the
reference count of an object with obj->ob_refcnt).

Seems ok. What I did to check this was placing this after building the list:

for (i=0; i < limit; i++) {
dummy = PyList_GetItem(python_return_value, i);
printf("%f\n", PyFloat_AsDouble(dummy));
Py_CLEAR(dummy);
}

Which gives reasonable numbers.

>
Finally note that in your case it would be much simpler and more efficient
to use the float constructor directly:

dummy = PyFloat_FromDouble(internal_list([i]))

I tried that (actually PyFloat_FromDouble(internal_list[i]) ): Same thing,
but now more like [<refcnt -1 at 0x94a474c>, <refcnt -1 at 0x94a475c>, etc.
(Note the -1.)

>
PS: always use Py_CLEAR(dummy) instead of Py_DECREF(dummy); dummy=NULL;
(though it doesn't really matter in this simple case - see
http://docs.python.org/api/countingRefs.html)

Good idea! Since I require 2.4 for users anyway, there is no harm in
reducing the code.

Christian

[Christian Meesters]

Thank you so much - I was such an idiot (see below).

>>I see nothing wrong with your code so I'd say it is somewhere else (did
you snip any code between the end of the loop and the return?).

>>No. (Apart from freeing allocated memory.)

I'm pretty sure we'll find something interesting here :)

Still not. I was about to prove it and already uploaded the file, when I
saw, what was really going wrong ...

>
PyList_GetItem returns a borrowed reference so you shoud _not_ unref it
(this explains the refcnt -1 I think)

This is THE crucial point. If I just delete the Py_CLEAR-line, everything is
working smoothly and calling PyFloat_FromDouble-is working too.

Again: Thanks a lot. 'Reference counting' won't become one of my
hobbies ;-).

Best,
Christian