Python Memory Leak using SWIG

[cody314]

Versions:
Python 2.5.1 (r251:54863, May 14 2007, 10:50:04)
SWIG Version 1.3.20

Hello I have some code that wraps a C++ library so I may use it in
python. The code basically just gets some data and puts it in the
PyArrayObject* which is returned as a PyObject*.
I then call it from python like so:
self.c = __f2.fdct2_wrap per(x,self.nbs, self.nba,self.a c)

I then loop (which pretty much only calls this function) over and
over. I put the variable as a self.c hoping the garbage collector
would know how to delete it after the class goes out of scope. I also
tried explicitly deleting the variable (del var) in the loop with no
success. In all cases quiet a large memory leak occurs (and grows
rather quickly).

I believe this is comming from the fact that the thing that is
returned is a pointer to the data. So the returning object is a
pointer. The python garbage collector then doesn't know how to delete
this structure and probably (maybe) just deletes the pointer after the
class goes out of scope. Leave the data there and causing the memory
leak issue. I however doesn't know how to tell python that this
variable is a pointer and to delete whats going to it. Or perhaps
tell SWIG to delete the data, and return the structure some other way?

Here is the c++ wrapping code, perhaps there is an easy way to fix
this memory leak (I believe a lot of SWIG people probably do this)
perhaps some function call from the python? or some special call from
the SWIG? Thanks a bunch!

// set up the list output
PyListObject* out;
PyArrayObject* output;
out = (PyListObject*) PyList_New(0);
npy_intp dims[2];
int i,j;

for(i=0;i<g.siz e();i++)
{
// append a list for this scale
PyList_Append(( PyObject*) out,PyList_New( 0));

for(j=0;j<g[i].size();j++)
{
// set the dimensions for this scale & angle
dims[0] = g[i][j].n();
dims[1] = g[i][j].m();

// make an array for this scale & angle's data
output = (PyArrayObject* ) PyArray_SimpleN ewFromData(2, dims,
PyArray_CDOUBLE ,g[i][j]._data);
Py_INCREF((PyOb ject*) output);

// append this angle's data to the list for this scale
PyList_Append(P yList_GetItem(( PyObject*) out,i),(PyObjec t*)
output);

// zero the CpxNumMat for this scale & angle, hand ownership to
numpy
g[i][j]._data = NULL;
g[i][j]._m = 0;
g[i][j]._n = 0;
output->flags = output->flags | NPY_OWNDATA;
}
}

return (PyObject*) out;

[Pierre Sangouard]

co*****@gmail.c om wrote:

Versions:
Python 2.5.1 (r251:54863, May 14 2007, 10:50:04)
SWIG Version 1.3.20

Hello I have some code that wraps a C++ library so I may use it in
python. The code basically just gets some data and puts it in the
PyArrayObject* which is returned as a PyObject*.
I then call it from python like so:
self.c = __f2.fdct2_wrap per(x,self.nbs, self.nba,self.a c)

I then loop (which pretty much only calls this function) over and
over. I put the variable as a self.c hoping the garbage collector
would know how to delete it after the class goes out of scope. I also
tried explicitly deleting the variable (del var) in the loop with no
success. In all cases quiet a large memory leak occurs (and grows
rather quickly).

I believe this is comming from the fact that the thing that is
returned is a pointer to the data. So the returning object is a
pointer. The python garbage collector then doesn't know how to delete
this structure and probably (maybe) just deletes the pointer after the
class goes out of scope. Leave the data there and causing the memory
leak issue. I however doesn't know how to tell python that this
variable is a pointer and to delete whats going to it. Or perhaps
tell SWIG to delete the data, and return the structure some other way?

Here is the c++ wrapping code, perhaps there is an easy way to fix
this memory leak (I believe a lot of SWIG people probably do this)
perhaps some function call from the python? or some special call from
the SWIG? Thanks a bunch!

// set up the list output
PyListObject* out;
PyArrayObject* output;
out = (PyListObject*) PyList_New(0);
npy_intp dims[2];
int i,j;

for(i=0;i<g.siz e();i++)
{
// append a list for this scale
PyList_Append(( PyObject*) out,PyList_New( 0));

for(j=0;j<g[i].size();j++)
{
// set the dimensions for this scale & angle
dims[0] = g[i][j].n();
dims[1] = g[i][j].m();

// make an array for this scale & angle's data
output = (PyArrayObject* ) PyArray_SimpleN ewFromData(2, dims,
PyArray_CDOUBLE ,g[i][j]._data);
Py_INCREF((PyOb ject*) output);

// append this angle's data to the list for this scale
PyList_Append(P yList_GetItem(( PyObject*) out,i),(PyObjec t*)
output);

// zero the CpxNumMat for this scale & angle, hand ownership to
numpy
g[i][j]._data = NULL;
g[i][j]._m = 0;
g[i][j]._n = 0;
output->flags = output->flags | NPY_OWNDATA;
}
}

return (PyObject*) out;

I think %newobject swig directive is a solution to your problem.

Pierre