471,330 Members | 1,783 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,330 software developers and data experts.

Python Memory Leak using SWIG

Versions:
Python 2.5.1 (r251:54863, May 14 2007, 10:50:04)
SWIG Version 1.3.20

Hello I have some code that wraps a C++ library so I may use it in
python. The code basically just gets some data and puts it in the
PyArrayObject* which is returned as a PyObject*.
I then call it from python like so:
self.c = __f2.fdct2_wrapper(x,self.nbs,self.nba,self.ac)

I then loop (which pretty much only calls this function) over and
over. I put the variable as a self.c hoping the garbage collector
would know how to delete it after the class goes out of scope. I also
tried explicitly deleting the variable (del var) in the loop with no
success. In all cases quiet a large memory leak occurs (and grows
rather quickly).

I believe this is comming from the fact that the thing that is
returned is a pointer to the data. So the returning object is a
pointer. The python garbage collector then doesn't know how to delete
this structure and probably (maybe) just deletes the pointer after the
class goes out of scope. Leave the data there and causing the memory
leak issue. I however doesn't know how to tell python that this
variable is a pointer and to delete whats going to it. Or perhaps
tell SWIG to delete the data, and return the structure some other way?

Here is the c++ wrapping code, perhaps there is an easy way to fix
this memory leak (I believe a lot of SWIG people probably do this)
perhaps some function call from the python? or some special call from
the SWIG? Thanks a bunch!

// set up the list output
PyListObject* out;
PyArrayObject* output;
out = (PyListObject*) PyList_New(0);
npy_intp dims[2];
int i,j;

for(i=0;i<g.size();i++)
{
// append a list for this scale
PyList_Append((PyObject*) out,PyList_New(0));

for(j=0;j<g[i].size();j++)
{
// set the dimensions for this scale & angle
dims[0] = g[i][j].n();
dims[1] = g[i][j].m();

// make an array for this scale & angle's data
output = (PyArrayObject*) PyArray_SimpleNewFromData(2, dims,
PyArray_CDOUBLE,g[i][j]._data);
Py_INCREF((PyObject*) output);

// append this angle's data to the list for this scale
PyList_Append(PyList_GetItem((PyObject*) out,i),(PyObject*)
output);

// zero the CpxNumMat for this scale & angle, hand ownership to
numpy
g[i][j]._data = NULL;
g[i][j]._m = 0;
g[i][j]._n = 0;
output->flags = output->flags | NPY_OWNDATA;
}
}

return (PyObject*) out;

Jun 4 '07 #1
1 3032
co*****@gmail.com wrote:
Versions:
Python 2.5.1 (r251:54863, May 14 2007, 10:50:04)
SWIG Version 1.3.20

Hello I have some code that wraps a C++ library so I may use it in
python. The code basically just gets some data and puts it in the
PyArrayObject* which is returned as a PyObject*.
I then call it from python like so:
self.c = __f2.fdct2_wrapper(x,self.nbs,self.nba,self.ac)

I then loop (which pretty much only calls this function) over and
over. I put the variable as a self.c hoping the garbage collector
would know how to delete it after the class goes out of scope. I also
tried explicitly deleting the variable (del var) in the loop with no
success. In all cases quiet a large memory leak occurs (and grows
rather quickly).

I believe this is comming from the fact that the thing that is
returned is a pointer to the data. So the returning object is a
pointer. The python garbage collector then doesn't know how to delete
this structure and probably (maybe) just deletes the pointer after the
class goes out of scope. Leave the data there and causing the memory
leak issue. I however doesn't know how to tell python that this
variable is a pointer and to delete whats going to it. Or perhaps
tell SWIG to delete the data, and return the structure some other way?

Here is the c++ wrapping code, perhaps there is an easy way to fix
this memory leak (I believe a lot of SWIG people probably do this)
perhaps some function call from the python? or some special call from
the SWIG? Thanks a bunch!

// set up the list output
PyListObject* out;
PyArrayObject* output;
out = (PyListObject*) PyList_New(0);
npy_intp dims[2];
int i,j;

for(i=0;i<g.size();i++)
{
// append a list for this scale
PyList_Append((PyObject*) out,PyList_New(0));

for(j=0;j<g[i].size();j++)
{
// set the dimensions for this scale & angle
dims[0] = g[i][j].n();
dims[1] = g[i][j].m();

// make an array for this scale & angle's data
output = (PyArrayObject*) PyArray_SimpleNewFromData(2, dims,
PyArray_CDOUBLE,g[i][j]._data);
Py_INCREF((PyObject*) output);

// append this angle's data to the list for this scale
PyList_Append(PyList_GetItem((PyObject*) out,i),(PyObject*)
output);

// zero the CpxNumMat for this scale & angle, hand ownership to
numpy
g[i][j]._data = NULL;
g[i][j]._m = 0;
g[i][j]._n = 0;
output->flags = output->flags | NPY_OWNDATA;
}
}

return (PyObject*) out;
I think %newobject swig directive is a solution to your problem.

Pierre
Jun 5 '07 #2

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

5 posts views Thread by Gary | last post: by
13 posts views Thread by Roy Smith | last post: by
2 posts views Thread by ajikoe | last post: by
reply views Thread by Uwe Mayer | last post: by
18 posts views Thread by diffuser78 | last post: by
reply views Thread by Kurt B. Kaiser | last post: by
113 posts views Thread by John Nagle | last post: by
reply views Thread by rosydwin | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.