473,246 Members | 1,383 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,246 software developers and data experts.

Python Memory Leak using SWIG

Versions:
Python 2.5.1 (r251:54863, May 14 2007, 10:50:04)
SWIG Version 1.3.20

Hello I have some code that wraps a C++ library so I may use it in
python. The code basically just gets some data and puts it in the
PyArrayObject* which is returned as a PyObject*.
I then call it from python like so:
self.c = __f2.fdct2_wrapper(x,self.nbs,self.nba,self.ac)

I then loop (which pretty much only calls this function) over and
over. I put the variable as a self.c hoping the garbage collector
would know how to delete it after the class goes out of scope. I also
tried explicitly deleting the variable (del var) in the loop with no
success. In all cases quiet a large memory leak occurs (and grows
rather quickly).

I believe this is comming from the fact that the thing that is
returned is a pointer to the data. So the returning object is a
pointer. The python garbage collector then doesn't know how to delete
this structure and probably (maybe) just deletes the pointer after the
class goes out of scope. Leave the data there and causing the memory
leak issue. I however doesn't know how to tell python that this
variable is a pointer and to delete whats going to it. Or perhaps
tell SWIG to delete the data, and return the structure some other way?

Here is the c++ wrapping code, perhaps there is an easy way to fix
this memory leak (I believe a lot of SWIG people probably do this)
perhaps some function call from the python? or some special call from
the SWIG? Thanks a bunch!

// set up the list output
PyListObject* out;
PyArrayObject* output;
out = (PyListObject*) PyList_New(0);
npy_intp dims[2];
int i,j;

for(i=0;i<g.size();i++)
{
// append a list for this scale
PyList_Append((PyObject*) out,PyList_New(0));

for(j=0;j<g[i].size();j++)
{
// set the dimensions for this scale & angle
dims[0] = g[i][j].n();
dims[1] = g[i][j].m();

// make an array for this scale & angle's data
output = (PyArrayObject*) PyArray_SimpleNewFromData(2, dims,
PyArray_CDOUBLE,g[i][j]._data);
Py_INCREF((PyObject*) output);

// append this angle's data to the list for this scale
PyList_Append(PyList_GetItem((PyObject*) out,i),(PyObject*)
output);

// zero the CpxNumMat for this scale & angle, hand ownership to
numpy
g[i][j]._data = NULL;
g[i][j]._m = 0;
g[i][j]._n = 0;
output->flags = output->flags | NPY_OWNDATA;
}
}

return (PyObject*) out;

Jun 4 '07 #1
1 3216
co*****@gmail.com wrote:
Versions:
Python 2.5.1 (r251:54863, May 14 2007, 10:50:04)
SWIG Version 1.3.20

Hello I have some code that wraps a C++ library so I may use it in
python. The code basically just gets some data and puts it in the
PyArrayObject* which is returned as a PyObject*.
I then call it from python like so:
self.c = __f2.fdct2_wrapper(x,self.nbs,self.nba,self.ac)

I then loop (which pretty much only calls this function) over and
over. I put the variable as a self.c hoping the garbage collector
would know how to delete it after the class goes out of scope. I also
tried explicitly deleting the variable (del var) in the loop with no
success. In all cases quiet a large memory leak occurs (and grows
rather quickly).

I believe this is comming from the fact that the thing that is
returned is a pointer to the data. So the returning object is a
pointer. The python garbage collector then doesn't know how to delete
this structure and probably (maybe) just deletes the pointer after the
class goes out of scope. Leave the data there and causing the memory
leak issue. I however doesn't know how to tell python that this
variable is a pointer and to delete whats going to it. Or perhaps
tell SWIG to delete the data, and return the structure some other way?

Here is the c++ wrapping code, perhaps there is an easy way to fix
this memory leak (I believe a lot of SWIG people probably do this)
perhaps some function call from the python? or some special call from
the SWIG? Thanks a bunch!

// set up the list output
PyListObject* out;
PyArrayObject* output;
out = (PyListObject*) PyList_New(0);
npy_intp dims[2];
int i,j;

for(i=0;i<g.size();i++)
{
// append a list for this scale
PyList_Append((PyObject*) out,PyList_New(0));

for(j=0;j<g[i].size();j++)
{
// set the dimensions for this scale & angle
dims[0] = g[i][j].n();
dims[1] = g[i][j].m();

// make an array for this scale & angle's data
output = (PyArrayObject*) PyArray_SimpleNewFromData(2, dims,
PyArray_CDOUBLE,g[i][j]._data);
Py_INCREF((PyObject*) output);

// append this angle's data to the list for this scale
PyList_Append(PyList_GetItem((PyObject*) out,i),(PyObject*)
output);

// zero the CpxNumMat for this scale & angle, hand ownership to
numpy
g[i][j]._data = NULL;
g[i][j]._m = 0;
g[i][j]._n = 0;
output->flags = output->flags | NPY_OWNDATA;
}
}

return (PyObject*) out;
I think %newobject swig directive is a solution to your problem.

Pierre
Jun 5 '07 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Gary | last post by:
Hi- I've been searching the web for a while and I've been unable to find a way to access c data objects in python without using SWIG. I can do methods just fine but I can't access variables. ...
0
by: Atul Kshirsagar | last post by:
I am embedding python in my C++ application. I am using Python *2.3.2* with a C++ extention DLL in multi-threaded environment. I am using SWIG-1.3.19 to generate C++ to Python interface. Now to...
13
by: Roy Smith | last post by:
I've got a C library with about 50 calls in it that I want to wrap in Python. I know I could use some tool like SWIG, but that will give me a too-literal translation; I want to make some...
2
by: ajikoe | last post by:
Hi, I tried to follow the example in swig homepage. I found error which I don't understand. I use bcc32, I already include directory where my python.h exist in bcc32.cfg. /* File : example.c...
0
by: Uwe Mayer | last post by:
Hi, I am wrapping a C function returning large amount of binary data back to Python using SWIG. I have the data malloc()ed or new()ed on the heap and buffer objects seem a good way to wrap it...
18
by: diffuser78 | last post by:
I have a python code which is running on a huge data set. After starting the program the computer becomes unstable and gets very diffucult to even open konsole to kill that process. What I am...
0
by: Kurt B. Kaiser | last post by:
Patch / Bug Summary ___________________ Patches : 378 open ( +3) / 3298 closed (+34) / 3676 total (+37) Bugs : 886 open (-24) / 5926 closed (+75) / 6812 total (+51) RFE : 224 open...
0
by: nejucomo | last post by:
Hi folks, Quick Synopsis: A test script demonstrates a memory leak when I use pythonic extensions of my builtin types, but if I use the builtin types themselves there is no memory leak. ...
113
by: John Nagle | last post by:
The major complaint I have about Python is that the packages which connect it to other software components all seem to have serious problems. As long as you don't need to talk to anything outside...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.