Dear list,
I spent the last 12 hours in catching this bug (?) and what I found out
is very difficult to explain:
Basically, I need to call a user-provided function many times, with a
tuple as parameter.
C/C++ side: (class A, constructed using a python function m_func)
// create in the constructor, used to hold the parameter
m_numArray = PyTuple_New(m_len);
...
// member function, will be called many times
void fun(){
....
// set the tuple
for( j=0; j<m_len; ++j)
PyTuple_SetItem(m_numArray, j, PyInt_FromLong(some_value));
// call function m_func(m_numArray)
PyObject* arglist = Py_BuildValue("(O)", m_numArray );
PyObject* result = PyEval_CallObject(m_func, arglist);
Py_DECREF(arglist);
// check if result == NULL ...
// get value from result, ...
Py_DECREF(result)
return the value of result
}
Python side: pyFun1, pyFun2 will be passed to C/C++ as m_func
def pyFun1(val):
return 0.5
def pyFun2(val):
return 0.5*val[0]
For objects A(pyFun1) and A(pyFun2), member function fun() will be
called many times. The weird thing is that calling A(pyFun1).fun()
endlessly goes well with constant memory consumption; while calling
A(pyFun2).fun() will use more and more memory until core dump. I tried
"valgrind --tool=memcheck --check-leak=yes" but did not find any sign of
memory leak.
The real code is much more complicated and I can not possibly explain
the details here. Can anyone *guess* what might go wrong? My guess is
that access to val[0] increases the ref_cnt of val[0] and make its value
stay in memory, but this is unlikely the case.
Many thanks in advance.
Bo 4 1392
Bo Peng wrote: The real code is much more complicated and I can not possibly explain the details here. Can anyone *guess* what might go wrong?
Sorry, are you saying that the code you posted does NOT
have a memory leak, but you want us to find the memory
leak in your real code sight unseen?
--
Steven.
Steven D'Aprano wrote: Sorry, are you saying that the code you posted does NOT have a memory leak, but you want us to find the memory leak in your real code sight unseen?
valgrind does not detect anything so it does not look like memory leak.
I just can not figure out why val[0], readonly access to a tuple
element, can cause this kind of problem.
Anyway, is there a way to list all or newly added python objects? I
guess I can trace the ref_cnt of the tuple element manually.
Bo
Bo Peng wrote: Sorry, are you saying that the code you posted does NOT have a memory leak, but you want us to find the memory leak in your real code sight unseen?
Problem found. It is hidden in a utility function that converts the
return value to a double. The refcnt of the middle result was not
decreased.
PyObject* res = PyNumber_Float(obj);
val = PyFloat_AsDouble(res);
// missing
Py_DECREF(res);
The problem is not that difficult to find, but it was 2am in the morning
and I was misled by the different behavior of pyFun1 and pyFun2.
Thanks,
Bo
In message <do**********@joe.rice.edu>, Bo Peng <bp***@rice.edu> writes The problem is not that difficult to find, but it was 2am in the morning and I was misled by the different behavior of pyFun1 and pyFun2.
Don't know if you were using Windows, but if you were then Python Memory
Validator would have helped you identify that bug. Its in beta at the
moment if you want to try it. http://www.softwareverify.com
Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk/software.html
Computer Consultancy, Software Development
Windows C++, Java, Assembler, Performance Analysis, Troubleshooting This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: jong |
last post by:
I have a problem with an C# application. The application is a 24x7 low volume message processing server. The server has a single thread of processing, running in a continuous loop, for each...
|
by: Matthias S. |
last post by:
I'm using C#Express Beta and the framework 2.0.
I recently realized that my application, when starting has a workingset
peak of ~128 MegaB. I startet wondering a bit because the application...
|
by: Andy |
last post by:
Along with many others I've noticed the large amount of memory that
can be taken up by the aspnet_wp.exe. I've found that I can better
control and limit this memory consumption by including a...
|
by: anandav2001 |
last post by:
Hello developers,
I have created an executable(system tray application) in VS.net 2003
using VB.net. My app was taking 30 MB memory(since some web services
call are there which happens for each...
|
by: Salvador |
last post by:
Hi,
I am using WMI to gather information about different computers (using win2K
and win 2K3), checking common classes and also WMI load balance. My
application runs every 1 minute and reports...
|
by: jedi200581 |
last post by:
Hi,
I'm new at python as I just started to learn it, but I found out
something weird. I have wrote a little program to compute Mersenne
number:
# Snipet on
def is_prime n:
for i in range(2,...
|
by: Bruno Barberi Gnecco |
last post by:
I'm using PHP to run a CLI application. It's a script run by cron that
parses some HTML files (with DOM XML), and I ended up using PHP to integrate with
the rest of the code that already runs the...
|
by: Jonas Maurus |
last post by:
Hello everybody,
I'm pondering the following problem:
I want to write a Python program that receives messages via SMTP and
stores them in a dict or an array. For my purposes it would be...
|
by: Cesar |
last post by:
Hello people. I'm having a Winform app that contains a webbrowser control
that keeps navigating from one page to another permanentrly to make some
tests. The problem I'm having is that after a...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
| |