Dear list,
I spent the last 12 hours in catching this bug (?) and what I found out
is very difficult to explain:
Basically, I need to call a user-provided function many times, with a
tuple as parameter.
C/C++ side: (class A, constructed using a python function m_func)
// create in the constructor, used to hold the parameter
m_numArray = PyTuple_New(m_l en);
...
// member function, will be called many times
void fun(){
....
// set the tuple
for( j=0; j<m_len; ++j)
PyTuple_SetItem (m_numArray, j, PyInt_FromLong( some_value));
// call function m_func(m_numArr ay)
PyObject* arglist = Py_BuildValue(" (O)", m_numArray );
PyObject* result = PyEval_CallObje ct(m_func, arglist);
Py_DECREF(argli st);
// check if result == NULL ...
// get value from result, ...
Py_DECREF(resul t)
return the value of result
}
Python side: pyFun1, pyFun2 will be passed to C/C++ as m_func
def pyFun1(val):
return 0.5
def pyFun2(val):
return 0.5*val[0]
For objects A(pyFun1) and A(pyFun2), member function fun() will be
called many times. The weird thing is that calling A(pyFun1).fun()
endlessly goes well with constant memory consumption; while calling
A(pyFun2).fun() will use more and more memory until core dump. I tried
"valgrind --tool=memcheck --check-leak=yes" but did not find any sign of
memory leak.
The real code is much more complicated and I can not possibly explain
the details here. Can anyone *guess* what might go wrong? My guess is
that access to val[0] increases the ref_cnt of val[0] and make its value
stay in memory, but this is unlikely the case.
Many thanks in advance.
Bo 4 1403
Bo Peng wrote: The real code is much more complicated and I can not possibly explain the details here. Can anyone *guess* what might go wrong?
Sorry, are you saying that the code you posted does NOT
have a memory leak, but you want us to find the memory
leak in your real code sight unseen?
--
Steven.
Steven D'Aprano wrote: Sorry, are you saying that the code you posted does NOT have a memory leak, but you want us to find the memory leak in your real code sight unseen?
valgrind does not detect anything so it does not look like memory leak.
I just can not figure out why val[0], readonly access to a tuple
element, can cause this kind of problem.
Anyway, is there a way to list all or newly added python objects? I
guess I can trace the ref_cnt of the tuple element manually.
Bo
Bo Peng wrote: Sorry, are you saying that the code you posted does NOT have a memory leak, but you want us to find the memory leak in your real code sight unseen?
Problem found. It is hidden in a utility function that converts the
return value to a double. The refcnt of the middle result was not
decreased.
PyObject* res = PyNumber_Float( obj);
val = PyFloat_AsDoubl e(res);
// missing
Py_DECREF(res);
The problem is not that difficult to find, but it was 2am in the morning
and I was misled by the different behavior of pyFun1 and pyFun2.
Thanks,
Bo
In message <do**********@j oe.rice.edu>, Bo Peng <bp***@rice.edu > writes The problem is not that difficult to find, but it was 2am in the morning and I was misled by the different behavior of pyFun1 and pyFun2.
Don't know if you were using Windows, but if you were then Python Memory
Validator would have helped you identify that bug. Its in beta at the
moment if you want to try it. http://www.softwareverify.com
Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk/software.html
Computer Consultancy, Software Development
Windows C++, Java, Assembler, Performance Analysis, Troubleshooting This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: jong |
last post by:
I have a problem with an C# application. The application is a 24x7 low volume message processing server. The server has a single thread of processing, running in a continuous loop, for each iteration thread sleeping for 5 seconds and than reading textual message files from a Windows Folder and applying them to a database.
From an external view, the server appears to have a memory-leek, continuously consuming RAM in a near linier fashion as...
|
by: Matthias S. |
last post by:
I'm using C#Express Beta and the framework 2.0.
I recently realized that my application, when starting has a workingset
peak of ~128 MegaB. I startet wondering a bit because the application
didn't yet do anything really usefull.
So I fired up the IDE, created an empty WinForms project and compiled a
release build. Please note that I have not added any extra code! Here is
what I have figured out:
|
by: Andy |
last post by:
Along with many others I've noticed the large amount of memory that
can be taken up by the aspnet_wp.exe. I've found that I can better
control and limit this memory consumption by including a GC.Collect()
in the Application_EndRequest() event handler in the Global.asax file.
Whilst this appears to help my memory consumption issues I've also
read that forced GC.Collect() can be inefficient.
Assuming that I don't see any adverse effects...
|
by: anandav2001 |
last post by:
Hello developers,
I have created an executable(system tray application) in VS.net 2003
using VB.net. My app was taking 30 MB memory(since some web services
call are there which happens for each 10 sec checking internet is
available or not). Inorder to reduce huge memory consumption, what i
used is
Public Class MemoryManagement
Private Declare Function SetProcessWorkingSetSize Lib
|
by: Salvador |
last post by:
Hi,
I am using WMI to gather information about different computers (using win2K
and win 2K3), checking common classes and also WMI load balance. My
application runs every 1 minute and reports the status of the machines.
Upon we follow the .NET object lifetime recommendations the application is
constantly consuming more memory! The problem is on the
ManagementObjectSearch, upon we Dispose the object it seems that is not
releasing the...
| |
by: jedi200581 |
last post by:
Hi,
I'm new at python as I just started to learn it, but I found out
something weird. I have wrote a little program to compute Mersenne
number:
# Snipet on
def is_prime n:
for i in range(2, n):
if (n % i) == 0:
|
by: Bruno Barberi Gnecco |
last post by:
I'm using PHP to run a CLI application. It's a script run by cron that
parses some HTML files (with DOM XML), and I ended up using PHP to integrate with
the rest of the code that already runs the website.
The problem is: it's eating more memory than a black hole. It eats the
current limit of 256MB set in php.ini, in an application that would hardly
consume 4MB if written in C. I don't care if this application takes much longer
to run...
|
by: Jonas Maurus |
last post by:
Hello everybody,
I'm pondering the following problem:
I want to write a Python program that receives messages via SMTP and
stores them in a dict or an array. For my purposes it would be
important that all received mail would be kept in RAM and not cached
out to disk. If a new message comes in that can't fit in the allocated
memory, a number of old messages would be discarded.
|
by: Cesar |
last post by:
Hello people. I'm having a Winform app that contains a webbrowser control
that keeps navigating from one page to another permanentrly to make some
tests. The problem I'm having is that after a while, the application is
using more than 100 or 150 Mb in RAM, and if I let it continue, it can leave
the system without memory. I've been watching in some pages that other
people has the same problem with this control when keep navigating for a...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
| |
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
| |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |