473,728 Members | 1,955 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Re: python has memory leak?

On Tue, 22 Apr 2008 14:54:37 -0700 (PDT), yz****@gmail.co m wrote:
>Hi all,

I feel that my python script is leaking memory. And this is a test I
have:

[snip]
The test doesn't demonstrate any leaks. It does demonstrate that memory
usage can remain at or near peak memory usage even after the objects for
which that memory was allocated are no longer live in the process. This
is only a leak if peak memory goes up again each time you create any new
objects. Try repeated allocations of a large dictionary and observe how
memory usage rises and falls.

Python 2.5 does a somewhat better job of releasing memory when actual use
falls below peak, but this is a difficult thing to do perfectly.

Jean-Paul
Jun 27 '08 #1
1 2040
Thanks to Christian and Jean-Paul for your reply. I moved away from
the stackless version (which I don't understand what uniqueness it
comes with) and downloaded a fresh copy of Python 2.5.2 (r252:60911,
Feb 21 2008, 13:11:45) [MSC v.1310 32 bit (Intel)] on win32. Following
J-P's suggestion, I included the block to be tested in a loop:

i=20
while i>0:
i=i-1
test(log)
log.flush()

And logged the memory usage in bytes instead of MB. The result as
included below shows the peak memory usage keeps going up in the first
7 iterations. Can someone point to some docs or books about how python
manages memory or share some experience?

Thanks,

-GH
[10:27:54] test() starting ... memory usage: 4599808L 4599808L
[10:27:56] test() ... memory usage: 107499520L 107499520L
[10:27:56] test() done. memory usage: 41754624L 107503616L
[10:27:56] test() starting ... memory usage: 41754624L 107503616L
[10:27:59] test() ... memory usage: 108216320L 108400640L
[10:27:59] test() done. memory usage: 58884096L 108400640L
[10:27:59] test() starting ... memory usage: 58884096L 108400640L
[10:28:02] test() ... memory usage: 108220416L 108433408L
[10:28:02] test() done. memory usage: 42233856L 108433408L
[10:28:02] test() starting ... memory usage: 42233856L 108433408L
[10:28:04] test() ... memory usage: 108228608L 108441600L
[10:28:05] test() done. memory usage: 75149312L 108441600L
[10:28:05] test() starting ... memory usage: 75149312L 108441600L
[10:28:07] test() ... memory usage: 108498944L 108707840L
[10:28:07] test() done. memory usage: 41992192L 108707840L
[10:28:07] test() starting ... memory usage: 41992192L 108707840L
[10:28:10] test() ... memory usage: 108122112L 108707840L
[10:28:10] test() done. memory usage: 58523648L 108707840L
[10:28:10] test() starting ... memory usage: 58523648L 108707840L
[10:28:13] test() ... memory usage: 108642304L 108855296L
[10:28:13] test() done. memory usage: 41873408L 108855296L
[10:28:13] test() starting ... memory usage: 41873408L 108855296L
[10:28:16] test() ... memory usage: 108224512L 108855296L
[10:28:16] test() done. memory usage: 58626048L 108855296L
[10:28:16] test() starting ... memory usage: 58626048L 108855296L
[10:28:18] test() ... memory usage: 108224512L 108855296L
[10:28:19] test() done. memory usage: 41975808L 108855296L
[10:28:19] test() starting ... memory usage: 41975808L 108855296L
[10:28:21] test() ... memory usage: 108224512L 108855296L
[10:28:22] test() done. memory usage: 58626048L 108855296L
[10:28:22] test() starting ... memory usage: 58626048L 108855296L
[10:28:24] test() ... memory usage: 108224512L 108855296L
[10:28:24] test() done. memory usage: 41975808L 108855296L
[10:28:24] test() starting ... memory usage: 41975808L 108855296L
[10:28:27] test() ... memory usage: 108224512L 108855296L
[10:28:27] test() done. memory usage: 58626048L 108855296L
[10:28:27] test() starting ... memory usage: 58626048L 108855296L
[10:28:30] test() ... memory usage: 108224512L 108855296L
[10:28:30] test() done. memory usage: 41975808L 108855296L
[10:28:30] test() starting ... memory usage: 41975808L 108855296L
[10:28:32] test() ... memory usage: 108224512L 108855296L
[10:28:33] test() done. memory usage: 58626048L 108855296L
[10:28:33] test() starting ... memory usage: 58626048L 108855296L
[10:28:35] test() ... memory usage: 108224512L 108855296L
[10:28:36] test() done. memory usage: 41975808L 108855296L
[10:28:36] test() starting ... memory usage: 41975808L 108855296L
[10:28:38] test() ... memory usage: 108224512L 108855296L
[10:28:38] test() done. memory usage: 58626048L 108855296L
[10:28:38] test() starting ... memory usage: 58626048L 108855296L
[10:28:41] test() ... memory usage: 108224512L 108855296L
[10:28:41] test() done. memory usage: 41975808L 108855296L
[10:28:41] test() starting ... memory usage: 41975808L 108855296L
[10:28:44] test() ... memory usage: 108224512L 108855296L
[10:28:44] test() done. memory usage: 58626048L 108855296L
[10:28:44] test() starting ... memory usage: 58626048L 108855296L
[10:28:46] test() ... memory usage: 108224512L 108855296L
[10:28:47] test() done. memory usage: 41975808L 108855296L
[10:28:47] test() starting ... memory usage: 41975808L 108855296L
[10:28:49] test() ... memory usage: 108224512L 108855296L
[10:28:50] test() done. memory usage: 58626048L 108855296L
Jun 27 '08 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

53
4366
by: Michael Tobis | last post by:
Someone asked me to write a brief essay regarding the value-add proposition for Python in the Fortran community. Slightly modified to remove a few climatology-related specifics, here it is. I would welcome comments and corrections, and would be happy to contribute some version of this to the Python website if it is of interest. ===
10
5664
by: Andrew Trevorrow | last post by:
No response to my last message, so I'll try a different tack... Does anyone know of, or even better, has anyone here written a C++ application for Mac/Windows that allows users to run Python scripts from within the app? Not just once, but many times in a single session, and without leaking memory. Preferably an open source app so I can see how it's done. Our app (http://golly.sourceforge.net/) currently uses calls like these every...
18
2937
by: diffuser78 | last post by:
I have a python code which is running on a huge data set. After starting the program the computer becomes unstable and gets very diffucult to even open konsole to kill that process. What I am assuming is that I am running out of memory. What should I do to make sure that my code runs fine without becoming unstable. How should I address the memory leak problem if any ? I have a gig of RAM. Every help is appreciated.
0
361
by: Kurt B. Kaiser | last post by:
Patch / Bug Summary ___________________ Patches : 378 open ( +3) / 3298 closed (+34) / 3676 total (+37) Bugs : 886 open (-24) / 5926 closed (+75) / 6812 total (+51) RFE : 224 open ( +7) / 227 closed ( +7) / 451 total (+14) New / Reopened Patches ______________________
0
1671
by: nejucomo | last post by:
Hi folks, Quick Synopsis: A test script demonstrates a memory leak when I use pythonic extensions of my builtin types, but if I use the builtin types themselves there is no memory leak. If you are interested in how builtin/pure-python inheritance interacts
5
3345
by: vishnu | last post by:
Hi there, I am embedding python 2.5 on embedded system running on RTOS where I had strict memory constraints. As python is a huge malloc intensive application, I observed huge memory fragmentation in my system which is leading to out of memory after running few scripts. So I decided to re-initialise the python with out restarting the whole python. I tried to use Py_Finalise() after completion of each script , then call Py_Initialise as...
1
3258
by: cody314 | last post by:
Versions: Python 2.5.1 (r251:54863, May 14 2007, 10:50:04) SWIG Version 1.3.20 Hello I have some code that wraps a C++ library so I may use it in python. The code basically just gets some data and puts it in the PyArrayObject* which is returned as a PyObject*. I then call it from python like so: self.c = __f2.fdct2_wrapper(x,self.nbs,self.nba,self.ac)
0
1295
by: Gabriel Genellina | last post by:
En Wed, 16 Apr 2008 13:10:42 -0300, rkmr.em@gmail.com <rkmr.em@gmail.comescribió: In principle pure Python code should have no memory leaks - if there are, they're on Python itself. Maybe this is an allocation problem, not an actual leak; if you can write a program that reproduces the problem, post a bug at http://bugs.python.org -- Gabriel Genellina
4
184
by: Christian Heimes | last post by:
Gabriel Genellina schrieb: Pure Python code can cause memory leaks. No, that's not a bug in the interpreter but the fault of the developer. For example code that messes around with stack frames and exception object can cause nasty reference leaks. Christian
0
8759
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9424
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
1
9198
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9135
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8133
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
4524
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
3236
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2668
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
2161
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.