473,890 Members | 1,218 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Memory Management in python 2.5

Hi, I am starting to have a look to a python program that does not free
memory (I am using python 2.4.3). As I have read about a new memory
management in python 2.5 (http://evanjones.ca/python-memory.html) I
decided to try the program with the new version.
With the new version of python the memory consumption is the same. Now
I am asking myself if python 2.5 has any improving in memory
management or maybe not yet. Thank you.

-- Cesar

Oct 9 '06 #1
4 1822
ce*********@gma il.com skrev:
Hi, I am starting to have a look to a python program that does not free
memory (I am using python 2.4.3). As I have read about a new memory
management in python 2.5 (http://evanjones.ca/python-memory.html) I
decided to try the program with the new version.
With the new version of python the memory consumption is the same. Now
I am asking myself if python 2.5 has any improving in memory
management or maybe not yet. Thank you.
In previous versions Python collected memory, but never released it
again. It simply kept on to it, so it could reuse it again later.

From 2.5 onwards it should release most of the unused memory. However
it doesn't use less memory. The peak memory usage should be the same as
before. So for one-off programs that starts up and runs once, there
should not be much gain.
--

hilsen/regards Max M, Denmark

http://www.mxm.dk/
IT's Mad Science
Oct 9 '06 #2
ce*********@gma il.com wrote:
Hi, I am starting to have a look to a python program that does not free
memory (I am using python 2.4.3). As I have read about a new memory
management in python 2.5 (http://evanjones.ca/python-memory.html) I
decided to try the program with the new version.
With the new version of python the memory consumption is the same. Now
I am asking myself if python 2.5 has any improving in memory
management or maybe not yet.
the new mechanism only helps if you temporarily create large structures and release
them quickly; it does not help if you're *fragmenting* the memory. Python requests
memory from the underlying system in blocks (called "arenas"), and it can only re-
turn them if they're entirely empty.

(this is discussed on the page you link to, and the various posts it links to).

</F>

Oct 9 '06 #3
I just checked the comsuptiom with the 'top' unix util. I am procesing
html docs and the amount of memory rises continiously.
I am using a lot of lists and docs. Some of them with objects. Do i
have to make any special thing in order to get them released back to
the Memory Manager? For instantec.. is it enough to do a clear() in a
dictionary?

-- Cesar

Oct 9 '06 #4
ce*********@gma il.com wrote:
>I just checked the comsuptiom with the 'top' unix util. I am procesing
html docs and the amount of memory rises continiously.
what library are you using for this ?
I am using a lot of lists and docs. Some of them with objects. Do i
have to make any special thing in order to get them released back to
the Memory Manager? For instantec.. is it enough to do a clear() in a
dictionary?
the garbage collector should take care of that, unless you're doing something
fishy. maybe you're keeping pointers to documents you've already processed
in some data structure somewhere?

</F>

Oct 9 '06 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
5125
by: Marcelo A. Camelo | last post by:
Hi! I will be presenting Python to an audience of game developers, mostly C/C++ programmers. In my presentation I will talk about using python and C/C++ extension instead of pure C/C++ to write commercial games and content creation tools. In a number of times I have presented these ideas to this same kind of audience and they were received with cold skepticism, despite the fact that some, including me, are already doing this with...
9
2485
by: Chris S. | last post by:
Is it possible to determine how much memory is allocated by an arbitrary Python object? There doesn't seem to be anything in the docs about this, but considering that Python manages memory allocation, why would such a module be more difficult to design than say, the GC?
35
5533
by: Alex Martelli | last post by:
Having fixed a memory leak (not the leak of a Python reference, some other stuff I wasn't properly freeing in certain cases) in a C-coded extension I maintain, I need a way to test that the leak is indeed fixed. Being in a hurry, I originally used a q&d hack...: if sys.platform in ('linux2', 'darwin'): def _memsize(): """ this function tries to return a measurement of how much memory this process is consuming, in some arbitrary unit...
18
2953
by: diffuser78 | last post by:
I have a python code which is running on a huge data set. After starting the program the computer becomes unstable and gets very diffucult to even open konsole to kill that process. What I am assuming is that I am running out of memory. What should I do to make sure that my code runs fine without becoming unstable. How should I address the memory leak problem if any ? I have a gig of RAM. Every help is appreciated.
11
7629
by: Hari Sekhon | last post by:
I do import zipfile zip=zipfile.ZipFile('d:\somepath\cdimage.zip') zip.namelist() then either of the two: A) file('someimage.iso','w').write(zip.read('someimage.iso'))
11
3974
by: seberino | last post by:
Suppose a C extension locally built an array of PyObject* 's as follows... my_array = malloc(n * sizeof(PyObject*)); for (i = 0; i < n; i++) { my_array = PyList_New(0); } Q1: Must I do a Py_DECREF(my_array) on all elements before exiting this C extension function? (What if
1
2282
by: Joe Peterson | last post by:
I've been doing a lot of searching on the topic of one of Python's more disturbing issues (at least to me): the fact that if a __del__ finalizer is defined and a cyclic (circular) reference is made, the garbage collector cannot clean it up. First of all, it seems that it's best to avoid using __del__. So far, I have never used it in my Python programming. So I am safe there. Or am I? Also, to my knowledge, I have never created a...
5
24915
by: kumarmdb2 | last post by:
Hi guys, For last few days we are getting out of private memory error. We have a development environment. We tried to figure out the problem but we believe that it might be related to the OS (I am new to Windows so not sure). We are currently bouncing the instance to overcome this error. This generally happen at the end of business day only (So maybe memory might be getting used up?). We have already increased the statement heap & ...
3
3464
by: Hank | last post by:
Steve Holden wrote: Hi, Steve, This not simply a pathological condition. My people are keeping trying many ways to have job done, and the memory problem became the focus we are paying attention on at this moment. Could you please give us some clear clues to obviously call python to free memory. We want to control its gc operation handily as we were using J**A.
0
9820
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10462
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9630
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
8016
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7169
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5849
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
6041
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
4270
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3276
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.