473,320 Members | 1,930 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Re: Python memory usage

On Oct 21, 5:19*pm, Rolf Wester <rolf.wes...@ilt.fraunhofer.dewrote:
Hi,

I have the problem that with long running Python scripts (many loops)
memory consumption increases until the script crashes. I used the
following small script to understand what might happen:
<snip>

AFAIK, python uses malloc behind the scenes to allocate memory. From
the malloc man page...

"The malloc() and free() functions provide a simple, general-purpose
memory allocation package. The malloc() function returns a pointer to
a block of at least size bytes suitably aligned for any use. If the
space assigned by malloc() is overrun, the results are undefined.

The argument to free() is a pointer to a block previously allocated by
malloc(), calloc(), or realloc(). After free() is executed, this space
is made available for further allocation by the application, though
not returned to the system. Memory is returned to the system only
upon termination of the application. If ptr is a null pointer, no
action occurs. If a random number is passed to free(), the results are
undefined."

HTH,

Pete
Oct 29 '08 #1
1 1831
On Wed, Oct 29, 2008 at 6:56 PM, pe********@gmail.com
<pe********@gmail.comwrote:
On Oct 21, 5:19 pm, Rolf Wester <rolf.wes...@ilt.fraunhofer.dewrote:
>Hi,

I have the problem that with long running Python scripts (many loops)
memory consumption increases until the script crashes. I used the
following small script to understand what might happen:
<snip>

AFAIK, python uses malloc behind the scenes to allocate memory. From
the malloc man page...

"The malloc() and free() functions provide a simple, general-purpose
memory allocation package. The malloc() function returns a pointer to
a block of at least size bytes suitably aligned for any use. If the
space assigned by malloc() is overrun, the results are undefined.

The argument to free() is a pointer to a block previously allocated by
malloc(), calloc(), or realloc(). After free() is executed, this space
is made available for further allocation by the application, though
not returned to the system. Memory is returned to the system only
upon termination of the application. If ptr is a null pointer, no
action occurs. If a random number is passed to free(), the results are
undefined."
Depending on your malloc implementation, that may not be true. IN
particular, with glibc, bit allocations are done with mmap, and those
areas are unmaped when free is called; any such area is immediatly
returned to the system

http://www.gnu.org/software/libtool/...ble-Parameters

David
Oct 29 '08 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: ulysses | last post by:
hi, I'm working in python 5 months. I think it's very cool language. I do a p2p python program GUI. First I make a software by wxpython. But I find wxpython use many many memory. Second I use...
8
by: Sridhar R | last post by:
Hi, I am a little experienced python programmer (2 months). I am somewhat experienced in C/C++. I am planning (now in design stage) to write an IDE in python. The IDE will not be a simple...
0
by: Robby Dermody | last post by:
Hey guys (thus begins a book of a post :), I'm in the process of writing a commercial VoIP call monitoring and recording application suite in python and pyrex. Basically, this software sits in a...
25
by: abhinav | last post by:
Hello guys, I am a novice in python.I have to implement a full fledged mail server ..But i am not able to choose the language.Should i go for C(socket API) or python for this project? What are the...
13
by: placid | last post by:
Hi All, Just wondering when i run the following code; for i in range(1000000): print i the memory usage of Python spikes and when the range(..) block finishes execution the memory usage...
3
by: test.07 | last post by:
I am wondering what happens to a thread in python in relation to win32com extensions. If I create a new thread, that uses the Dispatch method from win32com, what happens to the memory allocated...
17
by: frederic.pica | last post by:
Greets, I've some troubles getting my memory freed by python, how can I force it to release the memory ? I've tried del and gc.collect() with no success. Here is a code sample, parsing an XML...
0
by: greg.novak | last post by:
I am using Python to process particle data from a physics simulation. There are about 15 MB of data associated with each simulation, but there are many simulations. I read the data from each...
1
by: yzghan | last post by:
Hi all, I feel that my python script is leaking memory. And this is a test I have: log.write(" " + "test() ... memory usage: " + " ".join(repr(i/(1024*1024)) for i in getMemInfo()) + "\n") m...
1
by: Jean-Paul Calderone | last post by:
On Tue, 22 Apr 2008 14:54:37 -0700 (PDT), yzghan@gmail.com wrote: The test doesn't demonstrate any leaks. It does demonstrate that memory usage can remain at or near peak memory usage even after...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.