Hello everybody,
I try to solve the following problem: I have a python program that takes a
lot of memory (>hundred Mb). I made an improvement (I hope) and I want to
measure the gain (if possible on several platforms). I would like to be able
to print the max memory taken during the run upon exiting my Python program
(like I already do for the time taken to run).
I can see the total process size of the Python process (Task manager on MS
Win or Unix "ps" command) but that is not precise enough for me and also not
convenient. I don't care about the python interpreter overhead, I'm
interested only in the total size of Python objects allocated by my program.
Is there a Python way to get that (meaning without resorting to
non-portable, less precise OS features)?
I would like to get that size at different given moments in time to make up
a timeline of memory consumption. But, if that is too tricky, I would be
happy to just get the maximum.
Any suggestion welcome,
Adrian.
3  3413 
"Neagu, Adrian" <Ad**********@n uance.comwrites :
I would like to get that size at different given moments in time to make up
a timeline of memory consumption. But, if that is too tricky, I would be
happy to just get the maximum.
Any suggestion welcome,
See the docs for the gc module.
Paul Rubin wrote:
See the docs for the gc module.
This is the first place I've checked. I see no useful info there about
the actual size of the objects (bytes occupied in memeory).
The closest thing to what I need is get_objects( ). Then I have to go
over the list
and do something with the objects. Wouldn't that be too slow?
Thanks anyway,
Adrian.
Neagu, Adrian wrote:
I try to solve the following problem: I have a python program that takes a
lot of memory (>hundred Mb). I made an improvement (I hope) and I want to
measure the gain (if possible on several platforms). I would like to be able
to print the max memory taken during the run upon exiting my Python program
(like I already do for the time taken to run).
You could try PySizer: http://pysizer.8325.org/
--
Heikki Toivonen This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: Roy Smith |
last post by:
I understand that psyco significantly increases memory use. Is that for
code or data? More specifically, if I've got a memory intensive
application (it might use 100's of Mbytes of data), should I expect
memory use to go up significantly under psyco?
Also, for that memory intensive application, how should I expect Python
memory use to compare with C++? I'm really only interested in data; the
memory needed to store the code is almost...
|
by: Mike Peretz |
last post by:
I am trying to optimize my C# program, but no matter what I try the
application keeps eating memory. I verified all the references and even got
special software to count references. I made sure all the reference count
reaches zero for all the objects that are no longer used. I also call
Dispose as much as possible. Regardless, the memory is not freed. Even if I
force the GC, some memory gets freed but not much.
|
by: Derrick |
last post by:
Thanks to many on this newsgroup, I now have a prototype Windows Forms C#
app that reads xml documents into DataSets via XmlDataDocument, and does
*stuff* with it.
I basically have about 25 megs worth of xml data that I read in and operate
on. Once loaded, before doing anything of substance with the data, my app's
memory footprint goes up to nearly 140 megs right off the bat.
Question is, if this is not an expected observation, what...
|
by: Derrick |
last post by:
I am reading in xml files that equate to sql tables, via XmlDataDocument,
and then operating on the DataSet. With the most simple app that just loads
the xml doc, I see the memory footprint of the app grow to roughly 5x the
size of the xml document after the xml doc is read. Can anyone comment on
this? Is this expected?
Below are two samples, the first hits northwind, and creates a 19meg
(roughly, on my machine) xml file. The second...
|
by: Bob Dufour |
last post by:
We got a windows form application that we wrote in VB.Net.
Essentially its a manager for a list of persons and their contacts and some
other info about the persons. No rocket science but lots of textboxes and
comboxes to use lookup lists, some tab controls allow viewing of data about
a person in logical groupings.
We find that as we load the app with only a few hundred records of data in a
lookup list (Rough estimate of data only at...
| | |
by: Rune Strand |
last post by:
Is there a way to measure how much memory a data structure use? For
instance, what is the footprint of a particular list object like
?
|
by: Adrian Neagu |
last post by:
Hello everybody,
I try to solve the following problem: I have a python program that takes a lot of memory (>hundred Mb). I made an improvement (I hope) and I want to measure the gain (if possible on several platforms). I would like to be able to print the max memory taken during the run upon exiting my Python program (like I already do for the time taken to run).
I can see the total process size of the Python process (Task manager on MS...
|
by: nazgul |
last post by:
Hi all,
I have an app that runs on multiple boxes. On my slackware box, running
Python 2.5.1, top shows this:
Mem: 1002736k total, 453268k used, 549468k free, 31392k buffers
Swap: 2097136k total, 0k used, 2097136k free, 136876k cached
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
2741 dnm 16 0 267m 261m 2676 S 99.3 26.7 14:54.62 python
|
by: arobinson |
last post by:
I am a developer relatively new to python and brand new to pygtk
development. I wanted to write a bloglines tray icon notifier for my
linux box. Everything is working quite well functionality wise, but
the memory footprint is horrible.
This little tray icon that simply has a thread for updates, a
gtk.StatusIcon, a few relatively small dict objects and a context menu
is running at around 20-25 MB of memory.
Doing some initial research,...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
| | |
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: isladogs |
last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM).
In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules.
He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms.
Adolph will...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| | |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |
