473,411 Members | 2,289 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,411 software developers and data experts.

Memory stats

Hi list,

I am trying to find a general memory profiler that can measure the
memory usage in Python program
and gather some stats about object usages, and things like that.

I am trying to find a simple python module to be able to customize it
and integrates it to other tools i have.
(it should exists i guess)

Thanks,

Tarek

--
Tarek Ziadé, Nuxeo R&D (Paris, France)
CPS Platform : http://www.cps-project.org
Zope3 / ECM : http://www.z3lab.org
mail: tziade at nuxeo.com; tel: +33 (0) 6 30 37 02 63

Sep 25 '05 #1
8 2139
Tarek Ziadé wrote:
Hi list,

I am trying to find a general memory profiler that can measure the
memory usage in Python program
and gather some stats about object usages, and things like that.

I am trying to find a simple python module to be able to customize it
and integrates it to other tools i have.
(it should exists i guess)


Try the gc-module. It's a no-brainer to compute a class histogram. I
once even wrote my own Qt-based analyzer - maybe I'll release it one day :)

Diez
Sep 25 '05 #2
Tarek Ziadé wrote:
I am trying to find a general memory profiler that can measure the
memory usage in Python program
and gather some stats about object usages, and things like that.


As Diez says, use gc: gc.getobjects() gives you all container objects.
If you want a list of all objects (container or not), you have to
compile a debug build of Python.

Regards,
Martin
Sep 26 '05 #3
Tarek Ziadé wrote:
I am trying to find a general memory profiler that can measure the
memory usage in Python program
and gather some stats about object usages, and things like that.


As Diez says, use gc: gc.getobjects() gives you all container objects.
If you want a list of all objects (container or not), you have to
compile a debug build of Python.

Regards,
Martin
Sep 26 '05 #4
linux:
http://aspn.activestate.com/ASPN/Coo.../Recipe/286222

Martin v. Löwis wrote:
Tarek Ziadé wrote:
I am trying to find a general memory profiler that can measure the
memory usage in Python program
and gather some stats about object usages, and things like that.


As Diez says, use gc: gc.getobjects() gives you all container objects.
If you want a list of all objects (container or not), you have to
compile a debug build of Python.

Regards,
Martin


Sep 27 '05 #5
Martin v. Löwis wrote:
Tarek Ziadé wrote:

I am trying to find a general memory profiler that can measure the
memory usage in Python program
and gather some stats about object usages, and things like that.


As Diez says, use gc: gc.getobjects() gives you all container objects.
If you want a list of all objects (container or not), you have to
compile a debug build of Python.

Regards,
Martin

Ok thanks,
I am amazed not to find an existing implementation for this.

Regards
Tarek

--
Tarek Ziadé, Nuxeo R&D (Paris, France)
CPS Platform : http://www.cps-project.org
Zope3 / ECM : http://www.z3lab.org
mail: tziade at nuxeo.com; tel: +33 (0) 6 30 37 02 63

Sep 27 '05 #6
gene tani wrote:
linux:
http://aspn.activestate.com/ASPN/Coo.../Recipe/286222

Yes thanks I have found this one, I need to try it out,
but it does not provide a way to refcount,

Another solution could be to use trace maybe

I am going to try things out

Regards

Sep 27 '05 #7
In message <ma************************************@python.org >, Tarek
Ziadé <tz****@nuxeo.com> writes
I am trying to find a general memory profiler that can measure the
memory usage in Python program
and gather some stats about object usages, and things like that.


Not a Python module, but Python Memory Validator may fit the bill. No
data on the website, just go straight to the beta page and select the
product. Windows NT/W2K/XP/etc..

http://www.softwareverify.com

http://www.softwareverify.com/beta.php?product=PMVB000

Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk/software.html
Computer Consultancy, Software Development
Windows C++, Java, Assembler, Performance Analysis, Troubleshooting
Sep 27 '05 #8
Tarek Ziadé wrote:
If you want a list of all objects (container or not), you have to
compile a debug build of Python.


I am amazed not to find an existing implementation for this.


the debug build is an existing implementation, of course.

</F>

Sep 27 '05 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Greg | last post by:
Hello... Our system uses JBoss 3.2 with MySQL. Part of the system is a stats collection process that fires as a TimerListener every 15 sec or so. It grabs some info and stuffs it into a...
16
by: JCauble | last post by:
We have a large Asp.net application that is currently crashing our production servers. What we are seeing is the aspnet_wp eat up a bunch of memory and then stop unexpectedly. Does not recycle. ...
35
by: Alex Martelli | last post by:
Having fixed a memory leak (not the leak of a Python reference, some other stuff I wasn't properly freeing in certain cases) in a C-coded extension I maintain, I need a way to test that the leak is...
4
by: Hermann Maier | last post by:
hi, i need to find out the memory usage of a specific function that i use in my program. this function does some recursive calculations and i want my program to display the amount of memory the...
2
by: Rishan | last post by:
Hi, I'm trying to debug a memory leak on a production application for one of my clients. The aspnet_wp.exe, if left to grow, will eventually consume enough memory to throw an out of memory...
9
by: ankitdesai | last post by:
I would like to parse a couple of tables within an individual player's SHTML page. For example, I would like to get the "Actual Pitching Statistics" and the "Translated Pitching Statistics"...
2
by: Monu | last post by:
HI All, I am getting problem in using hotshot profiler. When I hotshot with lineevents=0, it works fine, but when I use lineevents=1, I get error in stats here is my code: import hotshot,...
7
by: Ravi | last post by:
I created a program just to amuse me. I consumes a lot of memory and never gives it up: #incluse <stdio.h> #include <stdlib.h> int main(void) { void *a; while(1) a = malloc(99999); }
2
by: Richard Cranium | last post by:
Someone posted a nifty trick a while ago for estimating the amount of memory currently in use in a running application. Can someone either repost that or let me know their own thoughts on the...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.