473,397 Members | 2,099 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,397 software developers and data experts.

memory usage of a specific function

hi,

i need to find out the memory usage of a specific function that i use in
my program. this function does some recursive calculations and i want my
program to display the amount of memory the function used to calculate a
specific value.

thx
Dec 21 '05 #1
4 3397

Hermann Maier wrote:
hi,

i need to find out the memory usage of a specific function that i use in
my program. this function does some recursive calculations and i want my
program to display the amount of memory the function used to calculate a
specific value.

thx


I was thinking that Heapy in Guppy-PE (http://guppy-pe.sourceforge.net)
could be of help, but I can not give any specific advice based on only
the information given.

Maybe you could be more specific about your requirements, and post
the actual function code or some other illustrating example.

Regards,

Sverker

Jan 4 '06 #2
In message <11**********************@g44g2000cwa.googlegroups .com>,
Sverker Nilsson <sv********@home.se> writes
i need to find out the memory usage of a specific function that i use in
my program. this function does some recursive calculations and i want my
program to display the amount of memory the function used to calculate a
specific value.


Python Memory Validator.

Run your program to completion.
Switch to the hotspots tab.
Search for your function.
All memory used in that function will be shown in the tree (with the
effective callstack) underneath that function node in the tree.

http://www.softwareverify.com

Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk/software.html
Computer Consultancy, Software Development
Windows C++, Java, Assembler, Performance Analysis, Troubleshooting
Jan 4 '06 #3

Stephen Kellett wrote:
In message <1136382937.162468.253...@g44g2000cwa.googlegroups .com>,

Sverker Nilsson <sverker...@home.se> writes

[Note that actually it was Hermann Maier that wrote the following
but as quoted, it may look like it was I that wrote it.]
i need to find out the memory usage of a specific function that i use in
my program. this function does some recursive calculations and i want my
program to display the amount of memory the function used to calculate a
specific value.
Python Memory Validator.

Run your program to completion.
Switch to the hotspots tab.
Search for your function.
All memory used in that function will be shown in the tree (with the
effective callstack) underneath that function node in the tree.


http://www.softwareverify.com
I can't try it out easily since it doesn't seem to support Linux. Also
looking at the home page I couldn't find any description of what it
was actually doing. The info links didn't work, they showed blank
pages with my browser (Netscape 3.01.)

So if you would like to explain here, it would be helpful.
Especially, it isn't clear to me what this means (requoting):
All memory used in that function


Since Python has a reference-counting allocator, and various
kinds of objects use specialized allocation pools, it seems
unclear and implementation defined what is meant with
'all memory used'.

I could think of some different things this could mean.

1. The total memory allocated, regardless of whether it is released
again. If it is to mean really all the memory, it should include
memory allocated by special allocators such as the allocator for
ints. In this case, the following example function

def f():
x = 0
while x < 1000000:
x += 1

would have allocated (at least) 1000000 objects,
one for each iteration since int's are immutable
and one is allocated each time x is assigned to
(although each one is also deallocated.)

So according to the meaning (1.), the
'total memory used' would be 1000000 * the size of ints.
[ Though with some alternative interpreter implementation
it might be an entirely different value such as 0 bytes since
it may not need to allocate every integer in the heap. Would
'allocating' on the stack count? In a CPU register?]

Is something like this what would be reported by your system?

2. The second alternative meaning would be the memory allocated in
the function, MINUS the memory deallocated in the function.

Is this what is reported by your system?

If the case is according to the 2nd alternative, 'memory used in' f
would
be 0 bytes since all memory allocated is deallocated, and for a
function like the following

def g():
return range(1000)

it would be the size allocated for the list of length 1000 + any int
objects allocated to fill the list. Given the above function g, its
memory usage could be tested using heapy and the following function.
def test(g):
from guppy import hpy
hp=hpy()
hp.setrelheap()
hp.setrelheap() # Has to be done twice after the first import (XXX)
x=g()
print hp.heap()
test(g) Partition of a set of 902 objects. Total size = 15364 bytes.
Index Count % Size % Cumulative % Kind (class / dict of
class)
0 900 100 10800 70 10800 70 int
1 1 0 4128 27 14928 97 list
2 1 0 436 3 15364 100 types.FrameType

[ Why it shows an object of type FrameType is something of a mystery,
I'm not sure of the best way to get rid of it.]

I guess I could add a function somewhat like test(g) to Heapy,
depending on what common use cases seem to require.

On the other hand, test for memory usage according to alternative (1)
would be harder to add, and it is somewhat undefined what it
means. And it is perhaps not so useful as the 2nd alternative?

Sverker

Project home page: http://guppy-pe.sourceforge.net

Jan 4 '06 #4
In message <11**********************@g44g2000cwa.googlegroups .com>,
Sverker Nilsson <sv********@home.se> writes
Python Memory Validator.

Run your program to completion.
Switch to the hotspots tab.
Search for your function.
All memory used in that function will be shown in the tree (with the
effective callstack) underneath that function node in the tree.

http://www.softwareverify.com


I can't try it out easily since it doesn't seem to support Linux.


Correct. Windows NT and above only at this time.
Also
looking at the home page I couldn't find any description of what it
was actually doing. The info links didn't work, they showed blank
pages with my browser (Netscape 3.01.)
IE 6, Opera and Firefox. No idea with NS 3.0

Maybe this more direct link helps.

http://www.softwareverify.com/python...tor/index.html
So if you would like to explain here, it would be helpful.
Especially, it isn't clear to me what this means (requoting):
All memory used in that function

Exactly that. In a GC language (Python, Ruby, Java, Javascript) each
object needs to memory to exist. PMV tracks the creation/destruction of
objects (and in the case of Python, the other special non-object
datatypes such as integers that have their own pool).

Consider the following psuedo code

Func 1
allocates A
calls Func 2
calls Func 3
calls Func 4

Func 2
allocates B
allocates C

Func 3
allocates B
calls Func 5
allocates C

Func 4
allocates D
allocates E

Func 5
loop 5 times
allocates F
allocates G

This gives you the following tree showing all memory allocations

Func1
A
Func 2
B
C
Func 3
B
Func 5
5 x F
5 x G
C
Func 4
D
E

The tree also includes stats so that you know which node has more (in %
terms) data in it (in bytes) that another node. From that tree we can
see that the following objects are created.
A 1
B 2
C 2
D 1
E 1
F 5
G 5
memory usage could be tested using heapy and the following function.
You cannot reliably use native GC language code to monitor the same
application's memory usage. You need to do this externally (from C or
what takes your fancy - something that won't create/destroy objects on
the native language).
On the other hand, test for memory usage according to alternative (1)
would be harder to add, and it is somewhat undefined what it
means. And it is perhaps not so useful as the 2nd alternative?


PMV provides both. If you want stats on the number of objects created,
no problem. If you want callstacks for all objects created and stats for
the memory allocated at these locations, no problem. If you want stats
for each generation of objects (a generation being the group allocated
between GC invocations), no problem. Plus a whole load of analysis and
query functions to allow you to find information on particular object
types etc, plus data export functions (HTML and XML).

Stephen
--
Stephen Kellett
Object Media Limited http://www.objmedia.demon.co.uk/software.html
Computer Consultancy, Software Development
Windows C++, Java, Assembler, Performance Analysis, Troubleshooting
Jan 5 '06 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

8
by: rbt | last post by:
Would a Python process consume more memory on a PC with lots of memory? For example, say I have the same Python script running on two WinXP computers that both have Python 2.4.0. One computer has...
6
by: Tom | last post by:
We have a VERY simple .NET C# Form Application, that has about a 23MB Memory Footprint. It starts a window runs a process and does a regular expression. I have done a GC.Collect to make sure that,...
2
by: Dennis McCrohan | last post by:
Hi- I'm wondering if anyone has run across any program for C++ development that will help track the memory usage of different objects (class instances) within a program. I find it's easy enough...
5
by: Jeong-Gun Lee | last post by:
I'm writing a code of writing a value to a specific memory address. ================================================================= #include <stdio.h> int main() { long air; long...
8
by: Afanasiy | last post by:
Should I be concerned with the amount of memory my C# applications use? I have 2 gigs of ram, and I use most of that without running any C# applications. So, when I see C# applications in...
4
by: Crirus | last post by:
I have an aplication (obviously :)) I look at the Task Manager, memory usage column and notice the following: First time I start the application, I have around 55.000- 60.000 K. After I minimise...
3
by: bamapookie | last post by:
I am new to VB, but not new to programming. I am using VB.Net 2003 and I have written a small app to monitor several running processes and everything is fine except the memory usage. When the app...
1
by: fishscience | last post by:
I have a problem about memory usage. The code is as below: #include <list> int main(int argc, char* argv) { { std::list<void*ptr_list; { for (int i=0; i<9368; i++) {
12
by: David Given | last post by:
I have a situation where I need to be able to allocate chunks on unmapped virtual memory. Because the memory I'm allocating isn't mapped, I can't use a normal memory allocator, as most of them...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.