How to get memory size/usage of python object

Santiago Romero wrote:

Is there a way to check the REAL size in memory of a python object?

in standard Python, without reading the interpreter source code
carefully, no.

to get an approximate value, create a thousand (or a million) objects
and check how much the interpreter grows when you do that.

</F>

Santiago Romero <sr*****@gmail.comwrote:

Is there a way to check the REAL size in memory of a python object?

Something like

>print sizeof(mylist)

[ ... ]

Would you care to precisely define "REAL size" first? Consider:

>>atuple = (1, 2)
mylist = [(0, 0), atuple]

Should sizeof(mylist) include sizeof(atuple) ?

>>del atuple

What about now, when mylist has the only reference to the (1, 2)
object that also used to be referred to as atuple?

--
\S -- si***@chiark.greenend.org.uk -- http://www.chaos.org.uk/~sion/
"Frankly I have no feelings towards penguins one way or the other"
-- Arthur C. Clarke
her nu becomeþ se bera eadward ofdun hlæddre heafdes bæce bump bump bump

Sion Arrowsmith wrote:

Santiago Romero <sr*****@gmail.comwrote:

>Is there a way to check the REAL size in memory of a python object?

Something like

>>print sizeof(mylist)

[ ... ]

Would you care to precisely define "REAL size" first? Consider:

>>>atuple = (1, 2)
mylist = [(0, 0), atuple]

Should sizeof(mylist) include sizeof(atuple) ?

>>>del atuple

What about now, when mylist has the only reference to the (1, 2)
object that also used to be referred to as atuple?

or add to the mix

>>mylist = [(0,0), atuple] * 1000

where the same atuple is referenced 1000 times. And then if you

>>del atuple

defining "sizeof()" becomes even more peculiar if you have a
thousand things that "have" the same item that nothing else
claims ownership of.

Or, if you have this:

>>alist = [1,2,3]
mylist = ['a', 'b', alist] * 10
s1 = sizeof(mylist)
alist.append(42)
s2 = sizeof(mylist)

should s1==s2 ?

-tkc

Would you care to precisely define "REAL size" first? Consider:

>atuple = (1, 2)
mylist = [(0, 0), atuple]

Should sizeof(mylist) include sizeof(atuple) ?

No, I'm talking about "simple" lists, without REFERENCES to another
objects into it.

I mean:

lists = [ 0, 1, 2, 3, 4, (1,2), 3]

or

array = [ [0,0,0,0,0,0,0], [1,1,1,1,2,1,2], ... ]

Maybe I can "pickle" the object to disk and see the filesize ... :-?

On Thu, 10 Jan 2008 00:14:42 -0800, Santiago Romero wrote:

>Would you care to precisely define "REAL size" first? Consider:

>>atuple = (1, 2)
mylist = [(0, 0), atuple]

Should sizeof(mylist) include sizeof(atuple) ?

No, I'm talking about "simple" lists, without REFERENCES to another
objects into it.

I mean:

lists = [ 0, 1, 2, 3, 4, (1,2), 3]

That list has 7 references to other objects. One of those objects has 2
references to objects.

In total, depending on implementation, there could be as many as 9
objects referenced by that list, or as few as 6 objects (both references
to 3 could be to the same object).

In the current CPython implementation, that list will have 7 references
to 6 objects. Including indirect references, there will be 9 references
to 6 objects. (Or so I understand.)

or

array = [ [0,0,0,0,0,0,0], [1,1,1,1,2,1,2], ... ]

Ignoring the '...', there will be a total of 16 references to 5 objects
in the current CPython implementation. Other Pythons (Jython, IronPython,
PyPy, ...) may be different.

Maybe I can "pickle" the object to disk and see the filesize ... :-?

That would measure something very different.

Possibly you want something like this heuristic:

def sizeof(obj):
"""APPROXIMATE memory taken by some Python objects in
the current 32-bit CPython implementation.

Excludes the space used by items in containers; does not
take into account overhead of memory allocation from the
operating system, or over-allocation by lists and dicts.
"""
T = type(obj)
if T is int:
kind = "fixed"
container = False
size = 4
elif T is list or T is tuple:
kind = "variable"
container = True
size = 4*len(obj)
elif T is dict:
kind = "variable"
container = True
size = 144
if len(obj) 8:
size += 12*(len(obj)-8)
elif T is str:
kind = "variable"
container = False
size = len(obj) + 1
else:
raise TypeError("don't know about this kind of object")
if kind == "fixed":
overhead = 8
else: # "variable"
overhead = 12
if container:
garbage_collector = 8
else:
garbage_collector = 0
malloc = 8 # in most cases
size = size + overhead + garbage_collector + malloc
# Round to nearest multiple of 8 bytes
x = size % 8
if x != 0:
size += 8-x
size = (size + 8)
return size
See:
http://mail.python.org/pipermail/pyt...ch/135223.html

to get you started.

--
Steven.