468,780 Members | 2,330 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 468,780 developers. It's quick & easy.

Analyzing Python GC output - what is a "cell", and what informationis available about it.

I'm printing out each entry in "gc.garbage" after a garbage collection in
DEBUG_LEAK mode, and I'm seeing many entries like

<cell at 0x00F7C170: function object at 0x00FDD6B0>

That's the output of "repr". Are "cell" objects created only from
external C libraries, or can regular Python code generate them? Is there
any way to find out what the 'function object' is from within Python?

(I'm looking for a possible memory leak, obviously.)

John Nagle
Jan 11 '08 #1
4 2604
John Nagle <na***@animats.comwrote:
I'm printing out each entry in "gc.garbage" after a garbage collection in
DEBUG_LEAK mode, and I'm seeing many entries like

<cell at 0x00F7C170: function object at 0x00FDD6B0>

That's the output of "repr". Are "cell" objects created only from
external C libraries, or can regular Python code generate them? Is there
any way to find out what the 'function object' is from within Python?
Cell objects are created whenever you have a function that references a
variable in an outer scope. e.g.
>>def outer():
x = 42
def inner():
return x
return inner
>>inner = outer()
inner.func_closure[0]
<cell at 0x00C5D450: int object at 0x009657AC>
>>inner.func_closure[0].cell_contents
42
So in your case, cell.cell_contents.func_name might help.
Jan 11 '08 #2
Duncan Booth wrote:
John Nagle <na***@animats.comwrote:
>I'm printing out each entry in "gc.garbage" after a garbage collection in
DEBUG_LEAK mode, and I'm seeing many entries like

<cell at 0x00F7C170: function object at 0x00FDD6B0>

That's the output of "repr". Are "cell" objects created only from
external C libraries, or can regular Python code generate them? Is there
any way to find out what the 'function object' is from within Python?
Cell objects are created whenever you have a function that references a
variable in an outer scope. e.g.

So in your case, cell.cell_contents.func_name might help.
Tried that:

print repr(item).encode('ascii','replace')
print "Type:",type(item)
try:
print item.cell_contents
except Exception, message:
print "Unprintable:",message

<cell at 0x00F88DF0: function object at 0x0100CFB0>
Type: <type 'cell'>
Unprintable: 'cell' object has no attribute 'cell_contents'

So it doesn't have a "cell_contents" attribute.

Tried:
print item.dir()
got:
'cell' object has no attribute 'dir'

Tried:
print item.__dict__
got:
'cell' object has no attribute '__dict__'

It looks like this is a low-level PyCellObject not generated from Python code.
Any ideas? I'm using the M2Crypto and MySQLdb libraries, and suspect
a reference count bug in one of those.

John Nagle
Jan 11 '08 #3
On Jan 11, 2008 6:20 PM, John Nagle <na***@animats.comwrote:
Tried:
print item.dir()
got:
'cell' object has no attribute 'dir'
I don't know nothing about cell objects...
but why don't you try dir(item) instead?

Francesco
Jan 11 '08 #4
Francesco Guerrieri wrote:
On Jan 11, 2008 6:20 PM, John Nagle <na***@animats.comwrote:
>Tried:
print item.dir()
got:
'cell' object has no attribute 'dir'
It's a problem inside MySQLdb's "connections.py":

def _get_unicode_literal():
def unicode_literal(u, dummy=None):
return db.literal(u.encode(unicode_literal.charset))
return unicode_literal

Each time a MySQLdb Connection object is created, it generates a closure,
with another instance of the function "unicode_literal" plus some other
junk. That generates circular references. Eventually those
things get garbage-collected, but if you're running GC in leak detection
mode, they show up.

The reason for the closure is that "unicode_literal.charset" of the
function is being stored into from outside the function. So there
actually is a reason to use a closure.

John Nagle
Jan 11 '08 #5

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

13 posts views Thread by Allison Bailey | last post: by
reply views Thread by Carlos Eduardo Peralta | last post: by
267 posts views Thread by Xah Lee | last post: by
1 post views Thread by tkpmep | last post: by
17 posts views Thread by David C. Ullrich | last post: by
1 post views Thread by CARIGAR | last post: by
2 posts views Thread by Marin | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.