471,337 Members | 851 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,337 software developers and data experts.

weakrefs and bound methods

Hi.

I have a problem with weak refs and bound methods. The best explanation for the
problem is a short bit of code. I have the three classes Wrapper, Foo and Bar:

import weakref

class Wrapper(object):
def __init__(self,x):
self.x = weakref.ref(x)

def __call__(self,*args,**kwargs):
x = self.x()
if x is None:
print "lost reference"
else:
return x(*args,**kwargs)

class Foo(object):
def __init__(self):
self._methods = set()
self._methods.add(Wrapper(self._foo))

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Foo"

class Bar(object):
def __init__(self):
self._methods = set()
self._methods.add(self._foo)

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Bar"

Now look what happens when I do this:
>>f=Foo()
f.callMethods()
lost reference
>>del f
del Foo
>>b=Bar()
b.callMethods()
_foo
>>del b
Foo looses the reference to its method but Bar on the other hand has a refloop and
never gets deleted.

Does anyone know what happens here? How can I write such classes without refloop and
without lost reference?

I'm using:
Python 2.5.1 (r251:54863, May 2 2007, 16:56:35)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2

-panzi
Oct 7 '07 #1
11 1384
When I change the class Wrapper to following, the class Foo works:

class Wrapper(object):
def __init__(self,x):
self.func_name = x.func_name
self.x = weakref.ref(x.im_self)

def __call__(self,*args,**kwargs):
x = self.x()
if x is None:
print "lost reference"
else:
return getattr(x,self.func_name)(*args,**kwargs)
But that's ugly. Any better idea?
Oct 7 '07 #2
On Sun, 07 Oct 2007 16:51:33 +0200, Mathias Panzenboeck wrote:
import weakref

class Wrapper(object):
def __init__(self,x):
self.x = weakref.ref(x)

def __call__(self,*args,**kwargs):
x = self.x()
if x is None:
print "lost reference"
else:
return x(*args,**kwargs)

class Foo(object):
def __init__(self):
self._methods = set()
self._methods.add(Wrapper(self._foo))

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Foo"

class Bar(object):
def __init__(self):
self._methods = set()
self._methods.add(self._foo)

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Bar"

Now look what happens when I do this:
>>>f=Foo()
f.callMethods()
lost reference
>>>del f
del Foo
>>>b=Bar()
b.callMethods()
_foo
>>>del b

Foo looses the reference to its method but Bar on the other hand has a refloop and
never gets deleted.
``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.

Drop all those `__del__()` methods as they prevent the garbage collector
from collecting "cycles".

Ciao,
Marc 'BlackJack' Rintsch
Oct 7 '07 #3
On Oct 7, 12:26 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.netwrote:
Drop all those `__del__()` methods as they prevent the garbage collector
from collecting "cycles".
I fully agree and I will add that __del__ methods are always
a bad idea. Also notice that recently Raymond Hetting said in
this list that he wanted to submit a PEP to remove __del__ from
Python 3000 (I don't know if this will ever happen tough).

Michele Simionato

Oct 7 '07 #4
Marc 'BlackJack' Rintsch wrote:
On Sun, 07 Oct 2007 16:51:33 +0200, Mathias Panzenboeck wrote:
>import weakref

class Wrapper(object):
def __init__(self,x):
self.x = weakref.ref(x)

def __call__(self,*args,**kwargs):
x = self.x()
if x is None:
print "lost reference"
else:
return x(*args,**kwargs)

class Foo(object):
def __init__(self):
self._methods = set()
self._methods.add(Wrapper(self._foo))

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Foo"

class Bar(object):
def __init__(self):
self._methods = set()
self._methods.add(self._foo)

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Bar"

Now look what happens when I do this:
>>>>f=Foo()
f.callMethods()
lost reference
>>>>del f
del Foo
>>>>b=Bar()
b.callMethods()
_foo
>>>>del b
>
Foo looses the reference to its method but Bar on the other hand has a refloop and
never gets deleted.

``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.
Why is it then that f (instance of Foo) really gets deleted? (__del__ is called)
Drop all those `__del__()` methods as they prevent the garbage collector
from collecting "cycles".

Ciao,
Marc 'BlackJack' Rintsch
I only inserted them so I can see if the objects are really freed. How can I see that
without a __del__ method?
Thanks so far,
panzi
Oct 7 '07 #5
Marc 'BlackJack' Rintsch wrote:
``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.
Actually I have the opposite problem. The reference (to the bound method)
gets lost but it shouldn't!
-panzi
Oct 7 '07 #6
On Sun, 07 Oct 2007 16:38:23 +0000, Michele Simionato wrote:
On Oct 7, 12:26 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.netwrote:
>Drop all those `__del__()` methods as they prevent the garbage
collector from collecting "cycles".

I fully agree and I will add that __del__ methods are always a bad idea.
Always?

I recently wrote a bit of code where I needed to check that releasing the
first object in a tree-like structure would allow Python to garbage
collect all the other objects in a tree. I thought it would, but I wanted
to be sure ("don't guess, test"), so I wrote a simple class, gave it a
__del__ method that just printed self, inserted them in the tree, and
then deleted the first one.

Worked like a charm.

Without __del__, what should I have done to test that my code was
deleting objects and not leaking memory?

What should I do when my objects need to perform some special processing
when they are freed, if I shouldn't use __del__?

--
Steven.
Oct 7 '07 #7
Mathias Panzenboeck wrote:
Marc 'BlackJack' Rintsch wrote:
>``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.

Actually I have the opposite problem. The reference (to the bound method)
gets lost but it shouldn't!
Ahh, so you expected that ``Wrapper(self._foo)`` would not immediately
lose the reference? It will, because every time you write
``self._foo``, a new bound method is created::
>>class C(object):
... def foo(self):
... pass
...
>>f = C.foo
g = C.foo
id(f), id(g)
(14931448, 14891008)

Thus, there is only the one reference to the bound method, and by
wrapping it in a weakref, you are allowing it to disappear immediately::
>>x = weakref.ref(C.foo)
print x()
None

What behavior do you want here? That is, when were you hoping that the
bound method would disappear?

STeVe
Oct 7 '07 #8
On Oct 7, 1:14 pm, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.auwrote:
On Sun, 07 Oct 2007 16:38:23 +0000, Michele Simionato wrote:
On Oct 7, 12:26 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.netwrote:
Drop all those `__del__()` methods as they prevent the garbage
collector from collecting "cycles".
I fully agree and I will add that __del__ methods are always a bad idea.

Always?

I recently wrote a bit of code where I needed to check that releasing the
first object in a tree-like structure would allow Python to garbage
collect all the other objects in a tree. I thought it would, but I wanted
to be sure ("don't guess, test"), so I wrote a simple class, gave it a
__del__ method that just printed self, inserted them in the tree, and
then deleted the first one.

Worked like a charm.

Without __del__, what should I have done to test that my code was
deleting objects and not leaking memory?

What should I do when my objects need to perform some special processing
when they are freed, if I shouldn't use __del__?
The best thing is to use explicit resource management,
for instance with a try .. finally or with the "with" statement
in Python 2.5. The next best thing is to use weak references.
I have some code for various experiments with wearefs I did
some time ago, here it is:

import itertools, weakref, sys, gc

reference_list = [] # cannot be a set, you would lose references
resource_counter = itertools.count(1)

def resource(before_closing_callback=None,
after_closing_callback=None):
private = '_resource_%s' % resource_counter.next()

def get(self):
return getattr(self, private)

def set(self, resource):
setattr(self, private, resource)
def close(ref):
if before_closing_callback:
before_closing_callback(resource)
resource.close()
if after_closing_callback:
after_closing_callback(resource)
reference_list.remove(ref)
reference_list.append(weakref.ref(self, close))

return property(get, set)

class FakeResource(object):
def __init__(self, name):
print 'opening resource %s' % name
self.name = name
def close(self):
print 'closing resource %s' % self.name
def __repr__(self):
return '<FakeResource %r>' % self.name

class Example(object):
def __init__(self):
self.resource1 = FakeResource('r1')
self.resource2 = FakeResource('r2')
def __del__(self):
print '**************'
self.resource1.close()
self.resource2.close()

def warn_before_closing(res):
sys.stdout.write('going to close %s\n' % res)

class Example2(object):

resource1 = resource(warn_before_closing)
resource2 = resource()

def __init__(self):
self.resource1 = FakeResource('r1')
self.resource2 = FakeResource('r2')
gc.set_debug(gc.DEBUG_LEAK)

#e = Example()
e = Example2()
e.e = e
del e
print reference_list

Oct 7 '07 #9
Mathias Panzenboeck <e0******@student.tuwien.ac.atwrote:
...
I only inserted them so I can see if the objects are really freed. How can
I see that without a __del__ method?
You can use weakref.ref instances with finalizer functions - see the
long post I just made on this thread for a reasonably rich and complex
example.
Alex
Oct 7 '07 #10
Mathias Panzenboeck <e0******@student.tuwien.ac.atwrote:
Marc 'BlackJack' Rintsch wrote:
``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.

Actually I have the opposite problem. The reference (to the bound method)
gets lost but it shouldn't!
weakrefs to bound methods require some subtlety, see
<http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/81253(or what
I believe is the better treatment of this recipe in the printed edition
of the Python Cookbook -- of course, being the latter's editor, I'm
biased;-).
Alex
Oct 7 '07 #11
Alex Martelli wrote:
Mathias Panzenboeck <e0******@student.tuwien.ac.atwrote:
>Marc 'BlackJack' Rintsch wrote:
>>``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.
Actually I have the opposite problem. The reference (to the bound method)
gets lost but it shouldn't!

weakrefs to bound methods require some subtlety, see
<http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/81253(or what
I believe is the better treatment of this recipe in the printed edition
of the Python Cookbook -- of course, being the latter's editor, I'm
biased;-).
Alex
Thank you, that really helped. :)
-panzi
Oct 7 '07 #12

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

1 post views Thread by William Trenker | last post: by
12 posts views Thread by Ron Garret | last post: by
reply views Thread by Rhamphoryncus | last post: by
12 posts views Thread by Russell E. Owen | last post: by
8 posts views Thread by Kevin Little | last post: by
3 posts views Thread by Michael Schneider | last post: by
reply views Thread by Kamilche | last post: by
1 post views Thread by srinivasan srinivas | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.