473,385 Members | 1,764 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

weakrefs and bound methods

Hi.

I have a problem with weak refs and bound methods. The best explanation for the
problem is a short bit of code. I have the three classes Wrapper, Foo and Bar:

import weakref

class Wrapper(object):
def __init__(self,x):
self.x = weakref.ref(x)

def __call__(self,*args,**kwargs):
x = self.x()
if x is None:
print "lost reference"
else:
return x(*args,**kwargs)

class Foo(object):
def __init__(self):
self._methods = set()
self._methods.add(Wrapper(self._foo))

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Foo"

class Bar(object):
def __init__(self):
self._methods = set()
self._methods.add(self._foo)

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Bar"

Now look what happens when I do this:
>>f=Foo()
f.callMethods()
lost reference
>>del f
del Foo
>>b=Bar()
b.callMethods()
_foo
>>del b
Foo looses the reference to its method but Bar on the other hand has a refloop and
never gets deleted.

Does anyone know what happens here? How can I write such classes without refloop and
without lost reference?

I'm using:
Python 2.5.1 (r251:54863, May 2 2007, 16:56:35)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2

-panzi
Oct 7 '07 #1
11 1449
When I change the class Wrapper to following, the class Foo works:

class Wrapper(object):
def __init__(self,x):
self.func_name = x.func_name
self.x = weakref.ref(x.im_self)

def __call__(self,*args,**kwargs):
x = self.x()
if x is None:
print "lost reference"
else:
return getattr(x,self.func_name)(*args,**kwargs)
But that's ugly. Any better idea?
Oct 7 '07 #2
On Sun, 07 Oct 2007 16:51:33 +0200, Mathias Panzenboeck wrote:
import weakref

class Wrapper(object):
def __init__(self,x):
self.x = weakref.ref(x)

def __call__(self,*args,**kwargs):
x = self.x()
if x is None:
print "lost reference"
else:
return x(*args,**kwargs)

class Foo(object):
def __init__(self):
self._methods = set()
self._methods.add(Wrapper(self._foo))

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Foo"

class Bar(object):
def __init__(self):
self._methods = set()
self._methods.add(self._foo)

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Bar"

Now look what happens when I do this:
>>>f=Foo()
f.callMethods()
lost reference
>>>del f
del Foo
>>>b=Bar()
b.callMethods()
_foo
>>>del b

Foo looses the reference to its method but Bar on the other hand has a refloop and
never gets deleted.
``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.

Drop all those `__del__()` methods as they prevent the garbage collector
from collecting "cycles".

Ciao,
Marc 'BlackJack' Rintsch
Oct 7 '07 #3
On Oct 7, 12:26 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.netwrote:
Drop all those `__del__()` methods as they prevent the garbage collector
from collecting "cycles".
I fully agree and I will add that __del__ methods are always
a bad idea. Also notice that recently Raymond Hetting said in
this list that he wanted to submit a PEP to remove __del__ from
Python 3000 (I don't know if this will ever happen tough).

Michele Simionato

Oct 7 '07 #4
Marc 'BlackJack' Rintsch wrote:
On Sun, 07 Oct 2007 16:51:33 +0200, Mathias Panzenboeck wrote:
>import weakref

class Wrapper(object):
def __init__(self,x):
self.x = weakref.ref(x)

def __call__(self,*args,**kwargs):
x = self.x()
if x is None:
print "lost reference"
else:
return x(*args,**kwargs)

class Foo(object):
def __init__(self):
self._methods = set()
self._methods.add(Wrapper(self._foo))

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Foo"

class Bar(object):
def __init__(self):
self._methods = set()
self._methods.add(self._foo)

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Bar"

Now look what happens when I do this:
>>>>f=Foo()
f.callMethods()
lost reference
>>>>del f
del Foo
>>>>b=Bar()
b.callMethods()
_foo
>>>>del b
>
Foo looses the reference to its method but Bar on the other hand has a refloop and
never gets deleted.

``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.
Why is it then that f (instance of Foo) really gets deleted? (__del__ is called)
Drop all those `__del__()` methods as they prevent the garbage collector
from collecting "cycles".

Ciao,
Marc 'BlackJack' Rintsch
I only inserted them so I can see if the objects are really freed. How can I see that
without a __del__ method?
Thanks so far,
panzi
Oct 7 '07 #5
Marc 'BlackJack' Rintsch wrote:
``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.
Actually I have the opposite problem. The reference (to the bound method)
gets lost but it shouldn't!
-panzi
Oct 7 '07 #6
On Sun, 07 Oct 2007 16:38:23 +0000, Michele Simionato wrote:
On Oct 7, 12:26 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.netwrote:
>Drop all those `__del__()` methods as they prevent the garbage
collector from collecting "cycles".

I fully agree and I will add that __del__ methods are always a bad idea.
Always?

I recently wrote a bit of code where I needed to check that releasing the
first object in a tree-like structure would allow Python to garbage
collect all the other objects in a tree. I thought it would, but I wanted
to be sure ("don't guess, test"), so I wrote a simple class, gave it a
__del__ method that just printed self, inserted them in the tree, and
then deleted the first one.

Worked like a charm.

Without __del__, what should I have done to test that my code was
deleting objects and not leaking memory?

What should I do when my objects need to perform some special processing
when they are freed, if I shouldn't use __del__?

--
Steven.
Oct 7 '07 #7
Mathias Panzenboeck wrote:
Marc 'BlackJack' Rintsch wrote:
>``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.

Actually I have the opposite problem. The reference (to the bound method)
gets lost but it shouldn't!
Ahh, so you expected that ``Wrapper(self._foo)`` would not immediately
lose the reference? It will, because every time you write
``self._foo``, a new bound method is created::
>>class C(object):
... def foo(self):
... pass
...
>>f = C.foo
g = C.foo
id(f), id(g)
(14931448, 14891008)

Thus, there is only the one reference to the bound method, and by
wrapping it in a weakref, you are allowing it to disappear immediately::
>>x = weakref.ref(C.foo)
print x()
None

What behavior do you want here? That is, when were you hoping that the
bound method would disappear?

STeVe
Oct 7 '07 #8
On Oct 7, 1:14 pm, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.auwrote:
On Sun, 07 Oct 2007 16:38:23 +0000, Michele Simionato wrote:
On Oct 7, 12:26 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.netwrote:
Drop all those `__del__()` methods as they prevent the garbage
collector from collecting "cycles".
I fully agree and I will add that __del__ methods are always a bad idea.

Always?

I recently wrote a bit of code where I needed to check that releasing the
first object in a tree-like structure would allow Python to garbage
collect all the other objects in a tree. I thought it would, but I wanted
to be sure ("don't guess, test"), so I wrote a simple class, gave it a
__del__ method that just printed self, inserted them in the tree, and
then deleted the first one.

Worked like a charm.

Without __del__, what should I have done to test that my code was
deleting objects and not leaking memory?

What should I do when my objects need to perform some special processing
when they are freed, if I shouldn't use __del__?
The best thing is to use explicit resource management,
for instance with a try .. finally or with the "with" statement
in Python 2.5. The next best thing is to use weak references.
I have some code for various experiments with wearefs I did
some time ago, here it is:

import itertools, weakref, sys, gc

reference_list = [] # cannot be a set, you would lose references
resource_counter = itertools.count(1)

def resource(before_closing_callback=None,
after_closing_callback=None):
private = '_resource_%s' % resource_counter.next()

def get(self):
return getattr(self, private)

def set(self, resource):
setattr(self, private, resource)
def close(ref):
if before_closing_callback:
before_closing_callback(resource)
resource.close()
if after_closing_callback:
after_closing_callback(resource)
reference_list.remove(ref)
reference_list.append(weakref.ref(self, close))

return property(get, set)

class FakeResource(object):
def __init__(self, name):
print 'opening resource %s' % name
self.name = name
def close(self):
print 'closing resource %s' % self.name
def __repr__(self):
return '<FakeResource %r>' % self.name

class Example(object):
def __init__(self):
self.resource1 = FakeResource('r1')
self.resource2 = FakeResource('r2')
def __del__(self):
print '**************'
self.resource1.close()
self.resource2.close()

def warn_before_closing(res):
sys.stdout.write('going to close %s\n' % res)

class Example2(object):

resource1 = resource(warn_before_closing)
resource2 = resource()

def __init__(self):
self.resource1 = FakeResource('r1')
self.resource2 = FakeResource('r2')
gc.set_debug(gc.DEBUG_LEAK)

#e = Example()
e = Example2()
e.e = e
del e
print reference_list

Oct 7 '07 #9
Mathias Panzenboeck <e0******@student.tuwien.ac.atwrote:
...
I only inserted them so I can see if the objects are really freed. How can
I see that without a __del__ method?
You can use weakref.ref instances with finalizer functions - see the
long post I just made on this thread for a reasonably rich and complex
example.
Alex
Oct 7 '07 #10
Mathias Panzenboeck <e0******@student.tuwien.ac.atwrote:
Marc 'BlackJack' Rintsch wrote:
``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.

Actually I have the opposite problem. The reference (to the bound method)
gets lost but it shouldn't!
weakrefs to bound methods require some subtlety, see
<http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/81253(or what
I believe is the better treatment of this recipe in the printed edition
of the Python Cookbook -- of course, being the latter's editor, I'm
biased;-).
Alex
Oct 7 '07 #11
Alex Martelli wrote:
Mathias Panzenboeck <e0******@student.tuwien.ac.atwrote:
>Marc 'BlackJack' Rintsch wrote:
>>``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.
Actually I have the opposite problem. The reference (to the bound method)
gets lost but it shouldn't!

weakrefs to bound methods require some subtlety, see
<http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/81253(or what
I believe is the better treatment of this recipe in the printed edition
of the Python Cookbook -- of course, being the latter's editor, I'm
biased;-).
Alex
Thank you, that really helped. :)
-panzi
Oct 7 '07 #12

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: William Trenker | last post by:
I'm trying to get a better grasp on the difference between weakref.ref and weakref.proxy. I've read PEP 205 and goolged for additional description information on weakrefs, but I'm not finding...
12
by: Ron Garret | last post by:
Why doesn't this work? >>> from weakref import ref >>> class C(str): pass .... >>> ref(C()) Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: cannot create weak...
0
by: Rhamphoryncus | last post by:
I know __del__ can't be relied on to do cleanup, so I'm thinking of using a globally reachable weakref to do the job, like so: import weakref class Foo: def __init__(self, *args, **kwargs):...
12
by: Russell E. Owen | last post by:
I have several situations in my code where I want a unique identifier for a method of some object (I think this is called a bound method). I want this id to be both unique to that method and also...
8
by: Kevin Little | last post by:
#!/usr/bin/env python ''' I want to dynamically add or replace bound methods in a class. I want the modifications to be immediately effective across all instances, whether created before or...
3
by: Michael Schneider | last post by:
Hello All, I am comming back to python after being away for several years. I would like to use weak refs in an observer pattern implementation. The problme that I have seems to be that...
3
by: Richard Albrecht | last post by:
I have been trying to figure out for days on how to read values from a Bound ListBox. The listBox gets the values from an Access Table. I can read values fine for Non-Bound ListBoxes, But the...
0
by: Kamilche | last post by:
''' event.py An event manager using publish/subscribe, and weakrefs. Any function can publish any event without registering it first, and any object can register interest in any event, even...
1
by: srinivasan srinivas | last post by:
HI Peter, It works will for instance and class methods. But it doesn't work for static methods. Can you tel me how to pickle static methods as well?? Thanks, Srini ----- Original Message ----...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.