473,224 Members | 1,344 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,224 software developers and data experts.

weakrefs and bound methods

Hi.

I have a problem with weak refs and bound methods. The best explanation for the
problem is a short bit of code. I have the three classes Wrapper, Foo and Bar:

import weakref

class Wrapper(object):
def __init__(self,x):
self.x = weakref.ref(x)

def __call__(self,*args,**kwargs):
x = self.x()
if x is None:
print "lost reference"
else:
return x(*args,**kwargs)

class Foo(object):
def __init__(self):
self._methods = set()
self._methods.add(Wrapper(self._foo))

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Foo"

class Bar(object):
def __init__(self):
self._methods = set()
self._methods.add(self._foo)

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Bar"

Now look what happens when I do this:
>>f=Foo()
f.callMethods()
lost reference
>>del f
del Foo
>>b=Bar()
b.callMethods()
_foo
>>del b
Foo looses the reference to its method but Bar on the other hand has a refloop and
never gets deleted.

Does anyone know what happens here? How can I write such classes without refloop and
without lost reference?

I'm using:
Python 2.5.1 (r251:54863, May 2 2007, 16:56:35)
[GCC 4.1.2 (Ubuntu 4.1.2-0ubuntu4)] on linux2

-panzi
Oct 7 '07 #1
11 1442
When I change the class Wrapper to following, the class Foo works:

class Wrapper(object):
def __init__(self,x):
self.func_name = x.func_name
self.x = weakref.ref(x.im_self)

def __call__(self,*args,**kwargs):
x = self.x()
if x is None:
print "lost reference"
else:
return getattr(x,self.func_name)(*args,**kwargs)
But that's ugly. Any better idea?
Oct 7 '07 #2
On Sun, 07 Oct 2007 16:51:33 +0200, Mathias Panzenboeck wrote:
import weakref

class Wrapper(object):
def __init__(self,x):
self.x = weakref.ref(x)

def __call__(self,*args,**kwargs):
x = self.x()
if x is None:
print "lost reference"
else:
return x(*args,**kwargs)

class Foo(object):
def __init__(self):
self._methods = set()
self._methods.add(Wrapper(self._foo))

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Foo"

class Bar(object):
def __init__(self):
self._methods = set()
self._methods.add(self._foo)

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Bar"

Now look what happens when I do this:
>>>f=Foo()
f.callMethods()
lost reference
>>>del f
del Foo
>>>b=Bar()
b.callMethods()
_foo
>>>del b

Foo looses the reference to its method but Bar on the other hand has a refloop and
never gets deleted.
``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.

Drop all those `__del__()` methods as they prevent the garbage collector
from collecting "cycles".

Ciao,
Marc 'BlackJack' Rintsch
Oct 7 '07 #3
On Oct 7, 12:26 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.netwrote:
Drop all those `__del__()` methods as they prevent the garbage collector
from collecting "cycles".
I fully agree and I will add that __del__ methods are always
a bad idea. Also notice that recently Raymond Hetting said in
this list that he wanted to submit a PEP to remove __del__ from
Python 3000 (I don't know if this will ever happen tough).

Michele Simionato

Oct 7 '07 #4
Marc 'BlackJack' Rintsch wrote:
On Sun, 07 Oct 2007 16:51:33 +0200, Mathias Panzenboeck wrote:
>import weakref

class Wrapper(object):
def __init__(self,x):
self.x = weakref.ref(x)

def __call__(self,*args,**kwargs):
x = self.x()
if x is None:
print "lost reference"
else:
return x(*args,**kwargs)

class Foo(object):
def __init__(self):
self._methods = set()
self._methods.add(Wrapper(self._foo))

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Foo"

class Bar(object):
def __init__(self):
self._methods = set()
self._methods.add(self._foo)

def _foo(self):
print "_foo"

def callMethods(self):
for method in self._methods:
method()

def __del__(self):
print "del Bar"

Now look what happens when I do this:
>>>>f=Foo()
f.callMethods()
lost reference
>>>>del f
del Foo
>>>>b=Bar()
b.callMethods()
_foo
>>>>del b
>
Foo looses the reference to its method but Bar on the other hand has a refloop and
never gets deleted.

``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.
Why is it then that f (instance of Foo) really gets deleted? (__del__ is called)
Drop all those `__del__()` methods as they prevent the garbage collector
from collecting "cycles".

Ciao,
Marc 'BlackJack' Rintsch
I only inserted them so I can see if the objects are really freed. How can I see that
without a __del__ method?
Thanks so far,
panzi
Oct 7 '07 #5
Marc 'BlackJack' Rintsch wrote:
``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.
Actually I have the opposite problem. The reference (to the bound method)
gets lost but it shouldn't!
-panzi
Oct 7 '07 #6
On Sun, 07 Oct 2007 16:38:23 +0000, Michele Simionato wrote:
On Oct 7, 12:26 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.netwrote:
>Drop all those `__del__()` methods as they prevent the garbage
collector from collecting "cycles".

I fully agree and I will add that __del__ methods are always a bad idea.
Always?

I recently wrote a bit of code where I needed to check that releasing the
first object in a tree-like structure would allow Python to garbage
collect all the other objects in a tree. I thought it would, but I wanted
to be sure ("don't guess, test"), so I wrote a simple class, gave it a
__del__ method that just printed self, inserted them in the tree, and
then deleted the first one.

Worked like a charm.

Without __del__, what should I have done to test that my code was
deleting objects and not leaking memory?

What should I do when my objects need to perform some special processing
when they are freed, if I shouldn't use __del__?

--
Steven.
Oct 7 '07 #7
Mathias Panzenboeck wrote:
Marc 'BlackJack' Rintsch wrote:
>``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.

Actually I have the opposite problem. The reference (to the bound method)
gets lost but it shouldn't!
Ahh, so you expected that ``Wrapper(self._foo)`` would not immediately
lose the reference? It will, because every time you write
``self._foo``, a new bound method is created::
>>class C(object):
... def foo(self):
... pass
...
>>f = C.foo
g = C.foo
id(f), id(g)
(14931448, 14891008)

Thus, there is only the one reference to the bound method, and by
wrapping it in a weakref, you are allowing it to disappear immediately::
>>x = weakref.ref(C.foo)
print x()
None

What behavior do you want here? That is, when were you hoping that the
bound method would disappear?

STeVe
Oct 7 '07 #8
On Oct 7, 1:14 pm, Steven D'Aprano <st...@REMOVE-THIS-
cybersource.com.auwrote:
On Sun, 07 Oct 2007 16:38:23 +0000, Michele Simionato wrote:
On Oct 7, 12:26 pm, Marc 'BlackJack' Rintsch <bj_...@gmx.netwrote:
Drop all those `__del__()` methods as they prevent the garbage
collector from collecting "cycles".
I fully agree and I will add that __del__ methods are always a bad idea.

Always?

I recently wrote a bit of code where I needed to check that releasing the
first object in a tree-like structure would allow Python to garbage
collect all the other objects in a tree. I thought it would, but I wanted
to be sure ("don't guess, test"), so I wrote a simple class, gave it a
__del__ method that just printed self, inserted them in the tree, and
then deleted the first one.

Worked like a charm.

Without __del__, what should I have done to test that my code was
deleting objects and not leaking memory?

What should I do when my objects need to perform some special processing
when they are freed, if I shouldn't use __del__?
The best thing is to use explicit resource management,
for instance with a try .. finally or with the "with" statement
in Python 2.5. The next best thing is to use weak references.
I have some code for various experiments with wearefs I did
some time ago, here it is:

import itertools, weakref, sys, gc

reference_list = [] # cannot be a set, you would lose references
resource_counter = itertools.count(1)

def resource(before_closing_callback=None,
after_closing_callback=None):
private = '_resource_%s' % resource_counter.next()

def get(self):
return getattr(self, private)

def set(self, resource):
setattr(self, private, resource)
def close(ref):
if before_closing_callback:
before_closing_callback(resource)
resource.close()
if after_closing_callback:
after_closing_callback(resource)
reference_list.remove(ref)
reference_list.append(weakref.ref(self, close))

return property(get, set)

class FakeResource(object):
def __init__(self, name):
print 'opening resource %s' % name
self.name = name
def close(self):
print 'closing resource %s' % self.name
def __repr__(self):
return '<FakeResource %r>' % self.name

class Example(object):
def __init__(self):
self.resource1 = FakeResource('r1')
self.resource2 = FakeResource('r2')
def __del__(self):
print '**************'
self.resource1.close()
self.resource2.close()

def warn_before_closing(res):
sys.stdout.write('going to close %s\n' % res)

class Example2(object):

resource1 = resource(warn_before_closing)
resource2 = resource()

def __init__(self):
self.resource1 = FakeResource('r1')
self.resource2 = FakeResource('r2')
gc.set_debug(gc.DEBUG_LEAK)

#e = Example()
e = Example2()
e.e = e
del e
print reference_list

Oct 7 '07 #9
Mathias Panzenboeck <e0******@student.tuwien.ac.atwrote:
...
I only inserted them so I can see if the objects are really freed. How can
I see that without a __del__ method?
You can use weakref.ref instances with finalizer functions - see the
long post I just made on this thread for a reasonably rich and complex
example.
Alex
Oct 7 '07 #10
Mathias Panzenboeck <e0******@student.tuwien.ac.atwrote:
Marc 'BlackJack' Rintsch wrote:
``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.

Actually I have the opposite problem. The reference (to the bound method)
gets lost but it shouldn't!
weakrefs to bound methods require some subtlety, see
<http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/81253(or what
I believe is the better treatment of this recipe in the printed edition
of the Python Cookbook -- of course, being the latter's editor, I'm
biased;-).
Alex
Oct 7 '07 #11
Alex Martelli wrote:
Mathias Panzenboeck <e0******@student.tuwien.ac.atwrote:
>Marc 'BlackJack' Rintsch wrote:
>>``del b`` just deletes the name `b`. It does not delete the object.
There's still the name `_` bound to it in the interactive interpreter.
`_` stays bound to the last non-`None` result in the interpreter.
Actually I have the opposite problem. The reference (to the bound method)
gets lost but it shouldn't!

weakrefs to bound methods require some subtlety, see
<http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/81253(or what
I believe is the better treatment of this recipe in the printed edition
of the Python Cookbook -- of course, being the latter's editor, I'm
biased;-).
Alex
Thank you, that really helped. :)
-panzi
Oct 7 '07 #12

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: William Trenker | last post by:
I'm trying to get a better grasp on the difference between weakref.ref and weakref.proxy. I've read PEP 205 and goolged for additional description information on weakrefs, but I'm not finding...
12
by: Ron Garret | last post by:
Why doesn't this work? >>> from weakref import ref >>> class C(str): pass .... >>> ref(C()) Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: cannot create weak...
0
by: Rhamphoryncus | last post by:
I know __del__ can't be relied on to do cleanup, so I'm thinking of using a globally reachable weakref to do the job, like so: import weakref class Foo: def __init__(self, *args, **kwargs):...
12
by: Russell E. Owen | last post by:
I have several situations in my code where I want a unique identifier for a method of some object (I think this is called a bound method). I want this id to be both unique to that method and also...
8
by: Kevin Little | last post by:
#!/usr/bin/env python ''' I want to dynamically add or replace bound methods in a class. I want the modifications to be immediately effective across all instances, whether created before or...
3
by: Michael Schneider | last post by:
Hello All, I am comming back to python after being away for several years. I would like to use weak refs in an observer pattern implementation. The problme that I have seems to be that...
3
by: Richard Albrecht | last post by:
I have been trying to figure out for days on how to read values from a Bound ListBox. The listBox gets the values from an Access Table. I can read values fine for Non-Bound ListBoxes, But the...
0
by: Kamilche | last post by:
''' event.py An event manager using publish/subscribe, and weakrefs. Any function can publish any event without registering it first, and any object can register interest in any event, even...
1
by: srinivasan srinivas | last post by:
HI Peter, It works will for instance and class methods. But it doesn't work for static methods. Can you tel me how to pickle static methods as well?? Thanks, Srini ----- Original Message ----...
0
by: veera ravala | last post by:
ServiceNow is a powerful cloud-based platform that offers a wide range of services to help organizations manage their workflows, operations, and IT services more efficiently. At its core, ServiceNow...
0
by: VivesProcSPL | last post by:
Obviously, one of the original purposes of SQL is to make data query processing easy. The language uses many English-like terms and syntax in an effort to make it easy to learn, particularly for...
3
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 3 Jan 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). For other local times, please check World Time Buddy In...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.