Generator inside a class prevent __del__ ??

[Emmanuel]

Hi,

I run across this problem, and couldn't find any solution (python 2.2.2)
:

Code :
===========
from __future__ import generators

class titi: def __init__(self):
print "init"
def __del__(self):
print "del"
def Gen(self):
yield 1
c = titi() init c = [] del
==============
Here, everything is normal...
But creating a generator :

Code :
===========
class toto: def __init__(self):
print "init"
self.Coroutine = self.Gen()
def __del__(self):
print "del"
def Gen(self):
yield 1
a = toto() init c = []

<--- Nothing there !!!
==============

I can't understand why the destructor is not called when a generator is
created, and what I should do to have a "correct" behavior.
(perhaps I missed something obvious, but I can't find it )
Thank you for any help,

Emmanuel

[Terry Reedy]

"Emmanuel" <ea*****@free.f r> wrote in message
news:40******** *******@free.fr ...

I run across this problem, and couldn't find any solution (python 2.2.2) .... Here, everything is normal...
But creating a generator :
You both defined generator function and called it to create generator
iterator.

Code :
===========

class toto: def __init__(self):
print "init"
self.Coroutine = self.Gen()
This creates a reference loop. Delete this (and correct typo below) and
'problem' will disappear.
def __del__(self):
print "del"
def Gen(self):
If you do not really use self in the resulting iterator, define this
outside of the class without self as a parameter, and problem will
disappear.
yield 1
a = toto()

did you mean 'c = toto()'?

init c = []

<--- Nothing there !!!
==============

I can't understand why the destructor is not called when a generator is
created, and what I should do to have a "correct" behavior.

Either do not create reference loop or break it with del c.Coroutine.

Terry J. Reedy

[Mark Day]

In article <40************ ***@free.fr>, Emmanuel <ea*****@free.f r>
wrote:

class toto: def __init__(self):
print "init"
self.Coroutine = self.Gen()
def __del__(self):
print "del"
def Gen(self):
yield 1
a = toto() init c = [] <--- Nothing there !!!

First of all, "a" is still referencing your toto object. I think you
meant "a = []" here. But even if you did "a = []", the destructor
still isn't called. There must still be a reference to the object. My
guess is that the generator (directly or indirectly) is referencing the
object, creating a self referential loop.

Consider the following modification that merely references a function,
and does not create a generator:

class tata: .... def __init__(self):
.... print "init"
.... self.Coroutine = self.Gen
.... def __del__(self):
.... print "del"
.... def Gen(self):
.... pass
.... a=tata() init a=[]

Here's how to break that loop:
b=tata() init b.Coroutine=Non e
b=[] del

-Mark

[Rob Nikander]

Mark Day wrote:

still isn't called. There must still be a reference to the object. My
guess is that the generator (directly or indirectly) is referencing the
object, creating a self referential loop.

Python has a garbage collector that will try to find these objects with
cyclic references.

from test import *

a = toto() init a = None
import gc
gc.garbage [] gc.collect() 4 gc.garbage [<test.toto instance at 0x81cb78c>]

I checked out the documentation for that gc.garbage list and it says
that the collector can't free objects in cycles if the cyles have
objects that have __del__ methods. So it puts them in this list.

I wonder what other garbage collectors do in this situation? Anyone
know? Java?

Rob

[Duncan Booth]

Rob Nikander <rn************ *@adelphia.net> wrote in
news:kN******** ************@ad elphia.com:

I checked out the documentation for that gc.garbage list and it says
that the collector can't free objects in cycles if the cyles have
objects that have __del__ methods. So it puts them in this list.

I wonder what other garbage collectors do in this situation? Anyone
know? Java?

Most garbage collectors will do peculiar things if you have destructors or
finalizers in the objects. The problem is that if two objects with
finalizers reference each other there is no correct order to release the
objects that will guarantee that the other object still exists, so the
system either has to choose an arbitrary order, or refuse to call the
finalizers.

The .Net garbage collector is typical. Objects may have finalizers, and
these finalizers are called as part of the garbage collection. The system
guarantees that any finalizer is called exactly 0 or more times --- usually
it is called once when the object is garbage collected, but if the object
is never collected it may not be called at all, and if the object
resurrects itself (e.g. during the finalizer it assigns itself to a global
variable) the finalizer could be called more than once.

A separate thread pool is used for finalizers, so your finalizer could be
called while a user thread is executing a method on the object, and two
objects which refer to each other could have their finalizers called in any
order, or even simultaneously on separate threads. Effectively, this makes
finalizers useless in all but the most obscure situations.

When resources need to be released you should try to do it explicitly. In
..Net this is handled by the Dispose() method, and the finalizer can then
either try calling Dispose() if it has not yet been called, or could try
logging an error although even that may be problematic from a finalizer.

[Emmanuel]

Terry Reedy a écrit :

"Emmanuel" <ea*****@free.f r> wrote in message
news:40******** *******@free.fr ...

I run across this problem, and couldn't find any solution (python 2.2.2) ...

Here, everything is normal...
But creating a generator :

You both defined generator function and called it to create generator
iterator.

Yes, I don't have all the generators vocabulary yet...

Code :
===========

>> class toto: def __init__(self):
print "init"
self.Coroutine = self.Gen()

This creates a reference loop. Delete this (and correct typo below) and
'problem' will disappear.

def __del__(self):
print "del"
def Gen(self):
If you do not really use self in the resulting iterator, define this
outside of the class without self as a parameter, and problem will
disappear.

I didn't use self in order to provide a simple example. In my real class, self
is used...

yield 1

>> a = toto()

did you mean 'c = toto()'?

Yes, sorry for that...

init

>> c = []

<--- Nothing there !!!
==============

I can't understand why the destructor is not called when a generator is
created, and what I should do to have a "correct" behavior.

Either do not create reference loop or break it with del c.Coroutine.

Terry J. Reedy

Thank you very much for your answer, but I'm still not sure I understand it.
If I understand your words right, creating self.Coroutine as an iterator on
the generator function will create a reference on self, so if I want to use a
generator in a class ( and I really want to ), I must delete explicitly the
iterator before I destroy the object.

Trouble is, I _would_ like not to care about the lifetime of the object, and I
don't know where it will be destroyed.
Should I encapsulate this object in another one, like this :

import toto

class TotoCapsule:
def __init__( self ):
self.toto = toto.toto()
def __del__(self):
del self.toto.Corou tine
self.toto = None

And use TotoCapsule ?
But it means I have to write a lot of more code to access toto's method.
Is there a pattern I missed to dea l with that ?

Thanks a lot,

Emmanuel

[Terry Reedy]

"Emmanuel" <ea*****@free.f r> wrote in message
news:40******** *******@free.fr ...

Terry Reedy a écrit :

>>> class toto:
def __init__(self):
print "init"
self.Coroutine = self.Gen()

This creates a reference loop. Delete this (and correct typo below) and 'problem' will disappear.

To amplify: the usual idiom for an instance-associated generator is to name
the generator function (method) __iter__ (with one param, self) and to
create and get a reference to the generator via iter() or let the for loop
mechanism do so for you.

c = C(*args)
cgen =iter(c)

Then there is no reference loop. And you can pass around the cgen object
just like any other. If you only need the instance after initialization to
get the generator and you only need one generator for the instance, then
combine the two lines into

cgen = iter(C(*args))

and the *only* reference to the instance is the one in the generator, which
will disappear at the end of a for loop or with an explicit 'del cgen'.

There is also the question whether you actually *need* to get rid of the
object while the program is still running instead of just letting the
program finish and clean up.

Terry J. Reedy

[Andrew Bennetts]

On Wed, Apr 21, 2004 at 02:53:33PM +0200, Emmanuel wrote:

Trouble is, I _would_ like not to care about the lifetime of the object, and I
don't know where it will be destroyed.

Then don't use __del__. Python can and will automatically collect cycles
when the objects *don't* define __del__ methods.

Out of curiousity, why are you defining __del__ anyway?

-Andrew.

[Emmanuel]

Andrew Bennetts a écrit :

On Wed, Apr 21, 2004 at 02:53:33PM +0200, Emmanuel wrote:

Trouble is, I _would_ like not to care about the lifetime of the object, and I
don't know where it will be destroyed.

Then don't use __del__. Python can and will automatically collect cycles
when the objects *don't* define __del__ methods.

Out of curiousity, why are you defining __del__ anyway?

-Andrew.

I don't want to use __del__, but I suspected I had an issue with the destruction of
my objects, and used a log in __del__ to monitor the destruction.

But defining __del__ has also a lot of valuable utilisation, or so I think...

Emmanuel

[Joe Mason]

In article <40************ ***@free.fr>, Emmanuel wrote:

Thank you very much for your answer, but I'm still not sure I understand it.
If I understand your words right, creating self.Coroutine as an iterator on
the generator function will create a reference on self, so if I want to use a
generator in a class ( and I really want to ), I must delete explicitly the
iterator before I destroy the object.

Trouble is, I _would_ like not to care about the lifetime of the object, and I
don't know where it will be destroyed.

Try looking up "weakref". (I've never used them myself, so I don't know
the exact syntax.)

Joe