473,327 Members | 1,936 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,327 software developers and data experts.

It is __del__ calling twice for some instances?

Hello!

It is correct behaviour for python to call __del__ on some
identity of a class object more than once?

In brief I shall describe a situation. Sorry for my english.

For debugin purposes I'm put in my module global counters
for counting __init__ and __del__ calls.

This is a sample code for clearness:
-------------------------------------------------------
init_cnt = 0
del_cnt = 0

class foo:
def __init__(self):
global init_cnt
init_cnt += 1

def __del__(self):
global del_cnt
del_cnt += 1

def stat():
print "init_cnt = %d" % init_cnt
print "del_cnt = %d" % del_cnt
print "difference = %d" % init_cnt-del_cnt
-------------------------------------------------------
And the result of a stat() call in some moment of time
looks like so:

init_cnt = 6233
del_cnt = 6234
difference = -1

It is __del__ called twice for some instance?

Thanks in advance!
--
GMT More Then ...
Aug 16 '06 #1
10 3064
Max Yuzhakov writes:

MY print "difference = %d" % init_cnt-del_cnt

Little correction.

print "difference = %d" % (init_cnt-del_cnt)
--
GMT More Then ...
Aug 16 '06 #2
Max Yuzhakov wrote:
This is a sample code for clearness:
that code snippet doesn't create any foo() instances, what I can see...

</F>

Aug 16 '06 #3
Max Yuzhakov wrote:
It is correct behaviour for python to call __del__ on some
identity of a class object more than once?
Not with the code which you gave as an example, but in the general case
yes, the only guarantee that Python gives about the __del__ method on an
instance is that it will be called zero, one or more than one times during
the run of the program. In practice there are various situations where
__del__ will not be called, but it is only called multiple times if you
resurrect the object during a call to __del__.

The output from your stat function could, of course, also be generated by
creating and destroying lots of foo objects in another thread. If a foo was
both created and destroyed between the first two print statements, and
another one was created and destroyed in the middle of the evaluation of
the last print statement then you could see the output you described
without any multiple __del__ calls in the same object.

You should post a working code sample which generates your output if you
want a more useful answer.

Aug 16 '06 #4
Duncan Booth ÐÉÛÅÔ:

DB Not with the code which you gave as an example, but in the general case
DB yes, the only guarantee that Python gives about the __del__ method on an
DB instance is that it will be called zero, one or more than one times during
DB the run of the program. In practice there are various situations where
DB __del__ will not be called, but it is only called multiple times if you
DB resurrect the object during a call to __del__.

No, only presented code in __del__ do
increment of global variable del_cnt.

DB The output from your stat function could, of course, also be generated by
DB creating and destroying lots of foo objects in another thread. If a foo was
DB both created and destroyed between the first two print statements, and
DB another one was created and destroyed in the middle of the evaluation of
DB the last print statement then you could see the output you described
DB without any multiple __del__ calls in the same object.

No, it's a single-threaded code.

DB You should post a working code sample which generates your output if you
DB want a more useful answer.

I'm sorry for mess with a code. I have forgotten to write, that is
only fragments for demonstration of essence of a question.

Real code in this module has 880 lines and I shall not
abuse your time. I shall try to solve a problem by debuger.

And thank for Your answer!
--
GMT More Then ...
Aug 16 '06 #5
Duncan Booth wrote:

DB You should post a working code sample which generates your output if you
DB want a more useful answer.

Hello!

Today I have found a compact variant of a code which shows my question:
-----------------------------------------------------------------------
#!/usr/local/bin/python -d
# -*- coding: koi8-u -*-

class foo:
def __init__(self, other):
self.other = other

global ini_cnt
ini_cnt +=1

def __del__(self):
global del_cnt
del_cnt +=1

def stat():
print "-"*20
print "ini_cnt = %d" % ini_cnt
print "del_cnt = %d" % del_cnt
print "difference = %d" % (ini_cnt-del_cnt)

ini_cnt = 0
del_cnt = 0
loop_cnt = 75

a = foo(None)

for i in xrange(loop_cnt):
a = foo(a)

stat()
a = None
stat()
-----------------------------------------------------------------------

And result is:
--------------------
ini_cnt = 76
del_cnt = 0
difference = 76
--------------------
ini_cnt = 76
del_cnt = 77
difference = -1

Why for some instance __del__ called twice?
Such behaviour of __del__ seems to me unpredictable.

Thanks for Your attention!
--
GMT More Then ...
Aug 17 '06 #6
Max Yuzhakov wrote:
Why for some instance __del__ called twice?
Such behaviour of __del__ seems to me unpredictable.
Here's a slightly modified version of your code. The 51st object destroyed
gets its __del__ method called twice. It doesn't matter how long your loop
is, every 50th object gets special treatment (or perhaps it is the 49th).

The good news is that there is a way to stop it happening: just add an
explicit "del self.other" at the end of __del__. The bad news is that if
your list is too long that will cause a flood of error messages and won't
call the destructors at all past the first 1000.

As to why it happens, there is a mechanism in Python to stop unlimited
stack being used when objects are freed: when the stack gets too deep then
instead of being released, the Py_DECREF call puts the object into a
trashcan list and the objects aren't released until the stack has unwound.
It looks like there must be a bug round the trashcan mechanism somewhere.

BTW, the behaviour is completely different if you use a new style class,
but still somewhat bizarre: for new style classes only the first 25 objects
get freed when you clear a, the remainder are only released by the garbage
collector.

#!/usr/local/bin/python -d
# -*- coding: koi8-u -*-

class foo:
def __init__(self, other):
self.other = other
self._deleted = False

global ini_cnt
ini_cnt +=1

def __del__(self):
if self._deleted:
print "aargh!"
self._deleted = True
global del_cnt
del_cnt +=1
print "del",del_cnt,"at",id(self)

def stat():
print "-"*20
print "ini_cnt = %d" % ini_cnt
print "del_cnt = %d" % del_cnt
print "difference = %d" % (ini_cnt-del_cnt)

ini_cnt = 0
del_cnt = 0
loop_cnt = 54

a = foo(None)

for i in xrange(loop_cnt):
a = foo(a)

stat()
a = None
stat()

Aug 18 '06 #7
Duncan Booth wrote:
As to why it happens, there is a mechanism in Python to stop unlimited
stack being used when objects are freed: when the stack gets too deep
then instead of being released, the Py_DECREF call puts the object
into a trashcan list and the objects aren't released until the stack
has unwound. It looks like there must be a bug round the trashcan
mechanism somewhere.
I figured out what is going on in the code to deallocate an old-style class
instance:

The reference count is temporarily incremented.

If the class has a __del__ method then a descriptor is created for the
method and called. When the call returns, the descriptor is released.

Then the object itself is released using special code to avoid a recursive
call to the deallocator.

However, if the trashcan mechanism is invoked by the attempt to release the
descriptor, it actually queues the descriptor in the trashcan. Since the
descriptor contains a reference to the object it has effectively
resurrected it. This means the special code to avoid the recursive call
simply decrements the reference count but does not release anything (the
object has been resurrected by the descriptor). When the descriptor is
later released the __del__ method is triggered a second time.
Aug 18 '06 #8

Duncan Booth wrote:
Duncan Booth wrote:
As to why it happens, there is a mechanism in Python to stop unlimited
stack being used when objects are freed: when the stack gets too deep
then instead of being released, the Py_DECREF call puts the object
into a trashcan list and the objects aren't released until the stack
has unwound. It looks like there must be a bug round the trashcan
mechanism somewhere.

I figured out what is going on in the code to deallocate an old-style class
instance:

The reference count is temporarily incremented.

If the class has a __del__ method then a descriptor is created for the
method and called. When the call returns, the descriptor is released.

Then the object itself is released using special code to avoid a recursive
call to the deallocator.

However, if the trashcan mechanism is invoked by the attempt to release the
descriptor, it actually queues the descriptor in the trashcan. Since the
descriptor contains a reference to the object it has effectively
resurrected it. This means the special code to avoid the recursive call
simply decrements the reference count but does not release anything (the
object has been resurrected by the descriptor). When the descriptor is
later released the __del__ method is triggered a second time.
This looks like some good code to add to the python unit tests.
>From your description, it appears the problem is that the object is
placed in the trashcan after calling __del__ once. Perhaps the choice
to place it in the trashcan could be made instead of calling __del__
the first time, rather than after calling __del__?

Regards,
Pat

Aug 18 '06 #9
Duncan Booth wrote:

DB I figured out what is going on in the code to deallocate an old-style class
DB instance:
DB>
DB The reference count is temporarily incremented.
DB>
DB If the class has a __del__ method then a descriptor is created for the
DB method and called. When the call returns, the descriptor is released.
DB>
DB Then the object itself is released using special code to avoid a recursive
DB call to the deallocator.
DB>
DB However, if the trashcan mechanism is invoked by the attempt to release the
DB descriptor, it actually queues the descriptor in the trashcan. Since the
DB descriptor contains a reference to the object it has effectively
DB resurrected it. This means the special code to avoid the recursive call
DB simply decrements the reference count but does not release anything (the
DB object has been resurrected by the descriptor). When the descriptor is
DB later released the __del__ method is triggered a second time.

Thank You for so detailed explanation!
--
GMT More Then ...
Aug 18 '06 #10
Duncan Booth wrote:

DB BTW, the behaviour is completely different if you use a new style class,
DB but still somewhat bizarre: for new style classes only the first 25 objects
DB get freed when you clear a, the remainder are only released by the garbage
DB collector.

If to add the third call of stat() after the second,
the result became such:

--------------------
ini_cnt = 500001
del_cnt = 0
difference = 500001
--------------------
ini_cnt = 500001
del_cnt = 25
difference = 499976
--------------------
ini_cnt = 500001
del_cnt = 500001
difference = 0

Preceding call to gc.disable() has no influence on result.
--
GMT More Then ...
Aug 18 '06 #11

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Kepes Krisztian | last post by:
Hi ! I very wonder, when I get exp. in java with GC. I'm Delphi programmer, so I get used to destructorin objects. In Java the final method is not same, but is like to destructor (I has...
0
by: John Hunter | last post by:
I have a class that uses some extension code I have written and I am trying to track down some memory leaks, which I presume to be in my extension code. The class is question is in a python...
2
by: David MacQuigg | last post by:
I'm __del__() to decrement a count of instances of a class whenever an instance is deleted, and I'm getting very erratic behavior. This is in CPython, which I am told has no problem with reference...
4
by: Baoqiu Cui | last post by:
Today I was playing with a small Python program using Python 2.4 on Cygwin (up-to-date version, on Windows XP), but ran into a strange error on the following small program (named bug.py): ...
0
by: phil | last post by:
> you haven't answered my question, btw: why are you using __del__ > to do something that the garbage collector should do for you? After more reading it seems I have made an ass of my self on...
13
by: Torsten Bronger | last post by:
Hallöchen! When my __del__ methods are called because the program is being terminated, I experience difficulties in calling functions that I need for a clean shutdown of my instances. So far,...
14
by: Chris Curvey | last post by:
I need to ensure that there is only one instance of my python class on my machine at a given time. (Not within an interpreter -- that would just be a singleton -- but on the machine.) These...
8
by: Gregor Horvath | last post by:
Hi, I do not understand why __del__ does not get executed in the following example. test.py: #!/usr/bin/python class A(object): def __init__(self):
6
by: George Sakkis | last post by:
I'm baffled with a situation that involves: 1) an instance of some class that defines __del__, 2) a thread which is created, started and referenced by that instance, and 3) a weakref proxy to the...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.