By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,320 Members | 2,109 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,320 IT Pros & Developers. It's quick & easy.

Destructors and exceptions

P: n/a
Hi all

I noticed something interesting while testing some RAII concepts
ported from C++ in Python. I haven't managed to find any information
about it on the web, hence this post.

The problem is that when an exception is raised, the destruction of
locals appears to be deferred to program exit. Am I missing
something? Is this behaviour by design? If so, is there any reason
for it? The only rationale I can think of is to speed up exception
handling; but as this approach breaks many safe programming idioms, I
see it as a poor trade.

Here is the code in question:

------------------------------------------
class Foo:
def __init__(self):
print "--foo %s created" % id(self)
def __del__(self):
print "--foo %s destroyed" % id(self)

def normal_exit():
print "normal_exit starts"
x = Foo()
print "normal_exit ends"

def premature_exit():
print "premature_exit starts"
x = Foo()
return 0
print "premature_exit ends"

def exceptional_exit():
print "exceptional_exit starts"
x = Foo()
raise "oops"
print "exceptional_exit ends"

if __name__ == "__main__":
print "main block starts"
try:
normal_exit()
premature_exit()
exceptional_exit()
except:
print "exception"
print "main block ends"
------------------------------------------

The output I get is:

------------------------------------------
main block starts
normal_exit starts
--foo 141819532 created
normal_exit ends
--foo 141819532 destroyed
premature_exit starts
--foo 141819532 created
--foo 141819532 destroyed
exceptional_exit starts
--foo 141819532 created
exception
main block ends
--foo 141819532 destroyed
------------------------------------------

....which indicates to me that the destruction of the local in
exceptional_exit() only happens when the program exits. Surely it
should occur at the point at which the exception is raised?

Regards
David Turner
Jul 18 '05 #1
Share this Question
Share on Google+
16 Replies


P: n/a

"David Turner" <dk******@telkomsa.net> wrote in message
news:e2**************************@posting.google.c om...
The problem is that when an exception is raised, the destruction of
locals appears to be deferred to program exit. Am I missing
something?


When the last reference to an object disappears, the interpreter *may* but
is *not required* to forget or destruct the object, if indeed the concept
is meaningful. What happens is implementation dependent. CPython usually
cleans up immediately, which is sooner than with Jython. I am not sure
what human interpreters do. Write once, never erase storage of objects
(keeping a complete audit trail of objects created) would also be legal.

If you want to force the issue, give your class a close method, as with
file objects, and call it explicitly. Then code in try:finally:, which is
designed for this purpose.
try:
process
finally:
cleanup

Terry J. Reedy


Jul 18 '05 #2

P: n/a

"David Turner" <dk******@telkomsa.net> wrote in message
news:e2**************************@posting.google.c om...
The problem is that when an exception is raised, the destruction of
locals appears to be deferred to program exit. Am I missing
something? Is this behaviour by design? If so, is there any reason
for it? The only rationale I can think of is to speed up exception
handling; but as this approach breaks many safe programming idioms, I
see it as a poor trade.

There is no Python equivalent of C++'s "destructor garanteed to be called
upon scope exit", for a couple of reasons: scope exit only destroys
references to objects, not the objects themselves; destruction of objects is
left to the garbage collector and you have no influence on it. In
particular, the gc is not required to release resources, so finalizers (the
__del__ method, closest to C++'s destructor) may not get called. This means
__del__ is pretty much useless (AFAIMC), and you can't rely on them being
called before program exit (or ever, for that matter).

I agree, it is a real pitty that Python doesn't have a way of doing what you
mention, other than try-finally, which makes code more difficult to read. A
new specifier, e.g. "scoped", would be a required addtion to Python:
interpreter would garantee that __del__ of scoped objects would be called on
scope exit, and raise an exception if attempt to alias.

Oliver
Jul 18 '05 #3

P: n/a
Hi
If you want to force the issue, give your class a close method, as with
file objects, and call it explicitly. Then code in try:finally:, which is
designed for this purpose.
try:
process
finally:
cleanup


That's a great shame. "Finally" is of course susceptible to the
"forgetful programmer" syndrom. It also pollutes the code with what I
consider to be implementation details -- for example, the fact that a
file must be closed is a characteristic of the file object, and only
accidentally relevant to the _use_ of files. Python would be a much
stronger language if it could guarantee deterministic destruction of
at least those objects which have a __del__ method. Can anyone point
out any technical reasons why such a feature should not be included in
the language?

At any rate, I wonder if there is an another solution to the problem
of resource management. Perhaps something along the lines of Ruby
closures? Any suggestions? There has to be something more elegant
(and safer!) than "finally".

Regards
David Turner
Jul 18 '05 #4

P: n/a
David Turner wrote:
Hi

If you want to force the issue, give your class a close method, as with
file objects, and call it explicitly. Then code in try:finally:, which is
designed for this purpose.
try:
process
finally:
cleanup
That's a great shame. "Finally" is of course susceptible to the
"forgetful programmer" syndrom. It also pollutes the code with what I
consider to be implementation details -- for example, the fact that a
file must be closed is a characteristic of the file object, and only
accidentally relevant to the _use_ of files. Python would be a much
stronger language if it could guarantee deterministic destruction of

Well, there had been some discussion before. You can
wrap the open/close actions in a closure and write
something like with_file_do(myFunction).

Ciao,
Dominic
at least those objects which have a __del__ method. Can anyone point
out any technical reasons why such a feature should not be included in
the language?

At any rate, I wonder if there is an another solution to the problem
of resource management. Perhaps something along the lines of Ruby
closures? Any suggestions? There has to be something more elegant
(and safer!) than "finally".

Regards
David Turner

Jul 18 '05 #5

P: n/a
On 7 Jun 2004 07:51:23 -0700, rumours say that dk******@telkomsa.net
(David Turner) might have written:
Hi all

I noticed something interesting while testing some RAII concepts
ported from C++ in Python. I haven't managed to find any information
about it on the web, hence this post.

The problem is that when an exception is raised, the destruction of
locals appears to be deferred to program exit. Am I missing
something? Is this behaviour by design? If so, is there any reason
for it? The only rationale I can think of is to speed up exception
handling; but as this approach breaks many safe programming idioms, I
see it as a poor trade.


[snip code]

ISTR that when an exception is raised, there is a reference to the code
frame (and therefore to its locals) kept somewhere; you can access this
info through the sys.exc_info call. Have you tried running
sys.exc_clear after the exception was raised?

Apart from that, advice from others not to rely on finalisation of
objects to clean up still applies.
--
TZOTZIOY, I speak England very best,
"I have a cunning plan, m'lord" --Sean Bean as Odysseus/Ulysses
Jul 18 '05 #6

P: n/a
Hi

There is no Python equivalent of C++'s "destructor garanteed to be called
upon scope exit", for a couple of reasons: scope exit only destroys
references to objects, not the objects themselves; destruction of objects is
left to the garbage collector and you have no influence on it. In
particular, the gc is not required to release resources, so finalizers (the
__del__ method, closest to C++'s destructor) may not get called. This means
__del__ is pretty much useless (AFAIMC), and you can't rely on them being
called before program exit (or ever, for that matter).


In the documentation for the "gc" module, it states that objects with
a __del__ method are not subject to garbage collection; they are
collected using reference counting. Which means that one can rely on
locals (to which there is only one reference) being destroyed in a
predictable fashion -- but there's an exception. Section 3.1 of the
Python Reference Manual specifies that raising exceptions may keep
objects alive. What I take this to mean from an implementation point
of view is that the exception mechanism doesn't unwind scope.
Therefore, the destruction of locals in the presence of an exception
is deferred to global clean-up time when the program exits.

I can't think of any technical objections to having the exception
mechanism also release references to locals that are going to go out
of scope (unless one was planning to support resuming). Can you?

Regards
David Turner
Jul 18 '05 #7

P: n/a
On 8 Jun 2004 02:44:04 -0700, rumours say that dk******@telkomsa.net
(David Turner) might have written:
I can't think of any technical objections to having the exception
mechanism also release references to locals that are going to go out
of scope (unless one was planning to support resuming). Can you?


Debugging.
--
TZOTZIOY, I speak England very best,
"I have a cunning plan, m'lord" --Sean Bean as Odysseus/Ulysses
Jul 18 '05 #8

P: n/a
You know, I noticed this in the Python Reference Manual, p. 13, and
have been wondering about it.

"...note that catching an exception with a 'try...except' statement
may keep objects alive..."

No explanation is given, and I don't know why that's the case either.
But at least they're aware of it...HTH

--Nick

dk******@telkomsa.net (David Turner) wrote in message news:<e2**************************@posting.google. com>...
Hi all

I noticed something interesting while testing some RAII concepts
ported from C++ in Python. I haven't managed to find any information
about it on the web, hence this post.

The problem is that when an exception is raised, the destruction of
locals appears to be deferred to program exit. Am I missing
something? Is this behaviour by design? If so, is there any reason
for it? The only rationale I can think of is to speed up exception
handling; but as this approach breaks many safe programming idioms, I
see it as a poor trade.

Here is the code in question:

------------------------------------------
class Foo:
def __init__(self):
print "--foo %s created" % id(self)
def __del__(self):
print "--foo %s destroyed" % id(self)

def normal_exit():
print "normal_exit starts"
x = Foo()
print "normal_exit ends"

def premature_exit():
print "premature_exit starts"
x = Foo()
return 0
print "premature_exit ends"

def exceptional_exit():
print "exceptional_exit starts"
x = Foo()
raise "oops"
print "exceptional_exit ends"

if __name__ == "__main__":
print "main block starts"
try:
normal_exit()
premature_exit()
exceptional_exit()
except:
print "exception"
print "main block ends"
------------------------------------------

The output I get is:

------------------------------------------
main block starts
normal_exit starts
--foo 141819532 created
normal_exit ends
--foo 141819532 destroyed
premature_exit starts
--foo 141819532 created
--foo 141819532 destroyed
exceptional_exit starts
--foo 141819532 created
exception
main block ends
--foo 141819532 destroyed
------------------------------------------

...which indicates to me that the destruction of the local in
exceptional_exit() only happens when the program exits. Surely it
should occur at the point at which the exception is raised?

Regards
David Turner

Jul 18 '05 #9

P: n/a
ni***********@yahoo.com (Nick Jacobson) wrote in
news:f8**************************@posting.google.c om:
You know, I noticed this in the Python Reference Manual, p. 13, and
have been wondering about it.

"...note that catching an exception with a 'try...except' statement
may keep objects alive..."

No explanation is given, and I don't know why that's the case either.
But at least they're aware of it...HTH


When an exception is handled, you can access the stack traceback. This
contains references to all the local variables which were in scope at the
point when the exception was thrown. Normally these variables would be
destroyed when the functions return, but if there is a traceback
referencing them they stay alive as long as the traceback is accessible
which is usually until the next exception is thrown.

If you really want to be sure your objects are destroyed, then use 'del' in
the finally suite of a try..finally. This ensures that the traceback can't
be used to reference the objects that you deleted.
Jul 18 '05 #10

P: n/a
Hi
When an exception is handled, you can access the stack traceback. This
contains references to all the local variables which were in scope at the
point when the exception was thrown. Normally these variables would be
destroyed when the functions return, but if there is a traceback
referencing them they stay alive as long as the traceback is accessible
which is usually until the next exception is thrown.
Then why not unreference the traceback (and therefore destroy it and
the down-stack locals, if the exception handler hasn't made another
reference) at the end of the handling suite?

If you really want to be sure your objects are destroyed, then use 'del' in
the finally suite of a try..finally. This ensures that the traceback can't
be used to reference the objects that you deleted.


As I've pointed out elsewhere, this works but it doesn't address the
problem I'm trying to solve. The problem is that it's easier for the
user of an object (who didn't create it, and probably hasn't read the
documentation) to omit the try/finally construct. Therefore, this is
what he will tend to do. I'm looking at what I, as a designer, can do
to prevent this from happening.

Regards
David Turner
Jul 18 '05 #11

P: n/a
dk******@telkomsa.net (David Turner) wrote in
news:e2*************************@posting.google.co m:
When an exception is handled, you can access the stack traceback.
This contains references to all the local variables which were in
scope at the point when the exception was thrown. Normally these
variables would be destroyed when the functions return, but if there
is a traceback referencing them they stay alive as long as the
traceback is accessible which is usually until the next exception is
thrown.


Then why not unreference the traceback (and therefore destroy it and
the down-stack locals, if the exception handler hasn't made another
reference) at the end of the handling suite?


Fine, except that isn't what Python does now, and the current behaviour
needs to be kept for compatibility. So if that is the behaviour you want,
you have to do that explicitly yourself at the end of every exception
handler.
Jul 18 '05 #12

P: n/a
dk******@telkomsa.net (David Turner) wrote in message news:<e2**************************@posting.google. com>...
Hi all

I noticed something interesting while testing some RAII concepts
ported from C++ in Python.
AFAIK, cpp-style RAII is mostly unportable to other languages.
I haven't managed to find any information
about it on the web, hence this post.
This discussion may be helpful:
http://mail.python.org/pipermail/pyt...ch/090979.html

The problem is that when an exception is raised, the destruction of
locals appears to be deferred to program exit. Am I missing
something? Is this behaviour by design? If so, is there any reason
for it? The only rationale I can think of is to speed up exception
handling; but as this approach breaks many safe programming idioms, I
see it as a poor trade.


My impression is that many (or all?) languages with GC (especially
non-refcounting) don't guarantee deterministic destruction of objects.
I guess it's hard to have both GC and DD :) Please correct me if I am wrong.

See also "Deterministic Destruction can be a Bug" for an example when DD
can be a bad thing:
http://www.hpl.hp.com/personal/Hans_...det_destr.html

All that said, GC-enabled language still can provide some support for RAII
pattern. Consider this C# example:

using (Font f1 = new Font("Arial", 10), f2 = new Font("Arial", 12)) {
// use f1 and f2...
} // compiler will call Dispose on f1 and f2 either on exit or on exception;
// not sure about order of disposal, but you can nest "using" anyway

This saves you from trouble of coding try+finally+Dispose.
BTW, does anybody know whether MS borrowed or invented this construct?

- kv
Jul 18 '05 #13

P: n/a
"Humpty Dumpty" <ol***************@utoronto.ca> wrote in message news:<Ys********************@news20.bellglobal.com >...

There is no Python equivalent of C++'s "destructor garanteed to be called
upon scope exit", for a couple of reasons: scope exit only destroys
references to objects, not the objects themselves; destruction of objects is
left to the garbage collector and you have no influence on it. In
particular, the gc is not required to release resources, so finalizers (the
__del__ method, closest to C++'s destructor) may not get called. This means
__del__ is pretty much useless (AFAIMC), and you can't rely on them being
called before program exit (or ever, for that matter).

I agree, it is a real pitty that Python doesn't have a way of doing what you
mention, other than try-finally, which makes code more difficult to read. A
new specifier, e.g. "scoped", would be a required addtion to Python:
interpreter would garantee that __del__ of scoped objects would be called on
scope exit, and raise an exception if attempt to alias.


Is there a PEP or something for "scoped specifier"?

- kv
Jul 18 '05 #14

P: n/a
kv***********@yahoo.com (Konstantin Veretennicov) writes:
"Humpty Dumpty" <ol***************@utoronto.ca> wrote in message news:<Ys********************@news20.bellglobal.com >...

I agree, it is a real pitty that Python doesn't have a way of doing what you
mention, other than try-finally, which makes code more difficult to read. A
new specifier, e.g. "scoped", would be a required addtion to Python:
interpreter would garantee that __del__ of scoped objects would be called on
scope exit, and raise an exception if attempt to alias.


Is there a PEP or something for "scoped specifier"?


http://python.org/peps/pep-0310.html

Bernhard

--
Intevation GmbH http://intevation.de/
Skencil http://sketch.sourceforge.net/
Thuban http://thuban.intevation.org/
Jul 18 '05 #15

P: n/a
> >
Then why not unreference the traceback (and therefore destroy it and
the down-stack locals, if the exception handler hasn't made another
reference) at the end of the handling suite?


Fine, except that isn't what Python does now, and the current behaviour
needs to be kept for compatibility. So if that is the behaviour you want,
you have to do that explicitly yourself at the end of every exception
handler.


Are you sure the current behaviour needs to be kept? Isn't
referencing the traceback outside of an exception handler a little
dodgy in the first place? I'm sorry if I sound argumentative, but I
do want to understand the issues thoroughly :-).

Regards
David Turner
Jul 18 '05 #16

P: n/a
dk******@telkomsa.net (David Turner) wrote in
news:e2**************************@posting.google.c om:
>
> Then why not unreference the traceback (and therefore destroy it
> and the down-stack locals, if the exception handler hasn't made
> another reference) at the end of the handling suite?
Fine, except that isn't what Python does now, and the current
behaviour needs to be kept for compatibility. So if that is the
behaviour you want, you have to do that explicitly yourself at the
end of every exception handler.


Are you sure the current behaviour needs to be kept? Isn't
referencing the traceback outside of an exception handler a little
dodgy in the first place? I'm sorry if I sound argumentative, but I
do want to understand the issues thoroughly :-).

The documentation says:
exc_info( )

This function returns a tuple of three values that give information
about the exception that is currently being handled. The information
returned is specific both to the current thread and to the current
stack frame. If the current stack frame is not handling an exception,
the information is taken from the calling stack frame, or its caller,
and so on until a stack frame is found that is handling an exception.
Here, ``handling an exception'' is defined as ``executing or having
executed an except clause.'' For any stack frame, only information
about the most recently handled exception is accessible.


So, in fact I was wrong. The exception gets cleared when the function that
handled it returns:
import sys
def f(): .... try:
.... raise RuntimeError
.... except:
.... print sys.exc_info()
.... print sys.exc_info()
.... f() (<class exceptions.RuntimeError at 0x00864810>, <exceptions.RuntimeError
instance at 0x008EA940>, <traceback object at 0x008EA968>)
(<class exceptions.RuntimeError at 0x00864810>, <exceptions.RuntimeError
instance at 0x008EA940>, <traceback object at 0x008EA968>) def g(): .... sys.exc_clear()
.... f()
.... print sys.exc_info()
.... g() (<class exceptions.RuntimeError at 0x00864810>, <exceptions.RuntimeError
instance at 0x008EA990>, <traceback object at 0x008EA940>)
(<class exceptions.RuntimeError at 0x00864810>, <exceptions.RuntimeError
instance at 0x008EA990>, <traceback object at 0x008EA940>)
(None, None, None)

Evidently Python does clear the exception when you leave the exception
handler, it is just that the exception handler stretches a bit further than
you might expect. This of course matters more when you put the exception
handler inside a loop, or do more processing after the handler has caught
the exception. I'm not sure how much effect it would have to restrict the
handler to actually inside the except clause.

In fact further investigation shows that the exception context is saved and
restored across function calls:
def h(): .... try:
.... raise ValueError, 'h'
.... except:
.... print sys.exc_info()
.... f()
.... print sys.exc_info()
.... h() (<class exceptions.ValueError at 0x00864DB0>, <exceptions.ValueError
instance at 0x008EAAA8>, <traceback object at 0x008EA940>)
(<class exceptions.RuntimeError at 0x00864810>, <exceptions.RuntimeError
instance at 0x008EA760>, <traceback object at 0x008EAAD0>)
(<class exceptions.RuntimeError at 0x00864810>, <exceptions.RuntimeError
instance at 0x008EA760>, <traceback object at 0x008EAAD0>)
(<class exceptions.ValueError at 0x00864DB0>, <exceptions.ValueError
instance at 0x008EAAA8>, <traceback object at 0x008EA940>)


I never knew it did that.
Jul 18 '05 #17

This discussion thread is closed

Replies have been disabled for this discussion.