Garbage collector and threads

In article <hg*******************@nnrp1.uunet.ca>,
Nicolas Fleury <ni******@yahoo.com_remove_the_> wrote:

I just discovered, correct me if I'm wrong, that as long as a
threading.Thread is running, its destructor will not be called, because
it is referring to himself. So if I have something like:

class MyThread(threading.Thread):
def __init__(self):
self.cancelEvent = threading.Event()
threading.Thread.__init__(self)
def __del__(self):
self.cancel()
def run(self):
self.cancelEvent.wait()
def cancel(self):
self.cancelEvent.set()

I must call cancel from the outside if I want the destructor to be
called (note that I don't want deamon threads). I can make a wrapper
over threading.Thread to have the behaviour I want: have a thread with
__del__ called when it is not referred by another thread. But my
question is, and I don't have an overall vision of the issue at all,
should that be the default behaviour anyway?

Yes, Don't Do That. Do not rely on finalizers for external resources
(such as threads).

Note that your understanding of Python's memory management could use some
improvement: GC *never* runs on objects containing __del__
--
Aahz (aa**@pythoncraft.com) <*> http://www.pythoncraft.com/

"Do not taunt happy fun for loops. Do not change lists you are looping over."
--Remco Gerlich, comp.lang.python

Aahz wrote:

Yes, Don't Do That. Do not rely on finalizers for external resources
(such as threads).
Why? Isn't a kind of RAII applied to Python?
Note that your understanding of Python's memory management could use some
improvement: GC *never* runs on objects containing __del__

Well, I guess you're right. What I mean is the reasons behind the call
to __del__, whatever it is ref-counting or something else. Wouldn't a
class like CancellableThread, with a sub-class implemented cancel method
called when the object is not referred in any other thread be useful?

Regards,
Nicolas

In article <1E*******************@nnrp1.uunet.ca>,
Nicolas Fleury <ni******@yahoo.com_remove_the_> wrote:

Aahz wrote:

Yes, Don't Do That. Do not rely on finalizers for external resources
(such as threads).

Why? Isn't a kind of RAII applied to Python?

What's "RAII"?

Note that your understanding of Python's memory management could use some
improvement: GC *never* runs on objects containing __del__

Well, I guess you're right. What I mean is the reasons behind the call
to __del__, whatever it is ref-counting or something else. Wouldn't a
class like CancellableThread, with a sub-class implemented cancel method
called when the object is not referred in any other thread be useful?

Perhaps, but threading needs references to created threads in order to
implement its functionality. You might try building your own thread
constructs using ``thread`` directly.
--
Aahz (aa**@pythoncraft.com) <*> http://www.pythoncraft.com/

"Do not taunt happy fun for loops. Do not change lists you are looping over."
--Remco Gerlich, comp.lang.python

aa**@pythoncraft.com (Aahz) writes:

In article <1E*******************@nnrp1.uunet.ca>,
Nicolas Fleury <ni******@yahoo.com_remove_the_> wrote:

Aahz wrote:

Yes, Don't Do That. Do not rely on finalizers for external resources
(such as threads).

Why? Isn't a kind of RAII applied to Python?

What's "RAII"?

Resource Aquisition Is Initialization. Works in C++. Doesn't really
in Python.

Cheers,
mwh

--
Need to Know is usually an interesting UK digest of things that
happened last week or might happen next week. [...] This week,
nothing happened, and we don't care.
-- NTK Now, 2000-12-29, http://www.ntk.net/

Aahz wrote:

What's "RAII"?

"Resource Acquisition Is Initialization"
It's a term frequently used in C++ to describe the use of constructor
and destructor for resource allocation/deallocation.

to __del__, whatever it is ref-counting or something else. Wouldn't a
class like CancellableThread, with a sub-class implemented cancel method
called when the object is not referred in any other thread be useful?

Perhaps, but threading needs references to created threads in order to
implement its functionality. You might try building your own thread
constructs using ``thread`` directly.

I still don't know how to implement it. I can do something very simple
like:

class Canceller:
def __init__(self, thread):
self.thread = thread
def __del__(self):
self.thread.cancel()
self.thread.join()

but I can't merge the concept with a Thread class, because the self
passed to run would be a reference to the object... I don't know if a
language extension would be worth the effort or if there's a cool
solution I'm missing, but what is sure is that I write a lot of code to
cancel threads in finalizers of parent objects. I don't want deamon
threads so that everything is stopped cleanly, but at the same time it
is error-prone in a GUI application, because I risk to have the process
still running after a quit.

I just want a way to garantee that all my threads are cancelled when the
main thread is finished.

Regards,
Nicolas

In article <6u*******************@nnrp1.uunet.ca>,
Nicolas Fleury <ni******@yahoo.com_remove_the_> wrote:

Aahz wrote:

What's "RAII"?
"Resource Acquisition Is Initialization"
It's a term frequently used in C++ to describe the use of constructor
and destructor for resource allocation/deallocation.

As Michael Hudson said, it doesn't work that way in Python, because all
Python objects are global (and I mean truly global here, not just module
global). Essentially, all objects are heap objects that can be traced
by crawling through Python's internals.

Perhaps, but threading needs references to created threads in order to
implement its functionality. You might try building your own thread
constructs using ``thread`` directly.

I still don't know how to implement it. I can do something very simple
like:

class Canceller:
def __init__(self, thread):
self.thread = thread
def __del__(self):
self.thread.cancel()
self.thread.join()

but I can't merge the concept with a Thread class, because the self
passed to run would be a reference to the object... I don't know if a
language extension would be worth the effort or if there's a cool
solution I'm missing, but what is sure is that I write a lot of code to
cancel threads in finalizers of parent objects. I don't want deamon
threads so that everything is stopped cleanly, but at the same time it
is error-prone in a GUI application, because I risk to have the process
still running after a quit.

Right. As I said, you have to use the ``thread`` module directly; you
can't use ``threading``.
I just want a way to garantee that all my threads are cancelled when the
main thread is finished.

So hold references to them and call the ``cancel()`` method yourself.
--
Aahz (aa**@pythoncraft.com) <*> http://www.pythoncraft.com/

"Do not taunt happy fun for loops. Do not change lists you are looping over."
--Remco Gerlich, comp.lang.python

Aahz wrote:

As Michael Hudson said, it doesn't work that way in Python, because all
Python objects are global (and I mean truly global here, not just module
global). Essentially, all objects are heap objects that can be traced
by crawling through Python's internals.
I understand that. RAII is often used when speaking of scoped
variables. In my case I was talking about global scope in main thread.
Right. As I said, you have to use the ``thread`` module directly; you
can't use ``threading``.

But even if I use "thread", I have no way to put the functionality in
one class if I want access the members in the running thread.

I just want a way to garantee that all my threads are cancelled when the
main thread is finished.

So hold references to them and call the ``cancel()`` method yourself.

That's a possible solution. Make a custom Thread class with a cancel
method and with all instances registered somewhere. At the end of the
main thread, I call something to cancel all these threads. There's
however some problems with that solution:
- Parent threads need to be cancelled before their children threads.
- It only works at the end of the main thread; it cannot be used at the
middle of an appplication.
- The cancellations are not automatic (I guess there's a way to do it by
overriding something like exit, anyway...)

All these problems are solved by using RAII with the Canceller class I
shown in a previous message, so I prefer using this solution. In fact,
what I would really need, is a mechanism, and I don't know if there's
already one, to disable ref-counting of a thread object in its
corresponding thread, so that I could provide a single class for
cancellable threads.

Regards,
Nicolas

In article <R4*******************@nnrp1.uunet.ca>,
Nicolas Fleury <ni******@yahoo.com_remove_the_> wrote:

Aahz wrote:

Right. As I said, you have to use the ``thread`` module directly; you
can't use ``threading``.
But even if I use "thread", I have no way to put the functionality in
one class if I want access the members in the running thread.

Huh? I don't understand what you mean.

I just want a way to garantee that all my threads are cancelled when the
main thread is finished.

So hold references to them and call the ``cancel()`` method yourself.

That's a possible solution. Make a custom Thread class with a cancel
method and with all instances registered somewhere. At the end of the
main thread, I call something to cancel all these threads. There's
however some problems with that solution:
- Parent threads need to be cancelled before their children threads.

Why's that a problem? Just hold references to the parent threads in the
main thread and cascade the cancels down.
- It only works at the end of the main thread; it cannot be used at the
middle of an appplication.

Why?
--
Aahz (aa**@pythoncraft.com) <*> http://www.pythoncraft.com/

"Do not taunt happy fun for loops. Do not change lists you are looping over."
--Remco Gerlich, comp.lang.python

Aahz wrote:

In article <R4*******************@nnrp1.uunet.ca>,
Nicolas Fleury <ni******@yahoo.com_remove_the_> wrote:

But even if I use "thread", I have no way to put the functionality in
one class if I want access the members in the running thread.

Huh? I don't understand what you mean.

Well, suppose I want to have a class with the same interface of
threading.Thread, but with two additions:
class CancellableThread:
def __del__(self): self.cancel()

The first addition is that the finalizer use a sub-class implemented
mechanism for cancellation of the thread.
The second addition is that, as soon as the object is not referred
anymore in the parent thread, the thread object is finalized.

Note that it doesn't need to be the finalizer, what is necessary is
something to be called when the object is no more referred in the parent
thread.

I'm saying that I don't know of any way to do that with a single class,
because I expect the sub-class to define something like:
def run(self)
And the thread object will be referred in the running thread by "self",
so I'm stuck without any way to know when the thread object is not
referred in the main thread.

I can do something like:
class Canceller:
def __init__(self, thread): self.thread = thread
def __del__(self): self.thread.cancel()

so that I encapsulate all threads with "Cancellers" in the parent
thread, but then I have to use two objects to the job that could
possibly be done by one.

- Parent threads need to be cancelled before their children threads.

Why's that a problem? Just hold references to the parent threads in the
main thread and cascade the cancels down.

You're right, it works. At the moment I wrote that I expected mix of a
CancellableThread class implemented with this mechanism with other
threads classes to cause loss of link between threads. If that happens
all threads that are not CancellableThreads (like the main thread) must
call the function to cancel remaining CancellableThreads started
directly by them.

- It only works at the end of the main thread; it cannot be used at the
middle of an appplication.

Why?

Because the goal is to cancel threads that are no more referred in the
main thread. Since there's no way to know which they are, there's no
way to do it at the middle of an application. You have to do it at the
end of the main thread, as for other threads, where you can assumed that
cancellable threads should be cancelled and joined.

I don't know if my explications of what I'm searching are more clear. I
have the feeling it's impossible to do it without a language extension
and at the same time that it would be less error-prone than
threading.Thread as deamons or not. Maybe I could write a PEP just for
the exercise of it...

Regards,
Nicolas