Re: How to kill a thread?

It's always been my understanding that you can't forcibly kill a thread
in Python (at least not in a portable way). The best you can do is
politely ask it to die, IIRC.

--
code.py: A blog about life, the universe, and Python

http://pythonista.wordpress.com
** Posted from http://www.teranews.com **

On Jun 6, 12:44 pm, The Pythonista <n...@this.time wrote:

It's always been my understanding that you can't forcibly kill a thread
in Python (at least not in a portable way). The best you can do is
politely ask it to die, IIRC.

Inherently, the best you can do in most languages is ask them politely
to die. Otherwise you'll leave locks and various other datastructures
in an inconvenient state, which is too complex to handle correctly.
The exception is certain functional languages, which aren't capable of
having threads and complex state in the same sense.

However, you could go quite far with a standardized mechanism of
politely asking threads to die. For instance, all blocking I/O
functions could be made to support it. This is how cancellation works
in python-safethread.

Would it be possible for a 3rd-party threading library to an
'interruptible' version of Queue?

The same methods as the normal Queue, but when you call a new .kill()
method on the queue, all the threads locking on the queue get a
"QueueKille d" exception thrown.

It might be as simple as a Queue wrapper where .kill() adds a speciall
Killed value to the queue, and the .get() wrapper throws a QueueKilled
exception (after re-putting Killed) when it reads Killed.

Locking for queues with a limited size is more complicated. Maybe
..kill() can also set an .killed attribute, and pop anything already in
the queue before putting Killed. Then when the put() wrapper unlocks,
it checks if .killed is set, and throws a QueueKilled exception.

One side-effect is that the queue becomes unusable (get & put now
immediately throw QueueKilled), unless you make a .unkill() method.

David

David <wi******@gmail .comwrote:

Would it be possible for a 3rd-party threading library to an
'interruptible' version of Queue?

The same methods as the normal Queue, but when you call a new .kill()
method on the queue, all the threads locking on the queue get a
"QueueKille d" exception thrown.

It might be as simple as a Queue wrapper where .kill() adds a speciall
Killed value to the queue, and the .get() wrapper throws a QueueKilled
exception (after re-putting Killed) when it reads Killed.

There seem to me to be 2 basic types of threads - I'll call them
client threads and server threads.

A client thread listens on a Queue for its instructions and replies
via a different Queue, or via Queues passed in in its instructions.
That kind of thread can easily have an instruction which means quit.
When it isn't processing instructions there is a nice clean moment for
it to quit. However if the processing takes a long time then there
will be a long wait for the thread to die. This is the scenario
described above.

The server type thread is processing all the time. Every now and
again it sends its results via a Queue. An example of this type of
thread might be one listening on a socket. This type of thread has no
nice moment to listen for instructions to quit as it might well be
blocked on something else (eg listening to a socket). To make this
type of thread kill nicely you have to go back to breaking up your
work into chunks (eg polling the socket asynchronously) and polling
the command queue to wait for your quit instruction which is wastefull
of CPU resources.

So it seems to me that there isn't a good solution to killing python
threads of either type, other than coding them carefully and checking
for quit instructions regularly.

You can kill them using the internal python API and ctypes, but you
run the risk of leaving things in an inconsistent state which is why
that API isn't exposed.

Here is an attempt at a killable thread

http://aspn.activestate.com/ASPN/Coo.../Recipe/496960

and

http://sebulba.wikispaces.com/recipe+thread2

Read the caveats!

--
Nick Craig-Wood <ni**@craig-wood.com-- http://www.craig-wood.com/nick

On 2008-06-07, Rhamphoryncus <rh****@gmail.c omwrote:

On Jun 6, 12:44 pm, The Pythonista <n...@this.time wrote:

>It's always been my understanding that you can't forcibly kill a thread
in Python (at least not in a portable way). The best you can do is
politely ask it to die, IIRC.

Inherently, the best you can do in most languages is ask them politely
to die. Otherwise you'll leave locks and various other datastructures
in an inconvenient state, which is too complex to handle correctly.
The exception is certain functional languages, which aren't capable of
having threads and complex state in the same sense.

Well it would of course depend on what is considered asking politely?

If one thread could cause an exception being thrown in an other thread,
would this be considered a polite way to ask? Would it be considered
an acceptable way?

--
Antoon Pardon

On 2008-06-07, David <wi******@gmail .comwrote:

Would it be possible for a 3rd-party threading library to an
'interruptible' version of Queue?

The same methods as the normal Queue, but when you call a new .kill()
method on the queue, all the threads locking on the queue get a
"QueueKille d" exception thrown.

I have done something similar. The idea was that threads had to open
a queue before they could access it. If the last thread who had the
queue open in write mode, closed the queue, a reader trying to get
an element from an empty queue would have an "EOInformat ion" exception
raised.

--
Antoon Pardon

On Jun 9, 5:33 am, Antoon Pardon <apar...@forel. vub.ac.bewrote:

On 2008-06-07, Rhamphoryncus <rha...@gmail.c omwrote:

On Jun 6, 12:44 pm, The Pythonista <n...@this.time wrote:

It's always been my understanding that you can't forcibly kill a thread
in Python (at least not in a portable way). The best you can do is
politely ask it to die, IIRC.

Inherently, the best you can do in most languages is ask them politely
to die. Otherwise you'll leave locks and various other datastructures
in an inconvenient state, which is too complex to handle correctly.
The exception is certain functional languages, which aren't capable of
having threads and complex state in the same sense.

Well it would of course depend on what is considered asking politely?

If one thread could cause an exception being thrown in an other thread,
would this be considered a polite way to ask? Would it be considered
an acceptable way?

The exception must not be raised until a point explicitly designed as
safe is hit. Otherwise, any function that manipulates data you'll
still use will potentially be buggered. Consider sys.stdout: codecs,
buffering, lots to go wrong.

On Jun 9, 9:20*pm, Rhamphoryncus <rha...@gmail.c omwrote:

On Jun 9, 5:33 am, Antoon Pardon <apar...@forel. vub.ac.bewrote:

On 2008-06-07, Rhamphoryncus <rha...@gmail.c omwrote:

On Jun 6, 12:44 pm, The Pythonista <n...@this.time wrote:
>It's always been my understanding that you can't forcibly kill a thread
>in Python (at least not in a portable way). *The best you can do is
>politely ask it to die, IIRC.

Inherently, the best you can do in most languages is ask them politely
to die. *Otherwise you'll leave locks and various other datastructures
in an inconvenient state, which is too complex to handle correctly.
The exception is certain functional languages, which aren't capable of
having threads and complex state in the same sense.

Well it would of course depend on what is considered asking politely?

If one thread could cause an exception being thrown in an other thread,
would this be considered a polite way to ask? Would it be considered
an acceptable way?

The exception must not be raised until a point explicitly designed as
safe is hit. *Otherwise, any function that manipulates data you'll
still use will potentially be buggered. *Consider sys.stdout: codecs,
buffering, lots to go wrong.

Java and .NET both have ways of killing threads. They both work by
raising a 'ThreadAbort' (or similar) exception in the target thread.
In early implementations they both suffered from a similar problem.
You could protect locks (etc) by having a finally block that would
release all resources as needed - but what happens if the thread abort
exception is raised *inside* the finally block?

Java responded by deprecating thread aborting. .NET responded by
ensuring that a thread abort exception would never be raised inside a
finally block - if that happened the exception would only be raised
once the code has left the finally block.

Aborting threads in .NET can be extremely useful. Politely asking a
thread to die is no good if the task the thread is executing is
extremely coarse grained - it may not be able to respond to the
request for some time. If your code is structured correctly
(performing a long running background calculation for example) then
you may *know* that you can kill it without problems, but Python has
no native API to do this.

Michael Foord
http://www.ironpythoninaction.com/

On Jun 9, 2:52 pm, Fuzzyman <fuzzy...@gmail .comwrote:

On Jun 9, 9:20 pm, Rhamphoryncus <rha...@gmail.c omwrote:

On Jun 9, 5:33 am, Antoon Pardon <apar...@forel. vub.ac.bewrote:

On 2008-06-07, Rhamphoryncus <rha...@gmail.c omwrote:

On Jun 6, 12:44 pm, The Pythonista <n...@this.time wrote:
It's always been my understanding that you can't forcibly kill a thread
in Python (at least not in a portable way). The best you can do is
politely ask it to die, IIRC.

Inherently, the best you can do in most languages is ask them politely
to die. Otherwise you'll leave locks and various other datastructures
in an inconvenient state, which is too complex to handle correctly.
The exception is certain functional languages, which aren't capable of
having threads and complex state in the same sense.

Well it would of course depend on what is considered asking politely?

If one thread could cause an exception being thrown in an other thread,
would this be considered a polite way to ask? Would it be considered
an acceptable way?

The exception must not be raised until a point explicitly designed as
safe is hit. Otherwise, any function that manipulates data you'll
still use will potentially be buggered. Consider sys.stdout: codecs,
buffering, lots to go wrong.

Java and .NET both have ways of killing threads. They both work by
raising a 'ThreadAbort' (or similar) exception in the target thread.
In early implementations they both suffered from a similar problem.
You could protect locks (etc) by having a finally block that would
release all resources as needed - but what happens if the thread abort
exception is raised *inside* the finally block?

Java responded by deprecating thread aborting. .NET responded by
ensuring that a thread abort exception would never be raised inside a
finally block - if that happened the exception would only be raised
once the code has left the finally block.

Aborting threads in .NET can be extremely useful. Politely asking a
thread to die is no good if the task the thread is executing is
extremely coarse grained - it may not be able to respond to the
request for some time. If your code is structured correctly
(performing a long running background calculation for example) then
you may *know* that you can kill it without problems, but Python has
no native API to do this.

So how does .NET deal with the sys.stdout corruption? Does it?

If you've carefully written your code to use only safe primitives and
local state (discarded if interrupted) then yes, it could be
interruptible. At this point you could specially mark that block of
code as safe, leaving the vast majority of other code unsafe by
default. Then again, since you're going to the trouble of carefully
designing and auditing your code you could just make it cancellable
like blocking I/O should be - just by polling a flag at key points
(and you're CPU-bound anyway, so it's not expensive.)

The only place I know of that you *need* arbitrary interruption is
hitting CTRL-C in the interactive interpreter. At this point it's a
debugging tool though, so the risk of weirdness is acceptable.