Hello comp.lang.python,
I am attempting to write a threaded program right now in Python
(version 2.3 on Debian unstable) and the current behavior of
KeyboardInterrupt is causing me grief. From the documentation of the
thread module in Python 2.3.3:
"Threads interact strangely with interrupts: the KeyboardInterrupt
exception will be received by an arbitrary thread. (When the signal
module is available, interrupts always go to the main thread.)"
So the current behavior is that some "arbitrarily chosen" thread
receives the KeyboardInterrupt exception when the user hits Ctrl-C. My
question is whether there is some way to always make the main thread
receive the KeyboardInterrupt. The snippet at the end of the
documentation is vague/confusing. It sounds like KeyboardInterrupt
would always be delivered to the main thread, but I have tested this
and it is definitely not behaving that way. Maybe it just means that
the Unix signal is received by the main thread's signal handler (and
at that point an arbitrary thread is picked for KeyboardInterrupt). I
don't know.
The reason I cannot use the current "arbitrary thread"
KeyboardInterrupt delivery model:
The current model seems to deliver the KeyboardInterrupt to the
busiest thread (i.e. some sub-thread doing lots of computation, as
opposed to my main thread that is sitting in a sleep() loop). This
causes problems because the busy thread that I spawn runs library code
that was not written by me. That library code may contain lines like
"except:" that mask the KeyboardInterrupt so that the user experience
is that the app seems to ignore Ctrl-C.
If I could guarantee delivery of KeyboardInterrupt to the main thread,
I could control things much more easily/consistently. With the current
semantics code reuse becomes very difficult (I cannot use any code
that is "sloppy" with its exception catching or Ctrl-C will be
ignored).
Any ideas (besides rewriting the sloppy library code) would be much
appreciated.
Thanks,
-Ivan 8 8353
"Ivan Nestlerode" <q2*******@sneakemail.com> wrote in message
news:a6**************************@posting.google.c om... Hello comp.lang.python,
I am attempting to write a threaded program right now in Python (version 2.3 on Debian unstable) and the current behavior of KeyboardInterrupt is causing me grief. From the documentation of the thread module in Python 2.3.3:
"Threads interact strangely with interrupts: the KeyboardInterrupt exception will be received by an arbitrary thread. (When the signal module is available, interrupts always go to the main thread.)"
Is the signal module not available to you?
<snip> Any ideas (besides rewriting the sloppy library code) would be much appreciated.
Thanks, -Ivan
Assuming you have access to the signal module:
1. Create a boolean that you can periodically test for in your thread code,
to see if you should keep processing. This may be a global singleton, a
global boolean, or something, but it needs to be visible to both the main
thread and the threaded code. You probably have some compute intensive
loops such as:
while( prime_factors < 1e15 ):
or
while( optimization != converged ):
or even just
while( True ):
Change these to reference the "keep processing" flag.
while( keep_processing and prime_factors < 1e15 ):
or
while( keep_processing and optimization != converged ):
or even just
while( keep_processing ): # no need to 'and' with True
2. Create a signal handler to trap for SIGINT. In the signal handler, set
the "keep-processing" bit to False:
import signal
def discontinue_processing(signl, frme):
global keep_processing
keep_processing = False
return 0
signal.signal( signal.SIGINT, discontinue_processing )
Now your main thread will get the ^C interrupt and clear the
"keep_processing" flag, which will eventually get picked up by your worker
thread the next time it tests for it.
HTH,
-- Paul q2*******@sneakemail.com (Ivan Nestlerode) writes: Hello comp.lang.python,
I am attempting to write a threaded program right now in Python (version 2.3 on Debian unstable) and the current behavior of KeyboardInterrupt is causing me grief. From the documentation of the thread module in Python 2.3.3:
"Threads interact strangely with interrupts: the KeyboardInterrupt exception will be received by an arbitrary thread. (When the signal module is available, interrupts always go to the main thread.)"
On debian, the signal module certainly should be available, and the
KeyboardInterrupt exception should go to the main thread. Or at
least, that's what I thought. You're definitely seeing it being
delivered to an arbitrary thread?
Cheers,
mwh
--
The Programmer's Quick Guide To Python (Time Machine version):
You try to shoot yourself in the foot, only to realize that
there's no need, since Guido thoughtfully shot you in the foot
years ago. -- Nick Mathewson, comp.lang.python
Michael Hudson <mw*@python.net> wrote in message news:<m3************@pc150.maths.bris.ac.uk>... On debian, the signal module certainly should be available, and the KeyboardInterrupt exception should go to the main thread. Or at least, that's what I thought. You're definitely seeing it being delivered to an arbitrary thread?
Cheers, mwh
After a closer look, I am not seeing KeyboardInterrupt being delivered
to non-main threads. What I am seeing is that Ctrl-C does not work at
all when non-main threads are CPU intensive. I had previously jumped
to the conclusion that because Ctrl-C didn't work and because the
non-main thread was sloppy ("except:"), that Ctrl-C wasn't working
because KeyboardInterrupt was going to the sub-thread. That was an
incorrect conclusion.
So I guess the entire problem can be restated as:
How do I make Ctrl-C work properly when the non-main thread is busier
than the main thread?
The short program at the end of this post demonstrates the problem
with Ctrl-C and busy non-main threads. The idea behind this script is
that the main thread launches a busy sub-thread. Ctrl-C is supposed to
be caught in the main thread, stopping both threads, and ending the
program (with some printouts). As written, the program does not
respond to Ctrl-C even if I hold it down continuously. If you
uncomment the print statement in the main thread, all of the sudden it
starts to work. Why?
I think it has something to do with the main thread needing some
larger amount of CPU to detect the signal and the KeyboardInterrupt. I
think this is a bug though (I shouldn't have to put hacks into the
main thread to get this to work).
Any thoughts?
Thanks,
-Ivan
#!/usr/bin/python
from threading import Event, Thread
from time import sleep
class CpuHogThread(Thread):
def __init__(self):
Thread.__init__(self)
self.counter = 0
self.stopEvent = Event()
def join(self):
self.stopEvent.set()
Thread.join(self)
def run(self):
try:
while not self.stopEvent.isSet():
self.counter += 1
except KeyboardInterrupt:
print 'CPU hog subthread caught KeyboardInterrupt'
if __name__ == '__main__':
t = CpuHogThread()
t.start()
print 'CPU hog subthread spawned'
try:
while True:
sleep(2)
# without the next print, Ctrl-C is completely ignored
#print 'still sleeping'
except KeyboardInterrupt:
print 'main thread caught KeyboardInterrupt'
t.join()
print 'CPU hog subthread joined at counter %d' % t.counter
"Paul McGuire" <pt***@austin.rr.com> wrote in message news:<Xh*******************@fe2.texas.rr.com>... Is the signal module not available to you? <snip> Assuming you have access to the signal module: 1. Create a boolean that you can periodically test for in your thread code, to see if you should keep processing. This may be a global singleton, a global boolean, or something, but it needs to be visible to both the main thread and the threaded code. You probably have some compute intensive loops such as: while( prime_factors < 1e15 ): or while( optimization != converged ): or even just while( True ):
Change these to reference the "keep processing" flag. while( keep_processing and prime_factors < 1e15 ): or while( keep_processing and optimization != converged ): or even just while( keep_processing ): # no need to 'and' with True
2. Create a signal handler to trap for SIGINT. In the signal handler, set the "keep-processing" bit to False:
<snip>
Now your main thread will get the ^C interrupt and clear the "keep_processing" flag, which will eventually get picked up by your worker thread the next time it tests for it.
HTH, -- Paul
I do have the signal module (Python 2.3.2 on Linux), but
unfortunately, even registering a signal handler doesn't fix the
problem where busy non-main threads prevent Ctrl-C from working.
Here is another code snippet using a signal handler (this code is very
similar to the code from my second post). If the main thread's loop
has the print statement, everything works fine. If it is commented out
(as written here), Ctrl-C is ignored completely and the program just
keeps running. Why won't this work? Is this a bug?
Thanks,
-Ivan
#!/usr/bin/python
from signal import signal, SIGINT
from threading import Event, Thread
from time import sleep
class CpuHogThread(Thread):
def __init__(self):
Thread.__init__(self)
self.counter = 0
self.stopEvent = Event()
def join(self):
self.stopEvent.set()
Thread.join(self)
def run(self):
try:
while not self.stopEvent.isSet():
self.counter += 1
except KeyboardInterrupt:
print 'CPU hog subthread caught KeyboardInterrupt'
def handler(signal, frme):
stopEvent.set()
print 'main thread SIGINT handler'
t.join()
print 'CPU hog subthread joined at counter %d' % t.counter
return 0
if __name__ == '__main__':
stopEvent = Event()
signal(SIGINT, handler)
t = CpuHogThread()
t.start()
print 'CPU hog subthread spawned'
try:
while not stopEvent.isSet():
sleep(2)
# without the next print, Ctrl-C is completely ignored
#print 'still sleeping'
except KeyboardInterrupt:
print 'main thread caught KeyboardInterrupt'
t.join()
print 'CPU hog subthread joined at counter %d' % t.counter
Running your attached program, it works just fine whereever I try it,
except on a system so old it only has Python 1.5 installed. Even that
worked after a few seconds of editing.
Here's the behavior I saw on Fedora Core 1:
$ python ivan.py
CPU hog subthread spawned
main thread caught KeyboardInterrupt
CPU hog subthread joined at counter 76429
This is the behavior I saw on redhat 9:
$ python /tmp/ivan.py
CPU hog subthread spawned
main thread caught KeyboardInterrupt
CPU hog subthread joined at counter 89897
Here's the behavior I saw on Red Hat 7.2:
CPU hog subthread spawned
main thread caught KeyboardInterrupt
CPU hog subthread joined at counter 31764
Here's the behavior I saw on Windows XP:
C:\...> python ivan.py
CPU hog subthread spawned
main thread caught KeyboardInterrupt
CPU hog subthread joined at counter 665928
Here's the behavior I saw on Red Hat 6.2:
$ python1.5 ivan.py
File "ivan.py", line 19
self.counter += 1
^
SyntaxError: invalid syntax
$ vi ivan.py
$ python1.5 ivan.py
CPU hog subthread spawned
main thread caught KeyboardInterrupt
CPU hog subthread joined at counter 238983
Jeff q2*******@sneakemail.com (Ivan Nestlerode) writes: Michael Hudson <mw*@python.net> wrote in message news:<m3************@pc150.maths.bris.ac.uk>... On debian, the signal module certainly should be available, and the KeyboardInterrupt exception should go to the main thread. Or at least, that's what I thought. You're definitely seeing it being delivered to an arbitrary thread?
Cheers, mwh After a closer look, I am not seeing KeyboardInterrupt being delivered to non-main threads. What I am seeing is that Ctrl-C does not work at all when non-main threads are CPU intensive.
Oh...
I had previously jumped to the conclusion that because Ctrl-C didn't work and because the non-main thread was sloppy ("except:"), that Ctrl-C wasn't working because KeyboardInterrupt was going to the sub-thread. That was an incorrect conclusion.
Well, in a way, good, but perhaps not for you...
So I guess the entire problem can be restated as: How do I make Ctrl-C work properly when the non-main thread is busier than the main thread?
This is hard for me to answer -- it works fine for me. What platform
are you on? ISTR Debian, but not versions of anything.
I'm somewhat inclined to start blaming libc at this point...
As you may well be aware, the combination of threads and signals is a
bit of a minefield. Python runs all threads other than the main
thread with all signals masked. My understanding is that when a signal
is delivered to a process it's delivered to an arbitrary thread that
has that signal unmasked -- in Python's case, this should be the main
thread. It kind of sounds like in your case the signal is being
queued up on a non-main thread, but as the signal is never unblocked,
it never gets handled.
[...] I think this is a bug though (I shouldn't have to put hacks into the main thread to get this to work).
Agreed, but I'm not sure they're in Python.
Any thoughts?
Upgrade glibc?
Cheers,
mwh
--
ARTHUR: Why are there three of you?
LINTILLAS: Why is there only one of you?
ARTHUR: Er... Could I have notice of that question?
-- The Hitch-Hikers Guide to the Galaxy, Episode 11
Michael Hudson <mw*@python.net> wrote in message news:<m3************@pc150.maths.bris.ac.uk>... This is hard for me to answer -- it works fine for me. What platform are you on? ISTR Debian, but not versions of anything.
I'm somewhat inclined to start blaming libc at this point...
As you may well be aware, the combination of threads and signals is a bit of a minefield. Python runs all threads other than the main thread with all signals masked. My understanding is that when a signal is delivered to a process it's delivered to an arbitrary thread that has that signal unmasked -- in Python's case, this should be the main thread. It kind of sounds like in your case the signal is being queued up on a non-main thread, but as the signal is never unblocked, it never gets handled.
[...] I think this is a bug though (I shouldn't have to put hacks into the main thread to get this to work).
Agreed, but I'm not sure they're in Python.
Any thoughts?
Upgrade glibc?
Cheers, mwh
I've filed the bug with Debian since I think it is something wrong
with their Python or their glibc. The bug is #226547.
FYI, here are the exact details of the kernel/packages that don't seem
to work:
$ uname -a
Linux debian 2.4.23 #1 Sun Nov 30 21:11:49 EST 2003 i686 GNU/Linux
packages and versions:
ii libbz2-1.0 1.0.2-1
ii libc6 2.3.2.ds1-10
ii libdb4.1 4.1.25-10
ii libncurses5 5.3.20030719-4
ii libreadline4 4.3-8
ii libssl0.9.7 0.9.7c-5
ii python 2.3.3-4
ii python2.3 2.3.3-4
ii zlib1g 1:1.2.1-3
-Ivan
There is an easier way than using signals.
Add a keyboard interrupt handler for your main function.
And in the handler code, write a clean up function which
will be able to then manage threads.
This is the method I have used in my HarvestMan program.
I am copying this code here.
------------(copied code)----------------------------------------
try:
self.start_project()
except (KeyboardInterrupt, EOFError):
# Localise links
if not self._cfg.ignorekbinterrupt:
# dont allow to write cache, since it
# screws up existing cache.
GetObject('datamanager').conditional_cache_set()
self.clean_up()
---------------------------------------------------------------------
The clean_up method takes care of cleaning the thread.
There you can add code to pass on the interrupt to the
main thread or perform your own clean up actions.
One way to do this is to use the currentThread() method
of threading module which can tell you whether the thread
is the main or a worker.
One drawback with this method is that you might need to
set your threads as 'daemon' threads to make sure that
the threads dont hang your python program. This is because
the entire python program exits when the only threads left
are daemon threads.
-Anand q2*******@sneakemail.com (Ivan Nestlerode) wrote in message news:<a6**************************@posting.google. com>... Michael Hudson <mw*@python.net> wrote in message news:<m3************@pc150.maths.bris.ac.uk>... This is hard for me to answer -- it works fine for me. What platform are you on? ISTR Debian, but not versions of anything.
I'm somewhat inclined to start blaming libc at this point...
As you may well be aware, the combination of threads and signals is a bit of a minefield. Python runs all threads other than the main thread with all signals masked. My understanding is that when a signal is delivered to a process it's delivered to an arbitrary thread that has that signal unmasked -- in Python's case, this should be the main thread. It kind of sounds like in your case the signal is being queued up on a non-main thread, but as the signal is never unblocked, it never gets handled.
[...] I think this is a bug though (I shouldn't have to put hacks into the main thread to get this to work).
Agreed, but I'm not sure they're in Python.
Any thoughts?
Upgrade glibc?
Cheers, mwh
I've filed the bug with Debian since I think it is something wrong with their Python or their glibc. The bug is #226547.
FYI, here are the exact details of the kernel/packages that don't seem to work:
$ uname -a Linux debian 2.4.23 #1 Sun Nov 30 21:11:49 EST 2003 i686 GNU/Linux
packages and versions: ii libbz2-1.0 1.0.2-1 ii libc6 2.3.2.ds1-10 ii libdb4.1 4.1.25-10 ii libncurses5 5.3.20030719-4 ii libreadline4 4.3-8 ii libssl0.9.7 0.9.7c-5 ii python 2.3.3-4 ii python2.3 2.3.3-4 ii zlib1g 1:1.2.1-3
-Ivan This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Steve Holden |
last post by:
I'm mailing the list in the hope that somebody has come up with a solution
to the occasional spurious "Keyboard Interrupt" exception that gets raised
in the ASP environment. It's a little awkward...
|
by: PantherSE |
last post by:
Hello,
Ok, here's my situation. I have a small application that listens to
messages on a UDP port.
When I pass None to settimeout(), and I hit Ctrl+C to interrupt the
wait my exception...
|
by: Operation Latte Thunder |
last post by:
I have a simple test proggie that isn't behaving like I expect ( found
below ). The script will infinitely run ( as expected ), but seems to
completely ignore control-C's. Shouldn't the...
|
by: Tamas Nepusz |
last post by:
Hi everyone,
I have tried to do some googling before asking my question here, but I
haven't found any suitable answer. I am developing a Python API for a
graph library written in pure C. The...
|
by: Tony Nelson |
last post by:
I'm trying to find out what is eating some KeyboardInterrupt exceptions
in a fairly large program (yum). My KeyboardInterrupt handler is called
for some Ctl-C presses, but for others nothing seems...
|
by: ruka_at_ |
last post by:
Hi,
why is KeyboardInterrupt not caught (xp)?
import sys
try:
inp = sys.stdin.read()
except (KeyboardInterrupt, SystemExit):
print "kbd-interr,SystemExit"
except EOFError:
print "eof...
|
by: Michael Goerz |
last post by:
Hi,
when I try to catch ctrl+c with
except KeyboardInterrupt:
pychecker tells me
Catching a non-Exception object (KeyboardInterrupt)
|
by: radim.marek |
last post by:
Hi Guys,
during testing of my project on FreeBSD I've discovered stange
'feature' of time.sleep(). It works if single thread is running, but
when multi-threaded, the SIGINT signal seems not to...
|
by: Brendon Costa |
last post by:
Hi all,
I have a small python project i am working on. Basically i always have
two threads. A "Read" thread that sits in a loop reading a line at a
time from some input (Usually stdin) and then...
|
by: taylorcarr |
last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
|
by: aa123db |
last post by:
Variable and constants
Use var or let for variables and const fror constants.
Var foo ='bar';
Let foo ='bar';const baz ='bar';
Functions
function $name$ ($parameters$) {
}
...
|
by: ryjfgjl |
last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: Hystou |
last post by:
There are some requirements for setting up RAID:
1. The motherboard and BIOS support RAID configuration.
2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
| |