Hello comp.lang.pytho n,
I am attempting to write a threaded program right now in Python
(version 2.3 on Debian unstable) and the current behavior of
KeyboardInterru pt is causing me grief. From the documentation of the
thread module in Python 2.3.3:
"Threads interact strangely with interrupts: the KeyboardInterru pt
exception will be received by an arbitrary thread. (When the signal
module is available, interrupts always go to the main thread.)"
So the current behavior is that some "arbitraril y chosen" thread
receives the KeyboardInterru pt exception when the user hits Ctrl-C. My
question is whether there is some way to always make the main thread
receive the KeyboardInterru pt. The snippet at the end of the
documentation is vague/confusing. It sounds like KeyboardInterru pt
would always be delivered to the main thread, but I have tested this
and it is definitely not behaving that way. Maybe it just means that
the Unix signal is received by the main thread's signal handler (and
at that point an arbitrary thread is picked for KeyboardInterru pt). I
don't know.
The reason I cannot use the current "arbitrary thread"
KeyboardInterru pt delivery model:
The current model seems to deliver the KeyboardInterru pt to the
busiest thread (i.e. some sub-thread doing lots of computation, as
opposed to my main thread that is sitting in a sleep() loop). This
causes problems because the busy thread that I spawn runs library code
that was not written by me. That library code may contain lines like
"except:" that mask the KeyboardInterru pt so that the user experience
is that the app seems to ignore Ctrl-C.
If I could guarantee delivery of KeyboardInterru pt to the main thread,
I could control things much more easily/consistently. With the current
semantics code reuse becomes very difficult (I cannot use any code
that is "sloppy" with its exception catching or Ctrl-C will be
ignored).
Any ideas (besides rewriting the sloppy library code) would be much
appreciated.
Thanks,
-Ivan 8 8395
"Ivan Nestlerode" <q2*******@snea kemail.com> wrote in message
news:a6******** *************** ***@posting.goo gle.com... Hello comp.lang.pytho n,
I am attempting to write a threaded program right now in Python (version 2.3 on Debian unstable) and the current behavior of KeyboardInterru pt is causing me grief. From the documentation of the thread module in Python 2.3.3:
"Threads interact strangely with interrupts: the KeyboardInterru pt exception will be received by an arbitrary thread. (When the signal module is available, interrupts always go to the main thread.)"
Is the signal module not available to you?
<snip> Any ideas (besides rewriting the sloppy library code) would be much appreciated.
Thanks, -Ivan
Assuming you have access to the signal module:
1. Create a boolean that you can periodically test for in your thread code,
to see if you should keep processing. This may be a global singleton, a
global boolean, or something, but it needs to be visible to both the main
thread and the threaded code. You probably have some compute intensive
loops such as:
while( prime_factors < 1e15 ):
or
while( optimization != converged ):
or even just
while( True ):
Change these to reference the "keep processing" flag.
while( keep_processing and prime_factors < 1e15 ):
or
while( keep_processing and optimization != converged ):
or even just
while( keep_processing ): # no need to 'and' with True
2. Create a signal handler to trap for SIGINT. In the signal handler, set
the "keep-processing" bit to False:
import signal
def discontinue_pro cessing(signl, frme):
global keep_processing
keep_processing = False
return 0
signal.signal( signal.SIGINT, discontinue_pro cessing )
Now your main thread will get the ^C interrupt and clear the
"keep_processin g" flag, which will eventually get picked up by your worker
thread the next time it tests for it.
HTH,
-- Paul q2*******@sneak email.com (Ivan Nestlerode) writes: Hello comp.lang.pytho n,
I am attempting to write a threaded program right now in Python (version 2.3 on Debian unstable) and the current behavior of KeyboardInterru pt is causing me grief. From the documentation of the thread module in Python 2.3.3:
"Threads interact strangely with interrupts: the KeyboardInterru pt exception will be received by an arbitrary thread. (When the signal module is available, interrupts always go to the main thread.)"
On debian, the signal module certainly should be available, and the
KeyboardInterru pt exception should go to the main thread. Or at
least, that's what I thought. You're definitely seeing it being
delivered to an arbitrary thread?
Cheers,
mwh
--
The Programmer's Quick Guide To Python (Time Machine version):
You try to shoot yourself in the foot, only to realize that
there's no need, since Guido thoughtfully shot you in the foot
years ago. -- Nick Mathewson, comp.lang.pytho n
Michael Hudson <mw*@python.net > wrote in message news:<m3******* *****@pc150.mat hs.bris.ac.uk>. .. On debian, the signal module certainly should be available, and the KeyboardInterru pt exception should go to the main thread. Or at least, that's what I thought. You're definitely seeing it being delivered to an arbitrary thread?
Cheers, mwh
After a closer look, I am not seeing KeyboardInterru pt being delivered
to non-main threads. What I am seeing is that Ctrl-C does not work at
all when non-main threads are CPU intensive. I had previously jumped
to the conclusion that because Ctrl-C didn't work and because the
non-main thread was sloppy ("except:"), that Ctrl-C wasn't working
because KeyboardInterru pt was going to the sub-thread. That was an
incorrect conclusion.
So I guess the entire problem can be restated as:
How do I make Ctrl-C work properly when the non-main thread is busier
than the main thread?
The short program at the end of this post demonstrates the problem
with Ctrl-C and busy non-main threads. The idea behind this script is
that the main thread launches a busy sub-thread. Ctrl-C is supposed to
be caught in the main thread, stopping both threads, and ending the
program (with some printouts). As written, the program does not
respond to Ctrl-C even if I hold it down continuously. If you
uncomment the print statement in the main thread, all of the sudden it
starts to work. Why?
I think it has something to do with the main thread needing some
larger amount of CPU to detect the signal and the KeyboardInterru pt. I
think this is a bug though (I shouldn't have to put hacks into the
main thread to get this to work).
Any thoughts?
Thanks,
-Ivan
#!/usr/bin/python
from threading import Event, Thread
from time import sleep
class CpuHogThread(Th read):
def __init__(self):
Thread.__init__ (self)
self.counter = 0
self.stopEvent = Event()
def join(self):
self.stopEvent. set()
Thread.join(sel f)
def run(self):
try:
while not self.stopEvent. isSet():
self.counter += 1
except KeyboardInterru pt:
print 'CPU hog subthread caught KeyboardInterru pt'
if __name__ == '__main__':
t = CpuHogThread()
t.start()
print 'CPU hog subthread spawned'
try:
while True:
sleep(2)
# without the next print, Ctrl-C is completely ignored
#print 'still sleeping'
except KeyboardInterru pt:
print 'main thread caught KeyboardInterru pt'
t.join()
print 'CPU hog subthread joined at counter %d' % t.counter
"Paul McGuire" <pt***@austin.r r.com> wrote in message news:<Xh******* ************@fe 2.texas.rr.com> ... Is the signal module not available to you? <snip> Assuming you have access to the signal module: 1. Create a boolean that you can periodically test for in your thread code, to see if you should keep processing. This may be a global singleton, a global boolean, or something, but it needs to be visible to both the main thread and the threaded code. You probably have some compute intensive loops such as: while( prime_factors < 1e15 ): or while( optimization != converged ): or even just while( True ):
Change these to reference the "keep processing" flag. while( keep_processing and prime_factors < 1e15 ): or while( keep_processing and optimization != converged ): or even just while( keep_processing ): # no need to 'and' with True
2. Create a signal handler to trap for SIGINT. In the signal handler, set the "keep-processing" bit to False:
<snip>
Now your main thread will get the ^C interrupt and clear the "keep_processin g" flag, which will eventually get picked up by your worker thread the next time it tests for it.
HTH, -- Paul
I do have the signal module (Python 2.3.2 on Linux), but
unfortunately, even registering a signal handler doesn't fix the
problem where busy non-main threads prevent Ctrl-C from working.
Here is another code snippet using a signal handler (this code is very
similar to the code from my second post). If the main thread's loop
has the print statement, everything works fine. If it is commented out
(as written here), Ctrl-C is ignored completely and the program just
keeps running. Why won't this work? Is this a bug?
Thanks,
-Ivan
#!/usr/bin/python
from signal import signal, SIGINT
from threading import Event, Thread
from time import sleep
class CpuHogThread(Th read):
def __init__(self):
Thread.__init__ (self)
self.counter = 0
self.stopEvent = Event()
def join(self):
self.stopEvent. set()
Thread.join(sel f)
def run(self):
try:
while not self.stopEvent. isSet():
self.counter += 1
except KeyboardInterru pt:
print 'CPU hog subthread caught KeyboardInterru pt'
def handler(signal, frme):
stopEvent.set()
print 'main thread SIGINT handler'
t.join()
print 'CPU hog subthread joined at counter %d' % t.counter
return 0
if __name__ == '__main__':
stopEvent = Event()
signal(SIGINT, handler)
t = CpuHogThread()
t.start()
print 'CPU hog subthread spawned'
try:
while not stopEvent.isSet ():
sleep(2)
# without the next print, Ctrl-C is completely ignored
#print 'still sleeping'
except KeyboardInterru pt:
print 'main thread caught KeyboardInterru pt'
t.join()
print 'CPU hog subthread joined at counter %d' % t.counter
Running your attached program, it works just fine whereever I try it,
except on a system so old it only has Python 1.5 installed. Even that
worked after a few seconds of editing.
Here's the behavior I saw on Fedora Core 1:
$ python ivan.py
CPU hog subthread spawned
main thread caught KeyboardInterru pt
CPU hog subthread joined at counter 76429
This is the behavior I saw on redhat 9:
$ python /tmp/ivan.py
CPU hog subthread spawned
main thread caught KeyboardInterru pt
CPU hog subthread joined at counter 89897
Here's the behavior I saw on Red Hat 7.2:
CPU hog subthread spawned
main thread caught KeyboardInterru pt
CPU hog subthread joined at counter 31764
Here's the behavior I saw on Windows XP:
C:\...> python ivan.py
CPU hog subthread spawned
main thread caught KeyboardInterru pt
CPU hog subthread joined at counter 665928
Here's the behavior I saw on Red Hat 6.2:
$ python1.5 ivan.py
File "ivan.py", line 19
self.counter += 1
^
SyntaxError: invalid syntax
$ vi ivan.py
$ python1.5 ivan.py
CPU hog subthread spawned
main thread caught KeyboardInterru pt
CPU hog subthread joined at counter 238983
Jeff q2*******@sneak email.com (Ivan Nestlerode) writes: Michael Hudson <mw*@python.net > wrote in message news:<m3******* *****@pc150.mat hs.bris.ac.uk>. .. On debian, the signal module certainly should be available, and the KeyboardInterru pt exception should go to the main thread. Or at least, that's what I thought. You're definitely seeing it being delivered to an arbitrary thread?
Cheers, mwh After a closer look, I am not seeing KeyboardInterru pt being delivered to non-main threads. What I am seeing is that Ctrl-C does not work at all when non-main threads are CPU intensive.
Oh...
I had previously jumped to the conclusion that because Ctrl-C didn't work and because the non-main thread was sloppy ("except:"), that Ctrl-C wasn't working because KeyboardInterru pt was going to the sub-thread. That was an incorrect conclusion.
Well, in a way, good, but perhaps not for you...
So I guess the entire problem can be restated as: How do I make Ctrl-C work properly when the non-main thread is busier than the main thread?
This is hard for me to answer -- it works fine for me. What platform
are you on? ISTR Debian, but not versions of anything.
I'm somewhat inclined to start blaming libc at this point...
As you may well be aware, the combination of threads and signals is a
bit of a minefield. Python runs all threads other than the main
thread with all signals masked. My understanding is that when a signal
is delivered to a process it's delivered to an arbitrary thread that
has that signal unmasked -- in Python's case, this should be the main
thread. It kind of sounds like in your case the signal is being
queued up on a non-main thread, but as the signal is never unblocked,
it never gets handled.
[...] I think this is a bug though (I shouldn't have to put hacks into the main thread to get this to work).
Agreed, but I'm not sure they're in Python.
Any thoughts?
Upgrade glibc?
Cheers,
mwh
--
ARTHUR: Why are there three of you?
LINTILLAS: Why is there only one of you?
ARTHUR: Er... Could I have notice of that question?
-- The Hitch-Hikers Guide to the Galaxy, Episode 11
Michael Hudson <mw*@python.net > wrote in message news:<m3******* *****@pc150.mat hs.bris.ac.uk>. .. This is hard for me to answer -- it works fine for me. What platform are you on? ISTR Debian, but not versions of anything.
I'm somewhat inclined to start blaming libc at this point...
As you may well be aware, the combination of threads and signals is a bit of a minefield. Python runs all threads other than the main thread with all signals masked. My understanding is that when a signal is delivered to a process it's delivered to an arbitrary thread that has that signal unmasked -- in Python's case, this should be the main thread. It kind of sounds like in your case the signal is being queued up on a non-main thread, but as the signal is never unblocked, it never gets handled.
[...] I think this is a bug though (I shouldn't have to put hacks into the main thread to get this to work).
Agreed, but I'm not sure they're in Python.
Any thoughts?
Upgrade glibc?
Cheers, mwh
I've filed the bug with Debian since I think it is something wrong
with their Python or their glibc. The bug is #226547.
FYI, here are the exact details of the kernel/packages that don't seem
to work:
$ uname -a
Linux debian 2.4.23 #1 Sun Nov 30 21:11:49 EST 2003 i686 GNU/Linux
packages and versions:
ii libbz2-1.0 1.0.2-1
ii libc6 2.3.2.ds1-10
ii libdb4.1 4.1.25-10
ii libncurses5 5.3.20030719-4
ii libreadline4 4.3-8
ii libssl0.9.7 0.9.7c-5
ii python 2.3.3-4
ii python2.3 2.3.3-4
ii zlib1g 1:1.2.1-3
-Ivan
There is an easier way than using signals.
Add a keyboard interrupt handler for your main function.
And in the handler code, write a clean up function which
will be able to then manage threads.
This is the method I have used in my HarvestMan program.
I am copying this code here.
------------(copied code)----------------------------------------
try:
self.start_proj ect()
except (KeyboardInterr upt, EOFError):
# Localise links
if not self._cfg.ignor ekbinterrupt:
# dont allow to write cache, since it
# screws up existing cache.
GetObject('data manager').condi tional_cache_se t()
self.clean_up()
---------------------------------------------------------------------
The clean_up method takes care of cleaning the thread.
There you can add code to pass on the interrupt to the
main thread or perform your own clean up actions.
One way to do this is to use the currentThread() method
of threading module which can tell you whether the thread
is the main or a worker.
One drawback with this method is that you might need to
set your threads as 'daemon' threads to make sure that
the threads dont hang your python program. This is because
the entire python program exits when the only threads left
are daemon threads.
-Anand q2*******@sneak email.com (Ivan Nestlerode) wrote in message news:<a6******* *************** ****@posting.go ogle.com>... Michael Hudson <mw*@python.net > wrote in message news:<m3******* *****@pc150.mat hs.bris.ac.uk>. .. This is hard for me to answer -- it works fine for me. What platform are you on? ISTR Debian, but not versions of anything.
I'm somewhat inclined to start blaming libc at this point...
As you may well be aware, the combination of threads and signals is a bit of a minefield. Python runs all threads other than the main thread with all signals masked. My understanding is that when a signal is delivered to a process it's delivered to an arbitrary thread that has that signal unmasked -- in Python's case, this should be the main thread. It kind of sounds like in your case the signal is being queued up on a non-main thread, but as the signal is never unblocked, it never gets handled.
[...] I think this is a bug though (I shouldn't have to put hacks into the main thread to get this to work).
Agreed, but I'm not sure they're in Python.
Any thoughts?
Upgrade glibc?
Cheers, mwh
I've filed the bug with Debian since I think it is something wrong with their Python or their glibc. The bug is #226547.
FYI, here are the exact details of the kernel/packages that don't seem to work:
$ uname -a Linux debian 2.4.23 #1 Sun Nov 30 21:11:49 EST 2003 i686 GNU/Linux
packages and versions: ii libbz2-1.0 1.0.2-1 ii libc6 2.3.2.ds1-10 ii libdb4.1 4.1.25-10 ii libncurses5 5.3.20030719-4 ii libreadline4 4.3-8 ii libssl0.9.7 0.9.7c-5 ii python 2.3.3-4 ii python2.3 2.3.3-4 ii zlib1g 1:1.2.1-3
-Ivan This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics |
by: Steve Holden |
last post by:
I'm mailing the list in the hope that somebody has come up with a solution
to the occasional spurious "Keyboard Interrupt" exception that gets raised
in the ASP environment. It's a little awkward to explain to my client why
his COM server is failing this way, and the logic of the application makes
it difficult to trap the exception and repeat what's already been done.
I can't find anything with Google that makes it look like this problem...
|
by: PantherSE |
last post by:
Hello,
Ok, here's my situation. I have a small application that listens to
messages on a UDP port.
When I pass None to settimeout(), and I hit Ctrl+C to interrupt the
wait my exception handler catches the KeyboardInterrupt exception.
However, if I pass any value to settimeout() the exception is not
caught.
|
by: Operation Latte Thunder |
last post by:
I have a simple test proggie that isn't behaving like I expect ( found
below ). The script will infinitely run ( as expected ), but seems to
completely ignore control-C's. Shouldn't the interpreter pass along
KeyboardInterrupts and break out of the while loop, or am I missing
something?
Using python 2.4.2 on linux ( if it matters )
-- Script Below --
|
by: Tamas Nepusz |
last post by:
Hi everyone,
I have tried to do some googling before asking my question here, but I
haven't found any suitable answer. I am developing a Python API for a
graph library written in pure C. The library is doing an awful lot of
math computations, and some of them can take a pretty long time
(depending on the size of the input). If I invoke such calculation from
Python using my own extension, the interpreter locks until the
calculation is...
|
by: Tony Nelson |
last post by:
I'm trying to find out what is eating some KeyboardInterrupt exceptions
in a fairly large program (yum). My KeyboardInterrupt handler is called
for some Ctl-C presses, but for others nothing seems to happen.
Grepping the source (what of it I've found, looking at import
statements) doesn't turn up anything likely.
My thinking is that either some "except:" clause is eating them, or some
place I haven't looked is eating them, or possibly C...
| |
by: ruka_at_ |
last post by:
Hi,
why is KeyboardInterrupt not caught (xp)?
import sys
try:
inp = sys.stdin.read()
except (KeyboardInterrupt, SystemExit):
print "kbd-interr,SystemExit"
except EOFError:
print "eof encountered"
except:
|
by: Michael Goerz |
last post by:
Hi,
when I try to catch ctrl+c with
except KeyboardInterrupt:
pychecker tells me
Catching a non-Exception object (KeyboardInterrupt)
|
by: radim.marek |
last post by:
Hi Guys,
during testing of my project on FreeBSD I've discovered stange
'feature' of time.sleep(). It works if single thread is running, but
when multi-threaded, the SIGINT signal seems not to be handled in same
way.
I've found three discussion about similar behavior with os.system()
but so far none of the suggested patches worked.
|
by: Brendon Costa |
last post by:
Hi all,
I have a small python project i am working on. Basically i always have
two threads. A "Read" thread that sits in a loop reading a line at a
time from some input (Usually stdin) and then generating events to be
processed and a "Proc" thread that processes incoming events from a
queue. There will be additional threads as well that asynchronously
insert events into the queue to be processed, but they are not a part
of this so i...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
| |
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth.
The Art of Business Website Design
Your website is...
|
by: conductexam |
last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one.
At the time of converting from word file to html my equations which are in the word document file was convert into image.
Globals.ThisAddIn.Application.ActiveDocument.Select();...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
|
by: 6302768590 |
last post by:
Hai team
i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
|
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |
by: bsmnconsultancy |
last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...
| |