threading.Event usage causing intermitent exception

[akameswaran]

Admittedly this problem causes no actual functional issues aside from
an occasional error message when the program exits. The error is:

Unhandled exception in thread started by
Error in sys.excepthook:
Original exception was:

Yes all that info is blank. The application is a console application
that is waiting for some condition on the machine to happen. However,
I leave open the possiblitiy to cancel by a single key press at which
point the program terminates. Suffice it to say, I cannot perform both
checks without invoking threads as the key press gets "missed"
sometimes. Below is a simplification of the code

canceled = False
myEvent = threading.Event()

def watchForCancel()
global canceled
# turn of terminal buffering and capture key presses here
canceled = True
myEvent.set()
def watchForCondition()
# do a bunch of stuff checking the system
myEvent.set()

cancelThread = threading.Thread(target = watchForCancel)
cancelThread.setDaemon(True) # so I can exit the program when I want
to
cancelThread.start()
conditionThread = threading.Thread(target = watchForCondition)
conditionThread.setDaemon(True)
conditionThread.start()

myEvent.wait()

if cancelled:
sys.exit(2)

# do more stuff if the condition returned instead of cancel and then
I'm done
I've left out most of the active code, just cuz I think it muddies the
water. Now about 9 out of 10 times this works just fine. However,
every once in a while I get the exceptions mentioned above, but only
when I cancel out of the operation. I think the conditionThread is in
the process of shutting down and gets hosed up somehow and spits out an
exception, but the interpreter no longer has access to the info since
it is shutting down. I know that the condition is not being met in
these situations, since the condition is based on a network client
sending a certain set of data, and in my tests I know that no client
has sent said data. I haven't worried about this too much, as it
doesn't cause any functional problems, but it really irritates me. I
suppose I could make the threads aware of each other, but that just
seems stupid. Any suggestions on how to eliminate this intermittent
error?

Thanks

[Tim Peters]

[ak*********@gmail.com]

Admittedly this problem causes no actual functional issues aside from
an occasional error message when the program exits. The error is:

Unhandled exception in thread started by
Error in sys.excepthook:
Original exception was:

Yes all that info is blank.

That's typical when the interpreter has torn so much of itself down
that there's not enough left in the `sys` module even to print
exception info gracefully. The easiest way to stop that is to stop
/trying/ to run Python code while the interpreter is tearing itself
down.

The application is a console application that is waiting for some
condition on the machine to happen. However, I leave open the
possiblitiy to cancel by a single key press at which
point the program terminates. Suffice it to say, I cannot perform both
checks without invoking threads as the key press gets "missed"
sometimes. Below is a simplification of the code

canceled = False
myEvent = threading.Event()

def watchForCancel()
global canceled
# turn of terminal buffering and capture key presses here
canceled = True
myEvent.set()

Presumably this is some kind of poll-and-sleep loop? If so, add a
check to get out of the loop if myEvent.isSet(). Or if not, make it
some kind of poll-and-sleep loop ;-)

def watchForCondition()
# do a bunch of stuff checking the system
myEvent.set()

Ditto.

cancelThread = threading.Thread(target = watchForCancel)
cancelThread.setDaemon(True) # so I can exit the program when I want to

And get rid of that. The comment doesn't make sense to me, and
forcing a thread to be a daemon is exactly what /allows/ the thread to
keep running while the interpreter is tearing itself down. That's why
"daemonism" isn't the default: it's at best delicate. I don't see a
real reason for wanting this here.

cancelThread.start()
conditionThread = threading.Thread(target = watchForCondition)
conditionThread.setDaemon(True)

Ditto.

conditionThread.start()

myEvent.wait()

if cancelled:
sys.exit(2)

# do more stuff if the condition returned instead of cancel and then
I'm done
I've left out most of the active code, just cuz I think it muddies the
water. Now about 9 out of 10 times this works just fine. However,
every once in a while I get the exceptions mentioned above, but only
when I cancel out of the operation. I think the conditionThread is in
the process of shutting down and gets hosed up somehow and spits out an
exception, but the interpreter no longer has access to the info since
it is shutting down.

At this point it's likely that even sys.stdout and sys.stderr no
longer exist. The "Unhandled exception" message is printed directly
to the C-level `stderr` instead.

...
I suppose I could make the threads aware of each other, but that just
seems stupid. Any suggestions on how to eliminate this intermittent
error?

Stop forcing them to be daemon threads. The interpreter won't start
to tear itself down then before both threads terminate on their own.
To arrange for that, it's not necessary for the threads to become
aware of each other, but it is necessary for the threads to become
aware of another (shared) reason /for/ exiting. The most natural way
to do that, given what you said above, is to make both threads aware
that their shared myEvent event may get set "externally", and to stop
when they find it has been set.

[akameswaran]

Tim Peters wrote:

[ak*********@gmail.com]

Admittedly this problem causes no actual functional issues aside from
an occasional error message when the program exits. The error is:

Unhandled exception in thread started by
Error in sys.excepthook:
Original exception was:

Yes all that info is blank.

That's typical when the interpreter has torn so much of itself down
that there's not enough left in the `sys` module even to print
exception info gracefully. The easiest way to stop that is to stop
/trying/ to run Python code while the interpreter is tearing itself
down.

The application is a console application that is waiting for some
condition on the machine to happen. However, I leave open the
possiblitiy to cancel by a single key press at which
point the program terminates. Suffice it to say, I cannot perform both
checks without invoking threads as the key press gets "missed"
sometimes. Below is a simplification of the code

canceled = False
myEvent = threading.Event()

def watchForCancel()
global canceled
# turn of terminal buffering and capture key presses here
canceled = True
myEvent.set()

Presumably this is some kind of poll-and-sleep loop? If so, add a
check to get out of the loop if myEvent.isSet(). Or if not, make it
some kind of poll-and-sleep loop ;-)

after setting up the terminal correctly, ie changing buffering settings
etc. (I want to cancel with a single key-press with no "return" hit.
This was suprisingly difficult to do. The wait code is simply:

while keyPressed != 'q':
keyPressed = sys.stdin.read(1)

The really annoying thing here, is that I cannot put in any other steps
in the while loop without sometimes missing the key press. I have to
unbuffer the terminal(otherwise the read doesn't return until the enter
key is pressed), and for whatever reason performing any action no
matter how trivial leads to missing the key stroke on occasion - that's
why I pursued the daemon route.

>

def watchForCondition()
# do a bunch of stuff checking the system
myEvent.set()

Ditto.

cancelThread = threading.Thread(target = watchForCancel)
cancelThread.setDaemon(True) # so I can exit the program when I want to

The conditional thread will never terminate without making the threads
aware of eachother. Setting them daemon allows the interpreter to
shutdown with one of them running - and ocaisonally cuases the error.
What I find interesting, the behavior only happens when I cancel, not
when the condition is reached.

And get rid of that. The comment doesn't make sense to me, and
forcing a thread to be a daemon is exactly what /allows/ the thread to
keep running while the interpreter is tearing itself down. That's why
"daemonism" isn't the default: it's at best delicate. I don't see a
real reason for wanting this here.

cancelThread.start()
conditionThread = threading.Thread(target = watchForCondition)
conditionThread.setDaemon(True)

Ditto.

conditionThread.start()

myEvent.wait()

if cancelled:
sys.exit(2)

# do more stuff if the condition returned instead of cancel and then
I'm done
I've left out most of the active code, just cuz I think it muddies the
water. Now about 9 out of 10 times this works just fine. However,
every once in a while I get the exceptions mentioned above, but only
when I cancel out of the operation. I think the conditionThread is in
the process of shutting down and gets hosed up somehow and spits out an
exception, but the interpreter no longer has access to the info since
it is shutting down.

At this point it's likely that even sys.stdout and sys.stderr no
longer exist. The "Unhandled exception" message is printed directly
to the C-level `stderr` instead.

exactly - giving me no way to swallow it.

...
I suppose I could make the threads aware of each other, but that just
seems stupid. Any suggestions on how to eliminate this intermittent
error?

Stop forcing them to be daemon threads. The interpreter won't start
to tear itself down then before both threads terminate on their own.
To arrange for that, it's not necessary for the threads to become
aware of each other, but it is necessary for the threads to become
aware of another (shared) reason /for/ exiting. The most natural way
to do that, given what you said above, is to make both threads aware
that their shared myEvent event may get set "externally", and to stop
when they find it has been set.

Due to the oddities of capturing that keypress at the console, I
suspect I can un-daemonize the conditional thread and check for
cancelled=True and have it return. Since the error only occurs when
the conditional thread is still running, undaemonizing it will prevent
that and allow the interpreter to shut down more gracefully.

Thanks for the advice.