I have a multi-threaded python application which uses pexpect to connect to
multiple systems concurrently. Each thread within my application is a
connection to a remote system. The problem is when one of the child
threads runs a command which generates an unlimited amount of output. The
classic example of this is the "yes" command. If you
execute "pexpect.run('y es')", your cpu will sit at 100% forever.
Here is a simple multi-threaded program using pexpect which demonstrates the
problem. The command 'yes' is run in a thread. The parent says that when
the alarm goes off, run the handler function. The thread sets the alarm to
trigger after 5 seconds.
#!/usr/bin/env python
import signal, os, pexpect, threading
def handler(signum, frame):
# This should get called after 5 seconds when the alarm fires.
os.system('kill all yes')
print "The yes command has been killed!"
def runyes():
# Run the 'yes' command in a thread. Set an alarm
# to fire in 5 seconds.
signal.alarm(5)
print "Running yes command..."
# If you run 'sleep 10' instead, this works.
pexpect.run('ye s')
# Re-set the alarm. (This is never reached. The 'yes'
# command runs forever and is not interrupted by the alarm)
signal.alarm(0)
signal.signal(s ignal.SIGALRM, handler)
t = threading.Threa d(target=runyes )
t.start()
t.join()
# END of code
Note that if the 'yes' command is substituted for 'sleep 10', the code works
perfectly.
Why can't the 'yes' command be interrupted using the SIGALRM method when the
sleep command can? I realize that noone would really run 'yes' but what if
a user account has a buggy .bashrc and loops forever? Just one thread
locking up like this holds up all the others.
Any ideas or suggestions on how to handle such situations in a
multi-threaded way would be appreciated.
Cheers.
Adrian Casey.
Alice Springs Linux User Goup. http://www.aslug.org.au 7  11082 
Adrian Casey <ad**********@i nternode.on.net wrote:
I have a multi-threaded python application which uses pexpect to connect to
multiple systems concurrently. Each thread within my application is a
connection to a remote system. The problem is when one of the child
threads runs a command which generates an unlimited amount of output. The
classic example of this is the "yes" command. If you
execute "pexpect.run('y es')", your cpu will sit at 100% forever.
Here is a simple multi-threaded program using pexpect which demonstrates the
problem. The command 'yes' is run in a thread. The parent says that when
the alarm goes off, run the handler function. The thread sets the alarm to
trigger after 5 seconds.
1) Don't ever mix threads and signals - you are heading for trouble!
2) pexpect has a timeout parameter exactly for this case
import os, pexpect, threading
def runyes():
print "Running yes command..."
pexpect.run('ye s', timeout=5)
t = threading.Threa d(target=runyes )
t.start()
t.join()
--
Nick Craig-Wood <ni**@craig-wood.com-- http://www.craig-wood.com/nick
Nick Craig-Wood wrote:
Adrian Casey <ad**********@i nternode.on.net wrote:
> I have a multi-threaded python application which uses pexpect to connect
to
multiple systems concurrently. Each thread within my application is a
connection to a remote system. The problem is when one of the child
threads runs a command which generates an unlimited amount of output.
The
classic example of this is the "yes" command. If you
execute "pexpect.run('y es')", your cpu will sit at 100% forever.
Here is a simple multi-threaded program using pexpect which demonstrates
the
problem. The command 'yes' is run in a thread. The parent says that
when
the alarm goes off, run the handler function. The thread sets the alarm
to trigger after 5 seconds.
1) Don't ever mix threads and signals - you are heading for trouble!
2) pexpect has a timeout parameter exactly for this case
import os, pexpect, threading
def runyes():
print "Running yes command..."
pexpect.run('ye s', timeout=5)
t = threading.Threa d(target=runyes )
t.start()
t.join()
The timeout parameter will not work in this case. If you run the sample
code above, it will run forever. The 'yes' command presents a class of
command which can not be easily be handled by pexpect. As far as I know,
mixing threads and signals is OK provided the parent creates the alarm.
Adrian.
Adrian Casey <ad**********@i nternode.on.net wrote:
Adrian Casey <ad**********@i nternode.on.net wrote:
import os, pexpect, threading
def runyes():
print "Running yes command..."
pexpect.run('ye s', timeout=5)
t = threading.Threa d(target=runyes )
t.start()
t.join()
The timeout parameter will not work in this case. If you run the sample
code above, it will run forever.
The above runs just fine for me, stopping after 5 seconds. Did you
try it?
The 'yes' command presents a class of command which can not be
easily be handled by pexpect.
Worked for me under Debian/testing.
As far as I know, mixing threads and signals is OK provided the
parent creates the alarm.
There are so many pitfalls here that I advise you not to try. From
the linuxthreads FAQ
J.3: How shall I go about mixing signals and threads in my program?
The less you mix them, the better. Notice that all pthread_*
functions are not async-signal safe, meaning that you should not
call them from signal handlers. This recommendation is not to be
taken lightly: your program can deadlock if you call a pthread_*
function from a signal handler!
The only sensible things you can do from a signal handler is set a
global flag, or call sem_post on a semaphore, to record the delivery
of the signal. The remainder of the program can then either poll the
global flag, or use sem_wait() and sem_trywait() on the semaphore.
Another option is to do nothing in the signal handler, and dedicate
one thread (preferably the initial thread) to wait synchronously for
signals, using sigwait(), and send messages to the other threads
accordingly.
Note also that the signal can be delivered to any thread which
complicates things.
--
Nick Craig-Wood <ni**@craig-wood.com-- http://www.craig-wood.com/nick
Nick Craig-Wood wrote:
The only sensible things you can do from a signal handler is set a
global flag, or call sem_post on a semaphore, to record the delivery
of the signal. The remainder of the program can then either poll the
global flag, or use sem_wait() and sem_trywait() on the semaphore.
but that's exactly what Python's signal handlers do, right ?
(the interpreter uses a "pending call" queue to collect events, and
executes them from the interpreter main loop in a controlled fashion).
</F>
Fredrik Lundh <fr*****@python ware.comwrote:
Nick Craig-Wood wrote:
The only sensible things you can do from a signal handler is set a
global flag, or call sem_post on a semaphore, to record the delivery
of the signal. The remainder of the program can then either poll the
global flag, or use sem_wait() and sem_trywait() on the semaphore.
but that's exactly what Python's signal handlers do, right ?
(the interpreter uses a "pending call" queue to collect events, and
executes them from the interpreter main loop in a controlled
fashion).
Yes you are absolutely right
From http://docs.python.org/lib/module-signal.html
Some care must be taken if both signals and threads are used in
the same program. The fundamental thing to remember in using
signals and threads simultaneously is: always perform signal()
operations in the main thread of execution. Any thread can perform
an alarm(), getsignal(), or pause(); only the main thread can set
a new signal handler, and the main thread will be the only one to
receive signals (this is enforced by the Python signal module,
even if the underlying thread implementation supports sending
signals to individual threads). This means that signals can't be
used as a means of inter-thread communication. Use locks instead.
--
Nick Craig-Wood <ni**@craig-wood.com-- http://www.craig-wood.com/nick
Nick Craig-Wood wrote:
Adrian Casey <ad**********@i nternode.on.net wrote:
Adrian Casey <ad**********@i nternode.on.net wrote:
import os, pexpect, threading
def runyes():
print "Running yes command..."
pexpect.run('ye s', timeout=5)
t = threading.Threa d(target=runyes )
t.start()
t.join()
The timeout parameter will not work in this case. If you run the sample
code above, it will run forever.
The above runs just fine for me, stopping after 5 seconds. Did you
try it?
> The 'yes' command presents a class of command which can not be
easily be handled by pexpect.
Worked for me under Debian/testing.
>As far as I know, mixing threads and signals is OK provided the
parent creates the alarm.
There are so many pitfalls here that I advise you not to try. From
the linuxthreads FAQ
J.3: How shall I go about mixing signals and threads in my program?
The less you mix them, the better. Notice that all pthread_*
functions are not async-signal safe, meaning that you should not
call them from signal handlers. This recommendation is not to be
taken lightly: your program can deadlock if you call a pthread_*
function from a signal handler!
The only sensible things you can do from a signal handler is set a
global flag, or call sem_post on a semaphore, to record the delivery
of the signal. The remainder of the program can then either poll the
global flag, or use sem_wait() and sem_trywait() on the semaphore.
Another option is to do nothing in the signal handler, and dedicate
one thread (preferably the initial thread) to wait synchronously for
signals, using sigwait(), and send messages to the other threads
accordingly.
Note also that the signal can be delivered to any thread which
complicates things.
I'm running Kubuntu 06-06 with python 2.4.3 and the above code runs forever
at 100% cpu utilization. I shall look into semaphores. However, that means
another thread whose sole purpose is to watch the semaphore.
Thanks for you help.
Cheers.
Adrian.
Adrian Casey <ad**********@i nternode.on.net wrote:
I'm running Kubuntu 06-06 with python 2.4.3 and the above code runs forever
at 100% cpu utilization.
Interesting... I wonder if that is a fixed bug.
On Debian/etch with python-pexpect 2.1-1 I get
Python 2.4.4c0 (#2, Jul 30 2006, 15:43:58)
[GCC 4.1.2 20060715 (prerelease) (Debian 4.1.1-9)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.
>>import os, pexpect, threading
def runyes():
... print "Running yes command..."
... pexpect.run('ye s', timeout=5)
...
>>t = threading.Threa d(target=runyes )
t.start()
Running yes command...
t.join()
>>>
Wheras on Ubuntu/dapper with python-pexpect 0.999-5ubuntu2 I get
Python 2.4.3 (#2, Apr 27 2006, 14:43:58)
[GCC 4.0.3 (Ubuntu 4.0.3-1ubuntu5)] on linux2
Type "help", "copyright" , "credits" or "license" for more information.
>>import os, pexpect, threading
def runyes():
... print "Running yes command..."
... pexpect.run('ye s', timeout=5)
...
>>t = threading.Threa d(target=runyes )
t.start()
Running yes command...
t.join()
[never returns]
I'd guess at differences between the pexpect versions. You could try
the pexpect from debian/testing easily enough I expect.
--
Nick Craig-Wood <ni**@craig-wood.com-- http://www.craig-wood.com/nick This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics | |
by: Ishwar Rattan |
last post by:
System is Mandrale 9.1 Linux with Pyhton-2.2.3
I want to abort the stdin-read operation if no input is avalable. Is it possible
to achieve this?
Sample code is given below.
-ishwar
---
import sys, signal, time
|
by: Ishwar Rattan |
last post by:
System Mandrake-9.1 Linux with Python-2.2.3
Tyring to time out from a non respnding server using SOCK_STREAM
Source is shown below
The program quits after first receipt of alram signal with messge
socket.error(4, 'Interrupted system call')
|
by: Klaus Neuner |
last post by:
Hello,
consider the following two programs:
# (1)
import sys, signal
def alarm_handler(signum, frame):
raise
|
by: mkent |
last post by:
I'm trying to use signal.alarm to stop a run-away os.system command.
Can anyone exlain the following behavior?
Given following the trivial program:
import os
import signal
def timeoutHandler(signum, frame):
print "Timeout"
|
by: rh0dium |
last post by:
Hi all,
I don't understand the signal module. I guess I understand what it
does in principle but I can't figure out how to use it to timeout an
external rsh command after a 5 seconds. Does anyone know how to do
this.
Here is what I have so far - which is largely based on the example on
the man page..
def handler(signum, frame):
| | |
by: rh0dium |
last post by:
Hi all,
Another newbie question. So you can't use signals on threads but you
can use select. The reason I want to do this in the first place it I
need a timeout. Fundamentally I want to run a command on another
machine, but I need a timeout. I have to do this to a LOT of machines
( > 3000 ) and threading becomes necessary for timeliess. So I created
a function which works with signals ( until you throw threading at it..
) but I...
|
by: Jackie |
last post by:
Hi everyone,
I'd like to know when and how signals are used (e.g. SIGFPE, SIGABRT,
SIGTERM, SIGSEGV, SIGINT)? Thank you so much.
|
by: norm4h8 |
last post by:
Hi!
I have a question about how to create a process in such a way that it
would terminate itself if its wated for input for too long.
Here is the story. I have 2 different files, say R.c and S.c, which
have a named pipe between them and send each other signals. R process
starts first, it opens a named pipe and waits for S to connect. S
connects to the pipe, writes something to it and sends the signal to R
indicating that R can read....
|
by: david |
last post by:
I'm developing a program that runs using an asyncore loop. Right now
I can adequately terminate it using Control-C, but as things get
refined I need a better way to stop it. I've developed another
program that executes it as a child process using popen2.Popen4(). I
was attempting to use signals to stop it (using os.kill()) but I keep
running into a problem where sending the signal causes an infinite
loop of printing the message...
|
by: marktang |
last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look !
Part I. Meaning of...
|
by: Hystou |
last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it.
First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
| | |
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed.
This is as boiled down as I can make it.
Here is my compilation command:
g++-12 -std=c++20 -Wnarrowing bit_field.cpp
Here is the code in...
|
by: Hystou |
last post by:
Overview:
Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
|
by: agi2029 |
last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own....
Now, this would greatly impact the work of software developers. The idea...
|
by: TSSRALBI |
last post by:
Hello
I'm a network technician in training and I need your help.
I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs.
The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols.
I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
|
by: adsilva |
last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
| | |
by: muto222 |
last post by:
How can i add a mobile payment intergratation into php mysql website.
| |
