Proper way to kill child processes

[Bob Swerdlow]

My application starts up a number of processes for various purposes using:
self.popen = popen2.Popen3("/usr/local/bin/python -O "myscript.py")
and then shuts them down when appropriate with
os.kill(self.popen.pid, signal.SIGTERM)
Everything works fine on MacOSX. However, I'm doing a port to Solaris (so I
can run it on my web site) and find that the child processes are not
stopping! Solaris is creating TWO new processes: one for the SHELL and then
another started by the shell to run my Python script. The os.kill() is
killing the shell process, not the one running my Python code.

Actually, I really want to kill both of these processes, but I only have the
pid for the shell process. I cannot kill the whole process group because
that kills the main process, too (I tried it).

So, what is the best way to kill both the shell process (whose pid is
available from the Popen3 object) and its child process that is running my
Python script? It looks like the python script process id is always one
greater than the shell process id of the shell process, but I'm sure I
cannot rely on that.

Thanks,

Bob Swerdlow
COO
Transpose
rs*******@transpose.com
207-781-8284
http://www.transpose.com

----------------------------------
Fight Spam!
Add this link to your signature (as I did): http://wecanstopspam.org
Click through to find out more.
----------------------------------

[Jean Brouwers]

Try a different format for the cmd:

self.popen = popen2.Popen3(["/usr/local/bin/python", "-O",
"myscript.py"])

i.e. pass the cmd as a list of the individual arguments. That avoids
invoking the sh. See <your_python_install>/Lib/popen2.py for more
details.

/Jean Brouwers
ProphICy Semiconductor, Inc.

In article <ma*************************************@python.or g>, Bob
Swerdlow <rs*******@transpose.com> wrote:

My application starts up a number of processes for various purposes using:
self.popen = popen2.Popen3("/usr/local/bin/python -O "myscript.py")
and then shuts them down when appropriate with
os.kill(self.popen.pid, signal.SIGTERM)
Everything works fine on MacOSX. However, I'm doing a port to Solaris (so I
can run it on my web site) and find that the child processes are not
stopping! Solaris is creating TWO new processes: one for the SHELL and then
another started by the shell to run my Python script. The os.kill() is
killing the shell process, not the one running my Python code.

Actually, I really want to kill both of these processes, but I only have the
pid for the shell process. I cannot kill the whole process group because
that kills the main process, too (I tried it).

So, what is the best way to kill both the shell process (whose pid is
available from the Popen3 object) and its child process that is running my
Python script? It looks like the python script process id is always one
greater than the shell process id of the shell process, but I'm sure I
cannot rely on that.

Thanks,

Bob Swerdlow
COO
Transpose
rs*******@transpose.com
207-781-8284
http://www.transpose.com

----------------------------------
Fight Spam!
Add this link to your signature (as I did): http://wecanstopspam.org
Click through to find out more.
----------------------------------

[Mathias Waack]

Bob Swerdlow wrote:

My application starts up a number of processes for various purposes
using:
self.popen = popen2.Popen3("/usr/local/bin/python -O
"myscript.py")

This works for me:

self.popen = popen2.Popen3(["python", "/usr/local/bin/python",
"-O", "myscript.py"])

But I don't know if its a hack or a desired feature.

Mathias

[Jean Brouwers]

It should be the other way around

self.popen = popen2.Popen3(["/usr/local/bin/python",
"-O", "myscript.py"])

Item [0] must be the path to the executable and items[1:] are the
arguments. See the Popen3._run_cmd() method in Lib/popen2.py
partially copied here:

<pre>
def _run_child(self, cmd):
if isinstance(cmd, basestring):
cmd = ['/bin/sh', '-c', cmd]
.....
try:
os.execvp(cmd[0], cmd)
finally:
os._exit(1)
</pre>

/Jean Brouwers
ProphICy Semiconductor, Inc.

In article <17************@valpo.de>, Mathias Waack <M.*****@gmx.de>
wrote:

Bob Swerdlow wrote:

My application starts up a number of processes for various purposes
using:
self.popen = popen2.Popen3("/usr/local/bin/python -O
"myscript.py")

This works for me:

self.popen = popen2.Popen3(["python", "/usr/local/bin/python",
"-O", "myscript.py"])

But I don't know if its a hack or a desired feature.

Mathias

[Jean Brouwers]

Another issue *might* be that the TERM signal is passed to the Pyton
process but ignored on Solaris. Try using a different signal like
SIGQUIT or maybe even SIGKILL. The shell process and all its child
processes should be terminated, by convention on *nix. Also on RedHat
Linux it works.

There may be secondary issue with killing processes created by
popen2.Popen3(). The exit status of the child processes should be
checked to avoid zombies. After killing a child process do call the
wait() or poll() method. The wait() methods will wait forever and may
cause the parent process to hang. To avoid that use poll() in a loop
until the return value is non-negative. The sample code below may
help.

In any case, do use the list format for the cmd. That is better if
there is no compelling need for the intermediate shell process.

/Jean Brouwers
ProphICy Semiconductor, Inc.

PS) Sample code to avoid zombie (on *nix):

<pre>
p = self.popen
os.kill(p.pid, signal.SIGTERM)
t = 2.5 # max wait time in secs
while p.poll() < 0:
if t > 0.5:
t -= 0.25
os.sleep(0.25)
else: # still there, force kill
os.kill(p.pid, signal.SIGKILL)
os.sleep(0.5)
p.poll() # final try
break
</pre>

In article <ma*************************************@python.or g>, Bob
Swerdlow <rs*******@transpose.com> wrote:

My application starts up a number of processes for various purposes using:
self.popen = popen2.Popen3("/usr/local/bin/python -O "myscript.py")
and then shuts them down when appropriate with
os.kill(self.popen.pid, signal.SIGTERM)
Everything works fine on MacOSX. However, I'm doing a port to Solaris (so I
can run it on my web site) and find that the child processes are not
stopping! Solaris is creating TWO new processes: one for the SHELL and then
another started by the shell to run my Python script. The os.kill() is
killing the shell process, not the one running my Python code.

Actually, I really want to kill both of these processes, but I only have the
pid for the shell process. I cannot kill the whole process group because
that kills the main process, too (I tried it).

So, what is the best way to kill both the shell process (whose pid is
available from the Popen3 object) and its child process that is running my
Python script? It looks like the python script process id is always one
greater than the shell process id of the shell process, but I'm sure I
cannot rely on that.

Thanks,

Bob Swerdlow
COO
Transpose
rs*******@transpose.com
207-781-8284
http://www.transpose.com

----------------------------------
Fight Spam!
Add this link to your signature (as I did): http://wecanstopspam.org
Click through to find out more.
----------------------------------

[P]

Bob Swerdlow wrote:

My application starts up a number of processes for various purposes using:
self.popen = popen2.Popen3("/usr/local/bin/python -O "myscript.py")
and then shuts them down when appropriate with
os.kill(self.popen.pid, signal.SIGTERM)
Everything works fine on MacOSX. However, I'm doing a port to Solaris (so I
can run it on my web site) and find that the child processes are not
stopping! Solaris is creating TWO new processes: one for the SHELL and then
another started by the shell to run my Python script. The os.kill() is
killing the shell process, not the one running my Python code.

Actually, I really want to kill both of these processes, but I only have the
pid for the shell process. I cannot kill the whole process group because
that kills the main process, too (I tried it).

Try to get the popen2.Popen3() implementation to create it's own
process group. You can do this by adding an os.setpgrp() call
or maybe you can change the "sh -c" -> "sh -mc"

Pádraig.

[P]

Bob Swerdlow wrote:

My application starts up a number of processes for various purposes using:
self.popen = popen2.Popen3("/usr/local/bin/python -O "myscript.py")
and then shuts them down when appropriate with
os.kill(self.popen.pid, signal.SIGTERM)
Everything works fine on MacOSX. However, I'm doing a port to Solaris (so I
can run it on my web site) and find that the child processes are not
stopping! Solaris is creating TWO new processes: one for the SHELL andthen
another started by the shell to run my Python script. The os.kill() is
killing the shell process, not the one running my Python code.

Actually, I really want to kill both of these processes, but I only have the
pid for the shell process. I cannot kill the whole process group because
that kills the main process, too (I tried it).

Try to get the popen2.Popen3() implementation to create it's own
process group. You can do this by adding an os.setpgrp() call
or maybe you can change the "sh -c" -> "sh -mc"

Pádraig.