Connecting Tech Pros Worldwide Forums | Help | Site Map

Proper way to kill child processes

Bob Swerdlow
Guest
 
Posts: n/a
#1: Jul 18 '05
My application starts up a number of processes for various purposes using:
self.popen = popen2.Popen3("/usr/local/bin/python -O "myscript.py")
and then shuts them down when appropriate with
os.kill(self.popen.pid, signal.SIGTERM)
Everything works fine on MacOSX. However, I'm doing a port to Solaris (so I
can run it on my web site) and find that the child processes are not
stopping! Solaris is creating TWO new processes: one for the SHELL and then
another started by the shell to run my Python script. The os.kill() is
killing the shell process, not the one running my Python code.

Actually, I really want to kill both of these processes, but I only have the
pid for the shell process. I cannot kill the whole process group because
that kills the main process, too (I tried it).

So, what is the best way to kill both the shell process (whose pid is
available from the Popen3 object) and its child process that is running my
Python script? It looks like the python script process id is always one
greater than the shell process id of the shell process, but I'm sure I
cannot rely on that.

Thanks,

Bob Swerdlow
COO
Transpose
rswerdlow@transpose.com
207-781-8284
http://www.transpose.com

----------------------------------
Fight Spam!
Add this link to your signature (as I did): http://wecanstopspam.org
Click through to find out more.
----------------------------------




Jean Brouwers
Guest
 
Posts: n/a
#2: Jul 18 '05

re: Proper way to kill child processes



Try a different format for the cmd:

self.popen = popen2.Popen3(["/usr/local/bin/python", "-O",
"myscript.py"])

i.e. pass the cmd as a list of the individual arguments. That avoids
invoking the sh. See <your_python_install>/Lib/popen2.py for more
details.

/Jean Brouwers
ProphICy Semiconductor, Inc.



In article <mailman.10.1087331713.21521.python-list@python.org>, Bob
Swerdlow <rswerdlow@transpose.com> wrote:
[color=blue]
> My application starts up a number of processes for various purposes using:
> self.popen = popen2.Popen3("/usr/local/bin/python -O "myscript.py")
> and then shuts them down when appropriate with
> os.kill(self.popen.pid, signal.SIGTERM)
> Everything works fine on MacOSX. However, I'm doing a port to Solaris (so I
> can run it on my web site) and find that the child processes are not
> stopping! Solaris is creating TWO new processes: one for the SHELL and then
> another started by the shell to run my Python script. The os.kill() is
> killing the shell process, not the one running my Python code.
>
> Actually, I really want to kill both of these processes, but I only have the
> pid for the shell process. I cannot kill the whole process group because
> that kills the main process, too (I tried it).
>
> So, what is the best way to kill both the shell process (whose pid is
> available from the Popen3 object) and its child process that is running my
> Python script? It looks like the python script process id is always one
> greater than the shell process id of the shell process, but I'm sure I
> cannot rely on that.
>
> Thanks,
>
> Bob Swerdlow
> COO
> Transpose
> rswerdlow@transpose.com
> 207-781-8284
> http://www.transpose.com
>
> ----------------------------------
> Fight Spam!
> Add this link to your signature (as I did): http://wecanstopspam.org
> Click through to find out more.
> ----------------------------------[/color]
Mathias Waack
Guest
 
Posts: n/a
#3: Jul 18 '05

re: Proper way to kill child processes


Bob Swerdlow wrote:
[color=blue]
> My application starts up a number of processes for various purposes
> using:
> self.popen = popen2.Popen3("/usr/local/bin/python -O
> "myscript.py")[/color]

This works for me:

self.popen = popen2.Popen3(["python", "/usr/local/bin/python",
"-O", "myscript.py"])

But I don't know if its a hack or a desired feature.

Mathias
Jean Brouwers
Guest
 
Posts: n/a
#4: Jul 18 '05

re: Proper way to kill child processes



It should be the other way around

self.popen = popen2.Popen3(["/usr/local/bin/python",
"-O", "myscript.py"])

Item [0] must be the path to the executable and items[1:] are the
arguments. See the Popen3._run_cmd() method in Lib/popen2.py
partially copied here:

<pre>
def _run_child(self, cmd):
if isinstance(cmd, basestring):
cmd = ['/bin/sh', '-c', cmd]
.....
try:
os.execvp(cmd[0], cmd)
finally:
os._exit(1)
</pre>

/Jean Brouwers
ProphICy Semiconductor, Inc.

In article <1774q1-63c.ln1@valpo.de>, Mathias Waack <M.Waack@gmx.de>
wrote:
[color=blue]
> Bob Swerdlow wrote:
>[color=green]
> > My application starts up a number of processes for various purposes
> > using:
> > self.popen = popen2.Popen3("/usr/local/bin/python -O
> > "myscript.py")[/color]
>
> This works for me:
>
> self.popen = popen2.Popen3(["python", "/usr/local/bin/python",
> "-O", "myscript.py"])
>
> But I don't know if its a hack or a desired feature.
>
> Mathias[/color]
Jean Brouwers
Guest
 
Posts: n/a
#5: Jul 18 '05

re: Proper way to kill child processes



Another issue *might* be that the TERM signal is passed to the Pyton
process but ignored on Solaris. Try using a different signal like
SIGQUIT or maybe even SIGKILL. The shell process and all its child
processes should be terminated, by convention on *nix. Also on RedHat
Linux it works.

There may be secondary issue with killing processes created by
popen2.Popen3(). The exit status of the child processes should be
checked to avoid zombies. After killing a child process do call the
wait() or poll() method. The wait() methods will wait forever and may
cause the parent process to hang. To avoid that use poll() in a loop
until the return value is non-negative. The sample code below may
help.

In any case, do use the list format for the cmd. That is better if
there is no compelling need for the intermediate shell process.

/Jean Brouwers
ProphICy Semiconductor, Inc.

PS) Sample code to avoid zombie (on *nix):

<pre>
p = self.popen
os.kill(p.pid, signal.SIGTERM)
t = 2.5 # max wait time in secs
while p.poll() < 0:
if t > 0.5:
t -= 0.25
os.sleep(0.25)
else: # still there, force kill
os.kill(p.pid, signal.SIGKILL)
os.sleep(0.5)
p.poll() # final try
break
</pre>



In article <mailman.10.1087331713.21521.python-list@python.org>, Bob
Swerdlow <rswerdlow@transpose.com> wrote:
[color=blue]
> My application starts up a number of processes for various purposes using:
> self.popen = popen2.Popen3("/usr/local/bin/python -O "myscript.py")
> and then shuts them down when appropriate with
> os.kill(self.popen.pid, signal.SIGTERM)
> Everything works fine on MacOSX. However, I'm doing a port to Solaris (so I
> can run it on my web site) and find that the child processes are not
> stopping! Solaris is creating TWO new processes: one for the SHELL and then
> another started by the shell to run my Python script. The os.kill() is
> killing the shell process, not the one running my Python code.
>
> Actually, I really want to kill both of these processes, but I only have the
> pid for the shell process. I cannot kill the whole process group because
> that kills the main process, too (I tried it).
>
> So, what is the best way to kill both the shell process (whose pid is
> available from the Popen3 object) and its child process that is running my
> Python script? It looks like the python script process id is always one
> greater than the shell process id of the shell process, but I'm sure I
> cannot rely on that.
>
> Thanks,
>
> Bob Swerdlow
> COO
> Transpose
> rswerdlow@transpose.com
> 207-781-8284
> http://www.transpose.com
>
> ----------------------------------
> Fight Spam!
> Add this link to your signature (as I did): http://wecanstopspam.org
> Click through to find out more.
> ----------------------------------
>
>
>[/color]
P@draigBrady.com
Guest
 
Posts: n/a
#6: Jul 18 '05

re: Proper way to kill child processes


Bob Swerdlow wrote:[color=blue]
> My application starts up a number of processes for various purposes using:
> self.popen = popen2.Popen3("/usr/local/bin/python -O "myscript.py")
> and then shuts them down when appropriate with
> os.kill(self.popen.pid, signal.SIGTERM)
> Everything works fine on MacOSX. However, I'm doing a port to Solaris (so I
> can run it on my web site) and find that the child processes are not
> stopping! Solaris is creating TWO new processes: one for the SHELL and then
> another started by the shell to run my Python script. The os.kill() is
> killing the shell process, not the one running my Python code.
>
> Actually, I really want to kill both of these processes, but I only have the
> pid for the shell process. I cannot kill the whole process group because
> that kills the main process, too (I tried it).[/color]

Try to get the popen2.Popen3() implementation to create it's own
process group. You can do this by adding an os.setpgrp() call
or maybe you can change the "sh -c" -> "sh -mc"

Pádraig.
P@draigBrady.com
Guest
 
Posts: n/a
#7: Jul 18 '05

re: Proper way to kill child processes


Bob Swerdlow wrote:[color=blue]
> My application starts up a number of processes for various purposes using:
> self.popen = popen2.Popen3("/usr/local/bin/python -O "myscript.py")
> and then shuts them down when appropriate with
> os.kill(self.popen.pid, signal.SIGTERM)
> Everything works fine on MacOSX. However, I'm doing a port to Solaris (so I
> can run it on my web site) and find that the child processes are not
> stopping! Solaris is creating TWO new processes: one for the SHELL andthen
> another started by the shell to run my Python script. The os.kill() is
> killing the shell process, not the one running my Python code.
>
> Actually, I really want to kill both of these processes, but I only have the
> pid for the shell process. I cannot kill the whole process group because
> that kills the main process, too (I tried it).[/color]

Try to get the popen2.Popen3() implementation to create it's own
process group. You can do this by adding an os.setpgrp() call
or maybe you can change the "sh -c" -> "sh -mc"

Pádraig.

Closed Thread