Subprocess and pipe-fork-exec primitive

[Rafael Giannetti Viotti]

Hi,

I am working with the subprocess.py module in Python 2.4.4 and I am
confused about it's functionality. It uses the standard pipe-fork-exec
method to start a subprocess:

# create pipes

pid = fork()

if pid == 0:
# child
exec(...)

# parent
status = waitpid(pid, 0)

From my experience, this primitive will fail with 'no child
processes' at the waitpid call if the forked child dies very quickly -
before the parent is scheduled back for execution. This seems to happen
because Python has a default SIGCHLD handler that, in this case, will
reap the process before the parent has the chance to do it.

I would like to know if this is correct, or am I missing something here?

---
Rafael.

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

From my experience, this primitive will fail with 'no child processes'

at the waitpid call if the forked child dies very quickly - before the
parent is scheduled back for execution. This seems to happen because
Python has a default SIGCHLD handler that, in this case, will reap the
process before the parent has the chance to do it.

What operating system is your experience from? On a POSIX system,
this should not happen - i.e. delivery of SIGCHLD should not cause
to make the child waited-for. Python itself does not perform wait()
in response to SIGCHLD.

I would like to know if this is correct, or am I missing something here?

You must be missing something, although I'm uncertain what precisely
that is.

Regards,
Martin

[Rafael V.]

Hi Martin,

the operating system I'm using is SUSE Linux 10, kernel 2.6.13.

You're right, I was missing something. After you told me that it
couldn't be Python preforming wait() on SIGCHLD, I decided to
investigate further.

My application requires access to a Informix database, and uses
informixdb. The problem seems to be related to that module. I wrote a
small piece of code to test it:

---------------------------------------------------
#!/usr/bin/env python

from os import fork, execl, waitpid
from informixdb import connect

try:
conf = {}
conf['dsn'] = 'db@server'
conf['user'] = 'user'
conf['password'] = 'password'
connection = connect(**conf)

except:
pass

pid = fork()

if pid == 0:
# Child
execl("/bin/sh", "/bin/sh", "-c", "true")

# Parent
waitpid(pid, 0)

print pid
---------------------------------------------------

If you run the code above multiple times, some runs will trigger
exceptions on the waitpid calls. Commenting the call to
informixdb.connect(), no exceptions are triggered. I am concluding that
informixdb, not Python, is handling the signals and reaping the
subprocesses.

Now Martin, do you think I can use informixdb.py and subprocess.py in
the same application? I was thinking on forking subprocesses from the
main thread and using other threads to access the Informix database,
would that work?

Thanks,
Rafael.

Martin v. Löwis escreveu:

> From my experience, this primitive will fail with 'no child processes'
at the waitpid call if the forked child dies very quickly - before the
parent is scheduled back for execution. This seems to happen because
Python has a default SIGCHLD handler that, in this case, will reap the
process before the parent has the chance to do it.

What operating system is your experience from? On a POSIX system,
this should not happen - i.e. delivery of SIGCHLD should not cause
to make the child waited-for. Python itself does not perform wait()
in response to SIGCHLD.

> I would like to know if this is correct, or am I missing something here?

You must be missing something, although I'm uncertain what precisely
that is.

Regards,
Martin