473,327 Members | 1,967 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,327 software developers and data experts.

Subprocess and pipe-fork-exec primitive

Hi,

I am working with the subprocess.py module in Python 2.4.4 and I am
confused about it's functionality. It uses the standard pipe-fork-exec
method to start a subprocess:

# create pipes

pid = fork()

if pid == 0:
# child
exec(...)

# parent
status = waitpid(pid, 0)

From my experience, this primitive will fail with 'no child
processes' at the waitpid call if the forked child dies very quickly -
before the parent is scheduled back for execution. This seems to happen
because Python has a default SIGCHLD handler that, in this case, will
reap the process before the parent has the chance to do it.

I would like to know if this is correct, or am I missing something here?

---
Rafael.

Jul 30 '07 #1
2 2777
From my experience, this primitive will fail with 'no child processes'
at the waitpid call if the forked child dies very quickly - before the
parent is scheduled back for execution. This seems to happen because
Python has a default SIGCHLD handler that, in this case, will reap the
process before the parent has the chance to do it.
What operating system is your experience from? On a POSIX system,
this should not happen - i.e. delivery of SIGCHLD should not cause
to make the child waited-for. Python itself does not perform wait()
in response to SIGCHLD.
I would like to know if this is correct, or am I missing something here?
You must be missing something, although I'm uncertain what precisely
that is.

Regards,
Martin
Jul 31 '07 #2
Hi Martin,

the operating system I'm using is SUSE Linux 10, kernel 2.6.13.

You're right, I was missing something. After you told me that it
couldn't be Python preforming wait() on SIGCHLD, I decided to
investigate further.

My application requires access to a Informix database, and uses
informixdb. The problem seems to be related to that module. I wrote a
small piece of code to test it:

---------------------------------------------------
#!/usr/bin/env python

from os import fork, execl, waitpid
from informixdb import connect

try:
conf = {}
conf['dsn'] = 'db@server'
conf['user'] = 'user'
conf['password'] = 'password'
connection = connect(**conf)

except:
pass

pid = fork()

if pid == 0:
# Child
execl("/bin/sh", "/bin/sh", "-c", "true")

# Parent
waitpid(pid, 0)

print pid
---------------------------------------------------

If you run the code above multiple times, some runs will trigger
exceptions on the waitpid calls. Commenting the call to
informixdb.connect(), no exceptions are triggered. I am concluding that
informixdb, not Python, is handling the signals and reaping the
subprocesses.

Now Martin, do you think I can use informixdb.py and subprocess.py in
the same application? I was thinking on forking subprocesses from the
main thread and using other threads to access the Informix database,
would that work?

Thanks,
Rafael.

Martin v. Löwis escreveu:
> From my experience, this primitive will fail with 'no child processes'
at the waitpid call if the forked child dies very quickly - before the
parent is scheduled back for execution. This seems to happen because
Python has a default SIGCHLD handler that, in this case, will reap the
process before the parent has the chance to do it.

What operating system is your experience from? On a POSIX system,
this should not happen - i.e. delivery of SIGCHLD should not cause
to make the child waited-for. Python itself does not perform wait()
in response to SIGCHLD.
> I would like to know if this is correct, or am I missing something here?

You must be missing something, although I'm uncertain what precisely
that is.

Regards,
Martin


Aug 1 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
by: Marc Carter | last post by:
I am trying to rewrite a PERL automation which started a "monitoring" application on many machines, via RSH, and then multiplexed their collective outputs to stdout. In production there are lots...
3
by: Darren Dale | last post by:
I'm a developer on the matplotlib project, and I am having trouble with the subprocess module on windows (Python 2.4.2 on winXP). No trouble to report with linux. I need to use _subprocess instead...
5
by: Cameron Laird | last post by:
Question: import subprocess, StringIO input = StringIO.StringIO("abcdefgh\nabc\n") # I don't know of a compact, evocative, and # cross-platform way to exhibit this behavior. # For now, depend...
5
by: Grant Edwards | last post by:
I'm trying to use the py-gnuplot module on windows, and have been unable to get it to work reliably under Win2K and WinXP. By default, it uses popen(gnuplotcmd,'w'), but in some situations that...
9
by: Phoe6 | last post by:
Hi all, Consider this scenario, where in I need to use subprocess to execute a command like 'ping 127.0.0.1' which will have a continuous non- terminating output in Linux. # code # This...
12
by: bhunter | last post by:
Hi, I've used subprocess with 2.4 several times to execute a process, wait for it to finish, and then look at its output. Now I want to spawn the process separately, later check to see if it's...
1
by: Mrown | last post by:
Hi, I'm currently writing a python program that relies on a CLI program. What I'm currently doing is using subprocess.Popen on Python 2.5.1. Here's the line that I'm currently running: child...
7
by: skunkwerk | last post by:
Hi, i'm trying to call subprocess.popen on the 'rename' function in linux. When I run the command from the shell, like so: rename -vn 's/\.htm$/\.html/' *.htm it works fine... however when I...
25
by: Jeremy Banks | last post by:
Hi. I wondered if anyone knew the rationale behind the naming of the Popen class in the subprocess module. Popen sounds like the a suitable name for a function that created a subprocess, but the...
0
by: Amanda Jamin | last post by:
Subprocess issues with platform independence Postby ajamin on Wed Oct 08, 2008 10:46 am I am writing a python script that will act as a wrapper for another program. The python script will...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.