473,288 Members | 1,750 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,288 software developers and data experts.

Main-process doesn't wait for child-processes

Hi.

I created a little script:

for currenturl in sys.argv[1:]:
pid = os.fork()
if pid == 0:
signal.alarm(10)
do_something() # placeholder for the download and print
routine
break

This script is started by ./script.py http://www.website1.com
http://www.website2.com
As you see it creates child-processes for loading all websites specified on
the command line. (So all websites are loaded in parallel.) Each
child-process simply load a websites and print the contents to STDOUT (above
replaced by do_something).
(With signal.alarm(10) I make sure that each child-process will not run
longer than 10 seconds.)
But now I have a difficult problem: This script is executed by a
comand-line-function (shell_exec) in PHP, the response of the Python-script
is used inside the PHP application. In the python script the main process
seem to stop immediately after starting the script, only the child-processes
seem to run. Because of this my PHP application will not wait untill the
child-processes have finised and so I can't use the contents of the
websites. (I think the solution should be that the main-process waits untill
the child-processes terminate.)

Now my question is: How can I force the main-process to wait untill every
child-process is finished?
I tried:

for currenturl in sys.argv[1:]:
pid = os.fork()
if pid == 0:
signal.alarm(10)
do_something() # placeholder for the download and print
routine
break
else:
os.wait()

The version above behaves like a script that loads all given websites in
sequence...
I also tried:

for currenturl in sys.argv[1:]:
pid = os.fork()
if pid == 0:
signal.alarm(10)
do_something() # placeholder for the download and print
routine
break
time.sleep(4)

That didn't help me, too.

The following is a PHP application that may help you to understand the bad
behavoir of my application (you will guess my problem if you execute both
the PHP and the Python script):

<?php
echo shell_exec('./script.py http://www.website1.com
http://www.website2.com');
?>
Thank you.
Best regards

Markus Franz
Jul 18 '05 #1
1 3023
Keep track of the processes you create, then wait for them each to exit.
Untested code:

def do(task):
pass

processes = []
try:
for task in tasks:
pid = os.fork()
if pid == 0:
do(task)
os._exit(0)
else:
processes.append(pid)
finally:
while processes:
pid = waitpid(0,0) # Wait for any child, block
# print "reaped", pid
processes.remove(pid)

Instead of looping until 'processes' is empty, you could wait until
waitpid raises os.error with errno==ECHILD ("No child processes").

Jeff

Jul 18 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

192
by: Kwan Ting | last post by:
The_Sage, I see you've gotten yourself a twin asking for program in comp.lang.c++ . http://groups.google.co.uk/groups?hl=en&lr=&ie=UTF-8&oe=UTF-8&th=45cd1b289c71c33c&rnum=1 If you the oh so mighty...
15
by: Fred Zwarts | last post by:
In C++ execution of a program starts already before execution of main(). The initialization of static variables defined outside the scope of main is performed first. I could imagine a program where...
75
by: Beni | last post by:
I have been programming in C for about a year now. It sounds silly, but I never took the time to question why a C(or C++ or Java) program execution begins only at the main(). Is it a convention or...
5
by: Seong-Kook Shin | last post by:
Hi, I'm reading Steve's "C Programming FAQs" in book version, and have two question regarding to Q11.16 ... Also, a `return' from `main' cannot be expected to work if data local to main might be...
13
by: Sokar | last post by:
I have my main function set up as int main(int argv, char *argv) so taht i can read in a variable which is passed to the program on the command line. The problem is that main calls other...
16
by: Geoff Jones | last post by:
Hi What is the closest equivalent to Main in a VB.Net form? Geoff
2
by: psuaudi | last post by:
I have a main query that I would like to call two different subqueries. In MS Access, I usually just save the two subqueries as separate queries which are then called by a third separate and main...
4
by: urkel | last post by:
Hello everybody, I am getting bogged-down by these errors troubling me for the last few days I am writing a C program with a main and several user-defined functions. One of the user-defined...
28
by: ravi | last post by:
Hello everybody, I am writing a small application which does some work before the user main function starts execution. I am trying to #define the main function. But the problem is that,
37
by: nick | last post by:
Does anyone know the file where the main function is defined? thanks
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.