473,320 Members | 1,732 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

os.system(), HTTPServer, and finishing HTTP requests

Hi,

I am trying to spawn a daemon type program to go off on its own and do
some work (asynchoronously) from within an HTTPServer, but I am running into
a problem where the web browser (actually a Perl LWP program) still seems to
be waiting for more input until the child that is forked in the server
finishes (granchild, actually). I don't understand this. I need to be able
to have the POST to the server complete and go on even though there is a
(grand)child process still running.

So... that's the short summary, more specific code follows..

worker.py is a program that does the standard double-fork paradigm,
allowing the original parent to return and leaving an orphaned child behind
to do its own thing. If you call this program directly through the shell, it
returns you immediately to your shell, which is just like what I want and
would expect:
#! /usr/bin/python
"""
worker.py
"""

import sys, os, time

t = 30

pid = os.fork()
if (pid == 0):
# child: fork() again and exit so that the original
# parent is not left waiting on us.
pid = os.fork()
if (pid > 0):
# child
print "child (PID %d): forked grandchild %d. I'm exiting." %\
(os.getpid(), pid)
sys.exit()

# grandchild continues here, drops out bottom of if-block

elif (pid > 0):
# parent: exit so caller can continue
p_pid = os.getpid()
print "parent (PID %d): forked child %d. ." % (p_pid, pid)
(c_pid, status) = os.wait()
print "parent (PID %d): child %d is done. I'm exiting." % (p_pid, c_pid)
sys.exit() # normal exit
# grandchild continues here - my worker process
g_pid = os.getpid()
s = "grandchild (PID %d): going to sleep for %d seconds..." % (g_pid, t)
print s

# pretend to do some work
time.sleep(t)

# done
print "grandchild (PID %d) waking up... exiting." % g_pid

Here is sample output from running worker.py:
ej@sand:~/src/python/problem> worker.py
grandchild (PID 9697): going to sleep for 30 seconds...
child (PID 9696): forked grandchild 9697. I'm exiting.
parent (PID 9695): forked child 9696. .
parent (PID 9695): child 9696 is done. I'm exiting.
ej@sand:~/src/python/problem>

and then about 30 sec later, after getting back to my shell, my console
prints:

grandchild (PID 9697) waking up... exiting.
This is all fine and dandy. One step removed from that is to call this
program from another Python program via os.system(). It too exits
immediately and returns me to the shell even though my (grand)child process
is still running. Again, this is just what I want and expect. Here is the
code for that program:

#! /usr/bin/python
"""
call_worker.py
"""

import os, sys

PROG_NAME = sys.argv[0]

print PROG_NAME + ": calling worker.py..."
os.system('worker.py')
print PROG_NAME + ": all done. exiting."
Here is the output from running it (again, all fine and dandy):

ej@sand:~/src/python/problem> call_worker.py
../call_worker.py: calling worker.py...
grandchild (PID 9710): going to sleep for 30 seconds...
child (PID 9709): forked grandchild 9710. I'm exiting.
parent (PID 9708): forked child 9709. .
parent (PID 9708): child 9709 is done. I'm exiting.
../call_worker.py: all done. exiting.
ej@sand:~/src/python/problem>

and after 30 seconds, your again-active shell window should print:

grandchild (PID 9710) waking up... exiting.
But if I make this same os.system() call from within my own HTTPServer,
then the browser that is making the request is left hanging until that
sleeping grandchild is done. It's like there is still a socket connection
between it and my browser?!? But the grandchild is not forked from the
server - it's an os.system() call! It should not be inherting any file
descriptors and such, right? Even though the do_GET() function should be
over, and in fact, you can see in the shell that started the server that the
server process has already exited, the browser is apparently still waiting
for the web page to finish loading.

I don't get it! I have a long task I need to instantiate in my Python
server based on info in the HTTP POST, and the program that is making that
POST needs to be able to go on and do other stuff before my task is over.
Why am I not getting a clean exit in my POSTing process? What do I need to
do to get my HTTP request to my server to finish essentially "immediately"?
Thanks for taking the time to read my post. Any help greatly appreciated!
:) -ej
Below is the code for a simple server you can run. It is currently coded
to listen on port 5238 and only serve one request before the server exits.
It serves a little static chunk of HTML on a GET request regardless of what
PATH you call (I talked about POST above, but the browser-hanging behaviour
is the same).
#! /usr/bin/python

"""
server.py
"""

from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler
import os, sys

SERVER_PORT = 5238 # arbitrary port not in use (open on firewall?)
WORKER = 'worker.py' # name of the program to call via os.system()
PROG = sys.argv[0] # name of this program

#================================================= ======================
class RequestHandler(BaseHTTPRequestHandler):

#-------------------------------------------------------------------
def do_GET(self):

# call my "daemon" to get some stuff done in the background
print "%s: about to call os.system(%s)" % (PROG, WORKER)
os.system(WORKER)
print "%s: back from os.system(%s)" % (PROG, WORKER)

# send HTTP headers
self.send_response(200)
self.send_header("Content-type", 'text/html');
self.end_headers()

# start HTML output
html = """\
<html>
<body><h2>All done!</h2></body>
</html>
"""
self.wfile.write(html)
#-------------------------------------------------------------------

#================================================= ======================

# BEGIN main
if (__name__ == '__main__'):

# create a new HTTP server and handle a request
server = HTTPServer(('', SERVER_PORT), RequestHandler)
server.handle_request()

Jul 18 '05 #1
1 2673
In article <41********@nntp.zianet.com>,
"Erik Johnson" <ej <at> wellkeeper <dot> com> wrote:
....
I'm not sure why those would all be open, but this little bit seems to
resolve the problem
(it throws OSError when it hits the first invalid file descriptor).

for fd in xrange(3, 256):
try:
os.close(fd)
except OSError:
break

If you know a better/smarter way to effect the same thing, I'd be glad to
hear about it.


That's a variation on a common idiom. The difference is
the break, where more commonly you'd write "pass". If this
is happening in the context of a short-term programming effort
that has to be very efficient, it's a good idea, because the
exceptions make the close much more expensive than it would
have been in C, where we usually see this loop. If it needs
to work for the indefinite future, I would say change the break
to pass. You can't count on that continuous series of open
file descriptors.

Donn Cave, do**@u.washington.edu
Jul 18 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: WmGill | last post by:
I'm new to Python, and am dabbling with it to replace a lot of scattered scripts & programs. One thing I want to try is an HTTPserver like in the pydocgui module. This way I can use html to...
0
by: Will Stuyvesant | last post by:
I have a CGI server written in Python, useful for offline CGI testing. One of my CGI scripts ends with: print 'Location: http://myresourcelocation' print But this does not work with the...
1
by: aswinee | last post by:
I am running Microsoft SQL Server 2000 - 8.00.760 Enterprise Edition on Windows 2003 Enterprise Edition (NT 5.2 Build 3790:) I have 4CPU and 8GB of RAM. I have AWE enabled, /pae /3gb switch is on...
0
by: Chris Travers | last post by:
Hi All; I may be able to do this in Perl, but if there is enough interest in doing something like this in C, maybe I can still help (however, consider your self warned about my skill at coding...
2
by: Michael Per | last post by:
Does anybody know of a best way to limit system resources (CPU/memory) for each particular request? In my application based on user's parameters a request may take considerable amount of time and...
0
by: crowell | last post by:
Hello, I am having trouble getting the ThreadingMixIn to do what I want. Looking over the docs (and some old code I wrote which successfully does what I want), I arrived at the following: ...
20
by: djc | last post by:
I get this *intermittently* on a utility I am working on. I don't know whats going on but here are a few points about it: - using VS 2005, running on xp sp2 - program uses multiple threadpool...
18
by: troywalker | last post by:
I am new to LDAP and Directory Services, and I have a project that requires me to authenticate users against a Sun Java System Directory Server in order to access the application. I have found...
2
by: =?Utf-8?B?cmVk?= | last post by:
Hi Friends, We recently deployed our application to production and I am experiencing the below error message. Cannot access a disposed object named "System.Net.TlsStream" The error occurs...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.