471,309 Members | 1,622 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,309 software developers and data experts.

web status display for long running program

I have a command line Python program that sometimes takes a bit
(several minutes) to run. I want to provide an optional method for an
impatient user (me!) to check the status of the program. The type and
amount of status information doesn't fit nicely into a --verbose or
logger -- either too little or too much information at different
points.

I think an optional web page would be convenient interface. The
Python program would listen on some port, and if queried (by me
browsing to localhost:12345 for example) would return a pretty status
display. Hitting reload would update the status etc.

My problem is that I'm not sure how to do this:
- I don't want to embed a full web server into the application or
require any special PC setup.
- I think I know how to listen on a socket, but not sure how to send
stuff to to a web browser -- just start with <HTML>? Or like a CGI
script with the header stuff like text/html? (I don't care if I have
to write the HTML by hand or can use a toolkit -- not important).
- Do I need a separate thread to listen and send the HTML? The
application is currently single threaded. I'm confortable with
threads, but would prefer to avoid them if possible.

Or is there a better/different way of doing this? Any general advice
or pointers to some code that already does this would be very much
appreciated.

Python 2.3, under both Linux & Windows if that makes a difference.

Thanks,
Brian.
Jul 18 '05 #1
9 2004
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Brian Roberts wrote:
| I have a command line Python program that sometimes takes a bit
| (several minutes) to run. I want to provide an optional method for an
| impatient user (me!) to check the status of the program. The type and
| amount of status information doesn't fit nicely into a --verbose or
| logger -- either too little or too much information at different
| points.

dd and pppd (and probably others) solve this problem by registering
a signal handler that either toggles debug or prints status
information to stderr.

- --
John Lenton (jo**@except.com.ar) -- Random fortune:
Work consists of whatever a body is obliged to do.
Play consists of whatever a body is not obliged to do.
-- Mark Twain
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.0 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFCHpMZcxEeCh6qRm8RAg11AKCek0hV6QCHw6fm3TM3KA JIXPb1RQCg4qhy
HMr5y+w7fxESkJ2vQ0GEmm0=
=c4bs
-----END PGP SIGNATURE-----
Jul 18 '05 #2
Brian Roberts wrote:
I have a command line Python program ...
I think an optional web page would be convenient interface. The
Python program would listen on some port, and if queried (by me
browsing to localhost:12345 for example) would return a pretty status
display. Hitting reload would update the status etc.

My problem is that I'm not sure how to do this:
- I don't want to embed a full web server into the application or
require any special PC setup.
- I think I know how to listen on a socket, but not sure how to send
stuff to to a web browser -- just start with <HTML>?
See below. I don't care if you use the code, it was fun to
whip up on the spur of the moment. I'm quite certain that
if you try searching for "smallest web server" or something you'll
find a bunch of Perl guys who've competed to produce something
that does the same in a tenth the space...
- Do I need a separate thread to listen and send the HTML? The
application is currently single threaded. I'm confortable with
threads, but would prefer to avoid them if possible.


If you don't want a thread, you probably don't need one, but
this might be a little cleaner to put in a thread. I'm not
sure exactly what approach would be best to avoid a thread,
but using select() on the server socket, with a timeout of 0
to poll, instead of blocking as the following code does, would
probably be the simplest.

Try this out and maybe you can morph it into something like
what you want...

'''miniweb: microscopic web server'''

import socket
from select import select
from datetime import datetime

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('', 80))
s.listen(5)
while True:
cs = s.accept()[0]
try:
header = ''
while True:
r, w, e = select([cs], [], [cs], 2)
if not r or e:
break

header += cs.recv(1024)
if '\r\n\r\n' in header:
break

msg = ('<html><head><title>Miniweb!</title></head>'
'<body><h3>Clock</h3><p>The time is %s</p></body></html>' %
datetime.now())

headers = '\r\n'.join([
'HTTP/1.0 200 OK',
'Server: Miniweb-0.1',
'Content-Type: text/html',
'Connection: close',
'Content-Length: %s' % len(msg),
])

cs.send(headers + '\r\n\r\n' + msg + '\n')

finally:
cs.close()
I make no claims that this will survive any particular kind
of use or abuse. I did test with telnet, Firefox, Opera, and Explorer,
however. A couple of key points found while testing:

1. Explorer needs you to read the header in its entirety, so I
had to add the part that actually scans the header for the
terminating sequence (\r\n\r\n).

2. There's a two-second timeout on reading the header,
in case something connects but doesn't send anything... not
really tested.

3. Some or most of the headers in the reply may not be
required. Perhaps the first line and the Content-Length
(or not even that?) are all that are required. Play if you want.
No doubt others will pick this apart and we'll all learn
something. ;-)

4. MIT license ;-) as in don't complain to me if this
doesn't do what you need, but use it any way you wish...

-Peter
Jul 18 '05 #3
br***@mirror.org (Brian Roberts) writes:
- I don't want to embed a full web server into the application or
require any special PC setup.
That's not a big deal, just use the standard library's http server class.
- I think I know how to listen on a socket, but not sure how to send
stuff to to a web browser -- just start with <HTML>? Or like a CGI
script with the header stuff like text/html? (I don't care if I have
to write the HTML by hand or can use a toolkit -- not important).
You'd have to send a full http response, but the http server class
does some of it for you.
- Do I need a separate thread to listen and send the HTML? The
application is currently single threaded. I'm confortable with
threads, but would prefer to avoid them if possible.


Separate thread is the obvious way to do it.
Jul 18 '05 #4
Cute! Thanks for posting that. I too like the 'web interface' concept,
it has made SmoothWall a pleasure to use, even on older machines.

I was inspired to enhance your code, and perform a critical bug-fix.
Your code would not have sent large files out to dialup users, because
it assumed all data was sent on the 'send' command. I added code to
check for the number of bytes sent, and loop until it's all gone. I
also turned it into a class, and made it 'command' based.

Have fun with it, Brian!

'''
miniweb: microscopic web server by Peter Hansen

This is a simple web server that handles one request at
a time. Therefore, it's useful mainly for administrative
tasks that have one person connected at a time.

It doesn't actually serve pages, it executes commands
based on the web page you request. The commands it
currently includes are:

time - Send the time
header - Echo back the original header
largefile - Test sending a large file back to the person
quit - Shut down the web server

To add more commands, add new functions prefixed with
'request_' similar to the sample functions below.
As you add functions, the menu choices displayed to
the user are automatically updated.

I fixed the 'send' feature so it could send large files.
Before, it would send the first X bytes, then stop
without checking to make sure all bytes were went.
I also made it a class, created the 'command based'
feature, and added the auto-updating menu choices.

Have fun with it!

--Kamilche

'''

import socket
import select
import datetime

_HEADER = '\r\n'.join([
'HTTP/1.0 %s',
'Server: Miniweb-0.1',
'Content-Type: text/html',
'Connection: close',
'Content-Length: %d',
'',
'',
])

_BODY = '\r\n'.join([
'<html><head><title>%s</title></head>',
'<body><h3>%s</h3>\r\n%s\r\n</body></html>',
])

_QUIT = False
class WebServer(object):

def __init__(self, port):
' Start listening on the specified port'
self.socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.socket.bind(('', port))
self.socket.listen(5)

' Build the list of menu choices'
self.menu = '<pre>\n'
for key, value in self.__class__.__dict__.items():
if key[:8] == 'request_':
self.menu += "%-10s- %s\r\n" % (key[8:], value.__doc__)
self.menu += '</pre>\n'

def Timer(self):
' Process new requests'
r, w, e = select.select([self.socket], [], [], 0.0)
if not r:
return
cs = self.socket.accept()[0]
header = ''
while not '\r\n\r\n' in header:
r, w, e = select.select([cs], [], [cs], 2.0)
if not r or e:
break
header += cs.recv(1024)
status = '200 OK'
title = "Miniweb!"
if not '\r\n\r\n' in header:
status = '408 Request Timeout'
body = ['Your request was not received in time.']
else:
lines = header.split('\r\n')
words = lines[0].split(' ')
request = ' '.join(words[1:-1])
request = request[1:]
if request == '':
fn = self.default
else:
fn = getattr(self, 'request_' + request, None)
if not fn:
status = '404 Not Found'
fn = self.notfound
body = fn(header, request)

body = _BODY % (title, title,
''.join([str(arg) for arg in body]))
header = _HEADER % (status, len(body))
data = header + body
while len(data) > 0:
cc = cs.send(data)
data = data[cc:]
cs.close()

def default(self, header, request):
' Print the available choices'
return ('Welcome to Miniweb!',
'Available commands are:<p>',
self.menu)

def notfound(self, header, request):
' Handle unknown requests'
return ('Unknown request <pre>', request, '</pre>',
'Your header was:<br><pre>', header, '</pre>',
'Available commands are:<p>', self.menu)

def request_time(self, header, request):
' Send the time'
return ('The time is ', datetime.datetime.now(), '<p>')

def request_header(self, header, request):
' Echo back the original header'
return ('Your header was:<br><pre>', header, '</pre>')

def request_largefile(self, header, request):
' Test sending a large file back to the person'
temp = ['<pre>\r\n']
for i in range(10000):
temp.append("This is line %d of the result.\r\n" % i)
temp.append('</pre>')
return temp

def request_quit(self, header, request):
' Shut down the web server'
global _QUIT
_QUIT = True
return ('Web server shut down at ', datetime.datetime.now())
def main():
' Main loop'
server = WebServer(80)
while not _QUIT:
server.Timer()
print "Done!"

main()

Jul 18 '05 #5
"Kamilche" <kl*******@comcast.net> writes:
I was inspired to enhance your code, and perform a critical bug-fix.
Your code would not have sent large files out to dialup users, because
it assumed all data was sent on the 'send' command. I added code to
check for the number of bytes sent, and loop until it's all gone. I
also turned it into a class, and made it 'command' based.


That server seems to do a cpu-busy loop polling the listening socket
for incoming requests! Ouch! It also assumes that once it's gotten
\r\n in the input, that the entire request has been received. If
someone is telnetting into the server typing one character at a time,
that could fool it.

I think it's better to just launch a separate thread and use
BaseHTTPServer which is already in the stdlib.
Jul 18 '05 #6
> I was inspired to enhance your code, and perform a critical bug-fix.
Your code would not have sent large files out to dialup users, because
it assumed all data was sent on the 'send' command. I added code to
check for the number of bytes sent, and loop until it's all gone.


Another solution is to simply use the socket.sendall method, which
will continue to send all data until all the bytes have been sent.

Sw.
Jul 18 '05 #7
This is fun, so I will give my solution too (of course,
the effort here is to give the shortest solution, not the
more robust solution ;).

This is the server program, which just counts forever:

from threading import Thread
from CGIHTTPServer import test
import os

class Counter(Thread):
def run(self):
n = 0
self.loop = True
while self.loop:
n += 1
os.environ["COUNTER"] = str(n)
def stop(self):
self.loop = False
if __name__ == "__main__":
counter = Counter()
counter.start()
try:
test()
finally: # for instance, if CTRL-C is called
counter.stop()

And this is the CGI viewer:

#!/usr/bin/python
import os
print "Content-type: text/plain\n"
print "Counter: %s" % os.environ["COUNTER"]

Pretty bare-bone ;)

Michele Simionato

Jul 18 '05 #8
Not exactly on point, but this is what I use in many of my
programs to show progress on long running console apps.

Larry Bates
class progressbarClass:
def __init__(self, finalcount, progresschar=None):
import sys
self.finalcount=finalcount
self.blockcount=0
#
# See if caller passed me a character to use on the
# progress bar (like "*"). If not use the block
# character that makes it look like a real progress
# bar.
#
if not progresschar: self.block=chr(178)
else: self.block=progresschar
#
# Get pointer to sys.stdout so I can use the write/flush
# methods to display the progress bar.
#
self.f=sys.stdout
#
# If the final count is zero, don't start the progress gauge
#
if not self.finalcount : return
self.f.write('\n------------------ % Progress -------------------1\n')
self.f.write(' 1 2 3 4 5 6 7 8 9 0\n')
self.f.write('----0----0----0----0----0----0----0----0----0----0\n')
return

def progress(self, count):
#
# Make sure I don't try to go off the end (e.g. >100%)
#
count=min(count, self.finalcount)
#
# If finalcount is zero, I'm done
#
if self.finalcount:
percentcomplete=int(round(100*count/self.finalcount))
if percentcomplete < 1: percentcomplete=1
else:
percentcomplete=100

#print "percentcomplete=",percentcomplete
blockcount=int(percentcomplete/2)
#print "blockcount=",blockcount
if blockcount > self.blockcount:
for i in range(self.blockcount,blockcount):
self.f.write(self.block)
self.f.flush()

if percentcomplete == 100: self.f.write("\n")
self.blockcount=blockcount
return

if __name__ == "__main__":
from time import sleep
pb=progressbarClass(8,"*")
count=0
while count<9:
count+=1
pb.progress(count)
sleep(0.2)

pb=progressbarClass(100)
pb.progress(20)
sleep(0.2)
pb.progress(47)
sleep(0.2)
pb.progress(90)
sleep(0.2)
pb.progress(100)
print "testing 1:"
pb=progressbarClass(1)
pb.progress(1)

Brian Roberts wrote:
I have a command line Python program that sometimes takes a bit
(several minutes) to run. I want to provide an optional method for an
impatient user (me!) to check the status of the program. The type and
amount of status information doesn't fit nicely into a --verbose or
logger -- either too little or too much information at different
points.

I think an optional web page would be convenient interface. The
Python program would listen on some port, and if queried (by me
browsing to localhost:12345 for example) would return a pretty status
display. Hitting reload would update the status etc.

My problem is that I'm not sure how to do this:
- I don't want to embed a full web server into the application or
require any special PC setup.
- I think I know how to listen on a socket, but not sure how to send
stuff to to a web browser -- just start with <HTML>? Or like a CGI
script with the header stuff like text/html? (I don't care if I have
to write the HTML by hand or can use a toolkit -- not important).
- Do I need a separate thread to listen and send the HTML? The
application is currently single threaded. I'm confortable with
threads, but would prefer to avoid them if possible.

Or is there a better/different way of doing this? Any general advice
or pointers to some code that already does this would be very much
appreciated.

Python 2.3, under both Linux & Windows if that makes a difference.

Thanks,
Brian.

Jul 18 '05 #9
Paul Rubin wrote:
"Kamilche" <kl*******@comcast.net> writes:
I was inspired to enhance your code, and perform a critical bug-fix.
Your code would not have sent large files out to dialup users, because
it assumed all data was sent on the 'send' command. I added code to
check for the number of bytes sent, and loop until it's all gone. I
also turned it into a class, and made it 'command' based.
That server seems to do a cpu-busy loop polling the listening socket
for incoming requests! Ouch!


Not likely, as the socket is blocking by default. It won't
use any CPU at all until a connection arrives.
It also assumes that once it's gotten
\r\n in the input, that the entire request has been received.
Which is perfectly correct given the OP's specifications,
which were that he simply wanted to hit "Refresh" in his
browser to get a new page. Note also it's \r\n\r\n, not
just a single one.
If someone is telnetting into the server typing one character at a time,
that could fool it.
Fool it how? We're not talking actual data transfer, just
a simple HTTP request which ends its header section with
the aforementioned sequence, guaranteed.
I think it's better to just launch a separate thread and use
BaseHTTPServer which is already in the stdlib.


Very likely.... of course, that wouldn't be any fun. ;-)

-Peter
Jul 18 '05 #10

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

27 posts views Thread by John Roth | last post: by
8 posts views Thread by mark4asp | last post: by
5 posts views Thread by Victor | last post: by
1 post views Thread by Anonymous | last post: by
5 posts views Thread by Laetitia | last post: by
reply views Thread by rosydwin | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.