How to kill a SocketServer?

[Paul Rubin]

Let's say you have a SocketServer with the threading mix-in and you
run serve_forever on it. How can you shut it down, or rather, how can
it even shut itself down? Even if you use a handle_request loop
instead of serve_forever, it still seems difficult:

class myserver(ThreadingMixIn, TCPServer): pass
server = myserver(...)
server.shutdown = False
while not server.shutdown:
server.handle_request()

Some code in the request handler eventually sets server.shutdown to
True (say on receiving a "shutdown" command from the client), but by
then the main thread is already blocked waiting for another
connection. I ended up having the quit operation actually open a
loopback connection after setting the shutdown flag, just to unblock
the socket listener. But that seems insane.

Is there a better way?

[Andrew Dalke]

Paul Rubin wrote:

Let's say you have a SocketServer with the threading mix-in and you
run serve_forever on it. How can you shut it down, or rather, how can
it even shut itself down?

I looked at CherryPy's server because I know it uses Python's
BaseHTTPServer which is derived from SocketServer, and I know
it's multithreaded.

Turns out I don't know enough about it to make a reasonable
summary, unless I do a lot more research. Here's some text
from _cphttpserver

class PooledThreadServer(SocketServer.TCPServer):

allow_reuse_address = 1

"""A TCP Server using a pool of worker threads. This is superior to the
alternatives provided by the Python standard library, which only offer
(1) handling a single request at a time, (2) handling each request in
a separate thread (via ThreadingMixIn), or (3) handling each request in
a separate process (via ForkingMixIn). It's also superior in some ways
to the pure async approach used by Twisted because it allows a more
straightforward and simple programming model in the face of blocking
requests (i.e. you don't have to bother with Deferreds)."""

which means it went a different route. Looks like a
request comes in and is put to a work queue. Clients get
from it. There's a special work queue item to tell
the threads to stop.

Andrew
da***@dalkescientific.com

[Skip Montanaro]

Paul> Let's say you have a SocketServer with the threading mix-in and
Paul> you run serve_forever on it. How can you shut it down, or rather,
Paul> how can it even shut itself down? Even if you use a
Paul> handle_request loop instead of serve_forever, it still seems
Paul> difficult:

Paul> class myserver(ThreadingMixIn, TCPServer): pass
Paul> server = myserver(...)
Paul> server.shutdown = False
Paul> while not server.shutdown:
Paul> server.handle_request()

Paul> Some code in the request handler eventually sets server.shutdown
Paul> to True (say on receiving a "shutdown" command from the client),
Paul> but by then the main thread is already blocked waiting for another
Paul> connection.

I use precisely that scheme with (I think *) no problem. The only maybe
significant difference I see is that I subclass ThreadingMixin so that it
creates daemon threads:

class DaemonThreadMixIn(SocketServer.ThreadingMixIn):
def process_request_thread(self, request, client_address):
try:
self.finish_request(request, client_address)
self.close_request(request)
except:
self.handle_error(request, client_address)
self.close_request(request)

def process_request(self, request, client_address):
t = threading.Thread(target = self.process_request_thread,
args = (request, client_address))
t.setDaemon(1)
t.start()

class GenericServer(DaemonThreadMixIn, SocketServer.TCPServer):
...

Skip

(*) Maybe my XML-RPC server actually accepts one more connection after I set
server.shutdown to True before it actually exits. I'll have to take a look
at that.

[Paul Rubin]

Skip Montanaro <sk**@pobox.com> writes:

I use precisely that scheme with (I think *) no problem. The only maybe
significant difference I see is that I subclass ThreadingMixin so that it
creates daemon threads: ...
According to the docs, you don't need the subclass, you can just set the
"daemon_threads" property on the server, e.g.

class myserver(ThreadingMixIn, TCPServer): pass
server = myserver(...)
server.daemon_threads = True
server.serve_forever()

You have to set the flag before you actually start the server, as above.
(*) Maybe my XML-RPC server actually accepts one more connection after I set
server.shutdown to True before it actually exits. I'll have to take a look
at that.

Yes, this is precisely what I'm asking. How do you get the server to
go away without going out of your way to connect to it again? Don't you
notice if it stays around?

[Skip Montanaro]

I use precisely that scheme with (I think *) no problem. The only
maybe significant difference I see is that I subclass ThreadingMixin
so that it creates daemon threads: ...
Paul> According to the docs, you don't need the subclass, you can just
Paul> set the "daemon_threads" property on the server, e.g.
...

It's possible that my subclassing existed before this feature was available
(it's been in use for about 5 years now) or that I just missed that
mechanism.
(*) Maybe my XML-RPC server actually accepts one more connection
after I set server.shutdown to True before it actually exits. I'll
have to take a look at that.

Paul> Yes, this is precisely what I'm asking. How do you get the server
Paul> to go away without going out of your way to connect to it again?
Paul> Don't you notice if it stays around?

It fields lots of requests, so it's possible that another request arrives
after I exit, but I don't think that's the case. I'll try to take a look in
isolation when I have a moment.

Skip

[Paul Rubin]

Andrew Dalke <da***@dalkescientific.com> writes:

which means it went a different route. Looks like a
request comes in and is put to a work queue. Clients get
from it. There's a special work queue item to tell
the threads to stop.

Well, ok, so the worker threads stop. How do you get the listener
thread to stop, since it's blocked waiting for a new connection to arrive?

[Skip Montanaro]

Paul> Yes, this is precisely what I'm asking. How do you get the server
Paul> to go away without going out of your way to connect to it again?
Paul> Don't you notice if it stays around?

Skip> It fields lots of requests, so it's possible that another request
Skip> arrives after I exit, but I don't think that's the case. I'll try
Skip> to take a look in isolation when I have a moment.

Here's what I do:

def serve_forever(self):
while self.serving:
r,w,e = select.select([self.socket], [], [], self.pause)
if r:
self.handle_request()

and set self.pause to something short-ish. The select call times out and
the server exits. Sorry I failed to remember that. It's been several years
since I modified this code.

Skip

[Andrew Dalke]

After I gave a reference to CherryPy's threaded SocketServer-based
http server code Paul Rubin followed up:

Well, ok, so the worker threads stop. How do you get the listener
thread to stop, since it's blocked waiting for a new connection to arrive?

I don't know the code well enough. You might want to look
through it yourself. I think it works by implementing a
new 'server_activate()' to implement a timeout
class CherryHTTPServer(BaseHTTPServer.HTTPServer):
def server_activate(self):
"""Override server_activate to set timeout on our listener
socket"""
self.socket.settimeout(1)
BaseHTTPServer.HTTPServer.server_activate(self)
followed by a check for the timeeout

def handle_request(self):
"""Override handle_request to trap timeout exception."""
try:
BaseHTTPServer.HTTPServer.handle_request(self)
except socket.timeout:
# The only reason for the timeout is so we can notice keyboard
# interrupts on Win32, which don't interrupt accept() by default
return 1
except KeyboardInterrupt:
_cpLogMessage("<Ctrl-C> hit: shutting down", "HTTP")
self.shutdown()
with the redefined serve_forever() done as

def serve_forever(self):
"""Override serve_forever to handle shutdown."""
self.__running = 1
while self.__running:
self.handle_request()

The __running attribute is set in the shutdown() method

def shutdown(self):
self.__running = 0
It looks like some special attention is needed when the socket
is in timeout mode

def get_request(self):
# With Python 2.3 it seems that an accept socket in timeout (nonblocking
) mode
# results in request sockets that are also set in nonblocking mode. Sin
ce that doesn't play
# well with makefile() (where wfile and rfile are set in SocketServer.p
y) we explicitly set
# the request socket to blocking

request, client_address = self.socket.accept()
request.setblocking(1)
return request, client_address
Hmmm, and I've just copied and pasted the entire implementation of

class CherryHTTPServer(BaseHTTPServer.HTTPServer):

given in my CherryPy-2.0.0/cherrypy/_cphttpserver.py

Andrew
da***@dalkescientific.com

[Paul Rubin]

Skip Montanaro <sk**@pobox.com> writes:

def serve_forever(self):
while self.serving:
r,w,e = select.select([self.socket], [], [], self.pause)
if r:
self.handle_request()

and set self.pause to something short-ish. The select call times out and
the server exits.

Ah, good point. Something like this should probably be added to
SocketServer.py (optional timeout parameter to serve_forever), or at
least the trick should be mentioned in the docs.

Thanks.

[guido]

[Skip Montanaro]

def serve_forever(self):
while self.serving:
r,w,e = select.select([self.socket], [], [], self.pause) if r:
self.handle_request()

and set self.pause to something short-ish. The select call times out and the server exits.

[Paul Rubin] Ah, good point. Something like this should probably be added to
SocketServer.py (optional timeout parameter to serve_forever), or at
least the trick should be mentioned in the docs.

I like this idea, and perhaps you all could come up with some more
useful APIs in this area; I know that I often struggle a bit with
getting these servers to stop.

(I also like the idea of having the worker thread mixin part of the
standard library -- it seems generally useful.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

[Paul Rubin]

"gu***@python.org" <gv********@gmail.com> writes:

Ah, good point. Something like this should probably be added to
SocketServer.py (optional timeout parameter to serve_forever), or at
least the trick should be mentioned in the docs.

I like this idea, and perhaps you all could come up with some more
useful APIs in this area; I know that I often struggle a bit with
getting these servers to stop.

I put an RFE in sourceforge last night, suggesting adding a shutdown()
method to socket servers.

[Skip Montanaro]

>> > set self.pause to something short-ish. The select call times out

> and the server exits.

Guido> [Paul Rubin] Ah, good point. Something like this should probably be added to
SocketServer.py (optional timeout parameter to serve_forever), or at
least the trick should be mentioned in the docs.

Guido> I like this idea, and perhaps you all could come up with some
Guido> more useful APIs in this area...

I stumbled on a somewhat cleaner way to do this using socket timeouts:

class Server(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
pause = 0.25
allow_reuse_address = True

def __init__(self, server_address, RequestHandlerClass):
SocketServer.TCPServer.__init__(self, server_address,
RequestHandlerClass)
self.socket.settimeout(self.pause)
self.serving = 1
...

def serve_forever(self):
while self.serving:
self.handle_request()

As far as my simple testing went (a trivial little xmlrpc server exposing
only "noop" and "exit" methods), the above worked fine. I was mildly
surprised that I didn't have to catch socket.timeout exceptions.

Skip

[Guido van Rossum]

[Skip Montanaro]

I stumbled on a somewhat cleaner way to do this using socket timeouts:

class Server(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
pause = 0.25
allow_reuse_address = True

def __init__(self, server_address, RequestHandlerClass):
SocketServer.TCPServer.__init__(self, server_address,
RequestHandlerClass)
self.socket.settimeout(self.pause)
self.serving = 1
...

def serve_forever(self):
while self.serving:
self.handle_request()

As far as my simple testing went (a trivial little xmlrpc server exposing
only "noop" and "exit" methods), the above worked fine. I was mildly
surprised that I didn't have to catch socket.timeout exceptions.

I think this would cause timeouts in the middle of handling request
whenever a client is slow.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)