473,695 Members | 2,622 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

How to kill a SocketServer?

Let's say you have a SocketServer with the threading mix-in and you
run serve_forever on it. How can you shut it down, or rather, how can
it even shut itself down? Even if you use a handle_request loop
instead of serve_forever, it still seems difficult:

class myserver(Thread ingMixIn, TCPServer): pass
server = myserver(...)
server.shutdown = False
while not server.shutdown :
server.handle_r equest()

Some code in the request handler eventually sets server.shutdown to
True (say on receiving a "shutdown" command from the client), but by
then the main thread is already blocked waiting for another
connection. I ended up having the quit operation actually open a
loopback connection after setting the shutdown flag, just to unblock
the socket listener. But that seems insane.

Is there a better way?
Jul 19 '05 #1
12 13398
Paul Rubin wrote:
Let's say you have a SocketServer with the threading mix-in and you
run serve_forever on it. How can you shut it down, or rather, how can
it even shut itself down?


I looked at CherryPy's server because I know it uses Python's
BaseHTTPServer which is derived from SocketServer, and I know
it's multithreaded.

Turns out I don't know enough about it to make a reasonable
summary, unless I do a lot more research. Here's some text
from _cphttpserver

class PooledThreadSer ver(SocketServe r.TCPServer):

allow_reuse_add ress = 1

"""A TCP Server using a pool of worker threads. This is superior to the
alternatives provided by the Python standard library, which only offer
(1) handling a single request at a time, (2) handling each request in
a separate thread (via ThreadingMixIn) , or (3) handling each request in
a separate process (via ForkingMixIn). It's also superior in some ways
to the pure async approach used by Twisted because it allows a more
straightforward and simple programming model in the face of blocking
requests (i.e. you don't have to bother with Deferreds)."""

which means it went a different route. Looks like a
request comes in and is put to a work queue. Clients get
from it. There's a special work queue item to tell
the threads to stop.

Andrew
da***@dalkescie ntific.com

Jul 19 '05 #2

Paul> Let's say you have a SocketServer with the threading mix-in and
Paul> you run serve_forever on it. How can you shut it down, or rather,
Paul> how can it even shut itself down? Even if you use a
Paul> handle_request loop instead of serve_forever, it still seems
Paul> difficult:

Paul> class myserver(Thread ingMixIn, TCPServer): pass
Paul> server = myserver(...)
Paul> server.shutdown = False
Paul> while not server.shutdown :
Paul> server.handle_r equest()

Paul> Some code in the request handler eventually sets server.shutdown
Paul> to True (say on receiving a "shutdown" command from the client),
Paul> but by then the main thread is already blocked waiting for another
Paul> connection.

I use precisely that scheme with (I think *) no problem. The only maybe
significant difference I see is that I subclass ThreadingMixin so that it
creates daemon threads:

class DaemonThreadMix In(SocketServer .ThreadingMixIn ):
def process_request _thread(self, request, client_address) :
try:
self.finish_req uest(request, client_address)
self.close_requ est(request)
except:
self.handle_err or(request, client_address)
self.close_requ est(request)

def process_request (self, request, client_address) :
t = threading.Threa d(target = self.process_re quest_thread,
args = (request, client_address) )
t.setDaemon(1)
t.start()

class GenericServer(D aemonThreadMixI n, SocketServer.TC PServer):
...

Skip

(*) Maybe my XML-RPC server actually accepts one more connection after I set
server.shutdown to True before it actually exits. I'll have to take a look
at that.
Jul 19 '05 #3
Skip Montanaro <sk**@pobox.com > writes:
I use precisely that scheme with (I think *) no problem. The only maybe
significant difference I see is that I subclass ThreadingMixin so that it
creates daemon threads: ...
According to the docs, you don't need the subclass, you can just set the
"daemon_threads " property on the server, e.g.

class myserver(Thread ingMixIn, TCPServer): pass
server = myserver(...)
server.daemon_t hreads = True
server.serve_fo rever()

You have to set the flag before you actually start the server, as above.
(*) Maybe my XML-RPC server actually accepts one more connection after I set
server.shutdown to True before it actually exits. I'll have to take a look
at that.


Yes, this is precisely what I'm asking. How do you get the server to
go away without going out of your way to connect to it again? Don't you
notice if it stays around?
Jul 19 '05 #4
I use precisely that scheme with (I think *) no problem. The only
maybe significant difference I see is that I subclass ThreadingMixin
so that it creates daemon threads: ...
Paul> According to the docs, you don't need the subclass, you can just
Paul> set the "daemon_threads " property on the server, e.g.
...

It's possible that my subclassing existed before this feature was available
(it's been in use for about 5 years now) or that I just missed that
mechanism.
(*) Maybe my XML-RPC server actually accepts one more connection
after I set server.shutdown to True before it actually exits. I'll
have to take a look at that.


Paul> Yes, this is precisely what I'm asking. How do you get the server
Paul> to go away without going out of your way to connect to it again?
Paul> Don't you notice if it stays around?

It fields lots of requests, so it's possible that another request arrives
after I exit, but I don't think that's the case. I'll try to take a look in
isolation when I have a moment.

Skip
Jul 19 '05 #5
Andrew Dalke <da***@dalkesci entific.com> writes:
which means it went a different route. Looks like a
request comes in and is put to a work queue. Clients get
from it. There's a special work queue item to tell
the threads to stop.


Well, ok, so the worker threads stop. How do you get the listener
thread to stop, since it's blocked waiting for a new connection to arrive?
Jul 19 '05 #6

Paul> Yes, this is precisely what I'm asking. How do you get the server
Paul> to go away without going out of your way to connect to it again?
Paul> Don't you notice if it stays around?

Skip> It fields lots of requests, so it's possible that another request
Skip> arrives after I exit, but I don't think that's the case. I'll try
Skip> to take a look in isolation when I have a moment.

Here's what I do:

def serve_forever(s elf):
while self.serving:
r,w,e = select.select([self.socket], [], [], self.pause)
if r:
self.handle_req uest()

and set self.pause to something short-ish. The select call times out and
the server exits. Sorry I failed to remember that. It's been several years
since I modified this code.

Skip

Jul 19 '05 #7
After I gave a reference to CherryPy's threaded SocketServer-based
http server code Paul Rubin followed up:
Well, ok, so the worker threads stop. How do you get the listener
thread to stop, since it's blocked waiting for a new connection to arrive?


I don't know the code well enough. You might want to look
through it yourself. I think it works by implementing a
new 'server_activat e()' to implement a timeout
class CherryHTTPServe r(BaseHTTPServe r.HTTPServer):
def server_activate (self):
"""Override server_activate to set timeout on our listener
socket"""
self.socket.set timeout(1)
BaseHTTPServer. HTTPServer.serv er_activate(sel f)
followed by a check for the timeeout

def handle_request( self):
"""Override handle_request to trap timeout exception."""
try:
BaseHTTPServer. HTTPServer.hand le_request(self )
except socket.timeout:
# The only reason for the timeout is so we can notice keyboard
# interrupts on Win32, which don't interrupt accept() by default
return 1
except KeyboardInterru pt:
_cpLogMessage(" <Ctrl-C> hit: shutting down", "HTTP")
self.shutdown()
with the redefined serve_forever() done as

def serve_forever(s elf):
"""Override serve_forever to handle shutdown."""
self.__running = 1
while self.__running:
self.handle_req uest()

The __running attribute is set in the shutdown() method

def shutdown(self):
self.__running = 0
It looks like some special attention is needed when the socket
is in timeout mode

def get_request(sel f):
# With Python 2.3 it seems that an accept socket in timeout (nonblocking
) mode
# results in request sockets that are also set in nonblocking mode. Sin
ce that doesn't play
# well with makefile() (where wfile and rfile are set in SocketServer.p
y) we explicitly set
# the request socket to blocking

request, client_address = self.socket.acc ept()
request.setbloc king(1)
return request, client_address
Hmmm, and I've just copied and pasted the entire implementation of

class CherryHTTPServe r(BaseHTTPServe r.HTTPServer):

given in my CherryPy-2.0.0/cherrypy/_cphttpserver.p y

Andrew
da***@dalkescie ntific.com
Jul 19 '05 #8
Skip Montanaro <sk**@pobox.com > writes:
def serve_forever(s elf):
while self.serving:
r,w,e = select.select([self.socket], [], [], self.pause)
if r:
self.handle_req uest()

and set self.pause to something short-ish. The select call times out and
the server exits.


Ah, good point. Something like this should probably be added to
SocketServer.py (optional timeout parameter to serve_forever), or at
least the trick should be mentioned in the docs.

Thanks.
Jul 19 '05 #9
[Skip Montanaro]
def serve_forever(s elf):
while self.serving:
r,w,e = select.select([self.socket], [], [], self.pause) if r:
self.handle_req uest()

and set self.pause to something short-ish. The select call times out and the server exits.

[Paul Rubin] Ah, good point. Something like this should probably be added to
SocketServer.py (optional timeout parameter to serve_forever), or at
least the trick should be mentioned in the docs.


I like this idea, and perhaps you all could come up with some more
useful APIs in this area; I know that I often struggle a bit with
getting these servers to stop.

(I also like the idea of having the worker thread mixin part of the
standard library -- it seems generally useful.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

Jul 19 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
4164
by: Olivier Hoarau | last post by:
Hello, I have build a client/server application with the socket module. The server mades UDP broadcasting and the client only reads UDP broadcast messages. All work fine. Now I want to use for the same thing the socketserver module, it's ok for the client, but I don't succeed in making work the server :-(( Here is the client (which works)
3
4405
by: Ergin Aytac | last post by:
I'm trying to run a script written in python and have some socket connection problems. I cutted the origin script (more than 1000 lines) so it is only the part of the connection and there is no functionalty but connecting to a server. I have no idea about programming with python so every hint is a present for me :) ------------------------------------ import SocketServer import os
0
2932
by: Adil Hasan | last post by:
Hello Fred, I just ran across your question. I think that the following code will work: ----- SERVER CODE ------ import SocketServer import time class GreetingHandler(SocketServer.BaseRequestHandler): '''Class to handle sending a greeting to a client '''
5
2673
by: missiplicity | last post by:
Hi, I am newbie to Python language and am taking my baby steps. I am using Python2.4 from ActiveState on W2K. I am trying to create a simple SocketServer program. Just adding the following 2 lines of code in the program result in the errors given below: import SocketServer print "SocketServer imported"
3
13909
by: Magnus Lycka | last post by:
I have a socket server like below which I want to exit when it's out of data. If I interrupt the client, I'll get a broken pipe on the server side, and after a Ctrl-C, I can restart the server again, but if I let it run out of data, and exit via handle_error as can be seen below, I will get a socket.error: (98, 'Address already in use') when I try to restart the server. (Unless I wait a minute or so, or use another port.) I feel like I'm...
1
1603
by: rbt | last post by:
I've read more about sockets and now, I have a better understanding of them. However, I still have a few SocketServer module questions: When used with SocketServer how exactly does socket.setdefaulttimeout() work? Does it timeout the initial connect request to the socket server or does it timeout the session between the connecting client socket and the client socket the server generated to handle the incoming request? Also, since the...
0
1717
by: Tomi Hautakoski | last post by:
Hello, I'm a Python newbie trying to figure out how to use SocketServer with IPv6. I would like to set up a TCPServer working like below but how to tell SocketServer I need to use AF_INET6? import SocketServer import logging as l l.basicConfig(level=l.DEBUG,
5
2275
by: eliben | last post by:
Hello, I have a small wxPython application. Today I was trying to add some RPC capability to it, so I implemented an instance of SimpleXMLRPCServer that runs in a separate thread when invoked and answers requests. All went fine until I realized that I have to sometimes stop the server - which is where I ran into a problem. Python threads can not be killed after they've been started. They can be kindly requested to
1
4178
by: Okko Willeboordse | last post by:
All, With Python 2.5 SocketServer features the shutdown method that can be called from another thread to stop the serve_forever loop. However; When the shutdown method is called before serve_forever, shutdown will never return. This can happen when a server is stopped during startup.
0
8642
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9126
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8833
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7673
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5841
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4349
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
4588
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3021
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
1984
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.