473,396 Members | 1,784 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

socket function that loops AND returns something

I have a function that starts a socket server looping continuously
listening for connections. It works. Clients can connect to it and send
data.

The problem is this: I want to get the data that the clients send out of
the loop but at the same time keep the loop going so it can continue
listening for connections. If I return, I exit the function and the loop
stops. How might I handle this?

def listen(ip_param, port_param):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
while 1:
print "\nWaiting for new connection...\n"
s.listen(1)
conn, addr = s.accept()
print "Client", addr[0], "connected and was directed to port",
addr[1]
data = conn.recv(1024)
# I want to get 'data' out of the loop, but keep the loop going
print "Client sent this message:", data
conn.close()
Jul 18 '05 #1
8 1587
Quoth Brad Tilley <br********@usa.net>:
| I have a function that starts a socket server looping continuously
| listening for connections. It works. Clients can connect to it and send
| data.
|
| The problem is this: I want to get the data that the clients send out of
| the loop but at the same time keep the loop going so it can continue
| listening for connections. If I return, I exit the function and the loop
| stops. How might I handle this?
|
| def listen(ip_param, port_param):
| s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
| while 1:
| print "\nWaiting for new connection...\n"
| s.listen(1)
| conn, addr = s.accept()
| print "Client", addr[0], "connected and was directed to port", addr[1]
| data = conn.recv(1024)
| # I want to get 'data' out of the loop, but keep the loop going
| print "Client sent this message:", data
| conn.close()

Well, some people would address this problem by spawning a thread
(and then they'd have two problems! as the saying goes.)

It depends on what you want. Your main options:

- store whatever state you need to resume, and come back
and accept a new connection when you're done with the data -
class SocketConnection:
def __init__(self, ip_param, port_param):
...
def recv(self):
... accept
... recv
... close
return data
- turn it upside down, call the processing function
def listen(ip_param, port_param, data_function):
...
data = conn.recv(1024)
data = data_function(data)
if data:
conn.send(data)
conn.close()
...
- fork a new process (UNIX oriented.)
...
pid = os.fork()
if pid:
wait_for_these_pids_eventually.append(pid)
else:
try:
conn, addr = s.accept()
s.close()
conn_function(conn)
finally:
os._exit(0)
- spawn a new thread (similar idea)

Processes and threads will allow your service to remain responsive
while performing tasks that take a long time, because these tasks
will run concurrently. They do bring their own issues, though, and
it isn't worth it unless it's worth it, which only you can tell.
It's a pretty lengthy computation that takes so much time that a
thread or process is really called for.

If the inverted, functional approach suits your application, note
that you can use a Python class instance here to keep state between
connections - like,
class ConnectionReceiver:
...
def data_function(self, data):
self.count = self.count + 1
print >> sys.stderr, 'datum', self.count
...
...
receiver = ConnectionReceiver()
listen(ip_param, port_param, receiver.data_function)

Donn Cave, do**@drizzle.com
Jul 18 '05 #2
Brad Tilley wrote:
I have a function that starts a socket server looping continuously
listening for connections. It works. Clients can connect to it and send
data.

The problem is this: I want to get the data that the clients send out of
the loop but at the same time keep the loop going so it can continue
listening for connections. If I return, I exit the function and the loop
stops. How might I handle this?

def listen(ip_param, port_param):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
while 1:
print "\nWaiting for new connection...\n"
s.listen(1)
conn, addr = s.accept()
print "Client", addr[0], "connected and was directed to port",
addr[1]
data = conn.recv(1024)
# I want to get 'data' out of the loop, but keep the loop going
print "Client sent this message:", data
conn.close()


The classic solution to this problem is to create a new thread or fork a
new process to handle each connection and have the original server
imeediately loop to await a new connection.

This is what the threading and forking versions of the SocketServer
classes do. Take a look at the ForkingMixIn and ThreadingMixIn classes
of the SocketServer.py library to see how to turn your synchronous
design into something asynchronous.

regards
Steve
Jul 18 '05 #3
Brad Tilley wrote:
[...] I want to get the data that the clients send out of
the loop but at the same time keep the loop going so it can
continue listening for connections. [...]


I'd classify the ways to do it somewhat differently than the
other responses:

-Start multiple lines of execution
- using threads
- using processes (which, in Python, is less portable)

-Wait for action on multiple sockets within a single thread
- using os.select
- using some less-portable asynchronous I/O facility

Those are the low-level choices. Higher level facilities, such
as Asyncore and Twisted, are themselves based on one or more of
those.

Different servers have different needs, but when in doubt use
threads. Threading on the popular operating systems has
improved vastly in the last several years. Running a thousand
simultaneous threads is perfectly reasonable. Programmers using
threads have to be aware of things like race conditions, but
when threads are handling separate connections, most of their
operations are independent of other threads.
--
--Bryan
Jul 18 '05 #4
Bryan Olson wrote:
....
Different servers have different needs, but when in doubt use
threads. Threading on the popular operating systems has
improved vastly in the last several years. Running a thousand
simultaneous threads is perfectly reasonable.


If you want code to be portable this is false, and I'm amazed to see
this claim on c.l.p to be honest. It's a fairly good way to kill a
fair number of still well used OSs.

Just because a handful of OSs handle threading well these days does
not mean that you will end up with portable code this way. (Portable
in that you get the same overall behaviour - not the simple concept
of the code running)

_Small_ numbers of threads are very portable I would agree, but not
thousand(s).

Best Regards,
Michael.
--
Mi************@rd.bbc.co.uk
British Broadcasting Corporation, Research and Development
Kingswood Warren, Surrey KT20 6NP

This message (and any attachments) may contain personal views
which are not the views of the BBC unless specifically stated.
Jul 18 '05 #5
In article <tp******************@newssvr21.news.prodigy.com >,
Bryan Olson <fa*********@nowhere.org> wrote:
Jul 18 '05 #6
Cameron Laird wrote:
[...]
Different servers have different needs, but when in doubt use
threads. Threading on the popular operating systems has
improved vastly in the last several years. Running a thousand
simultaneous threads is perfectly reasonable. Programmers using
threads have to be aware of things like race conditions, but
when threads are handling separate connections, most of their
operations are independent of other threads.
Are you advising Python programmers to use only thread-based
approaches, and to judge Asyncore and Twisted on that basis?


No.
Do you intend that readers believe that it "is perfectly
reasonable" to design in terms of a single Python process
which manages up to "a thousand simultaneous *Python*
threads"?


Yes.
--
--Bryan
Jul 18 '05 #7
In article <C9******************@newssvr21.news.prodigy.com >,
Bryan Olson <fa*********@nowhere.org> wrote:
Jul 18 '05 #8
In article <os************@lairds.us>, I offered:
In article <C9******************@newssvr21.news.prodigy.com >,
Bryan Olson <fa*********@nowhere.org> wrote:
.
.
.
> Do you intend that readers believe that it "is perfectly
> reasonable" to design in terms of a single Python process
> which manages up to "a thousand simultaneous *Python*
> threads"?


Yes.

Thank you for this and your other unambiguous clarifications.

My daily world includes several Win* boxes running on 100-200 MHz
*86 processors, with memory ranging from 32 Mb up. Perhaps I
should make time during the next month to write and run a few
benchmarks applicable to my needs; I confess I haven't done so for
the case of a thousand simultaneous Python threads.
.
.
.

Do
http://groups.google.com/groups?th=181172231bcfb4a
and especially
http://groups.google.com/groups?fram...2e83a9ad69f788
make my hesitation about reliance on kilothreads on older platforms
more understandable?
--

Cameron Laird <cl****@phaseit.net>
Business: http://www.Phaseit.net
Jul 18 '05 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
by: Robert A. van Ginkel | last post by:
Hello Fellow Developer, I use the System.Net.Sockets to send/receive data (no tcpclient/tcplistener), I made a receivethread in my wrapper, the receivethread loops/sleeps while waiting for data...
6
by: Abubakar | last post by:
Hi, lets say I have a connected SOCKET s. At some point in time, I want to know if the "s" is still valid, that it is still connected. Is there any API that I can give me this information? And...
8
by: =?Utf-8?B?Sm9obg==?= | last post by:
Hi all, I am new to .net technologies. ASP.NET supports socket programming like send/receive in c or c++? I am developing web-site application in asp.net and code behind is Visual C#. In...
10
by: John Nagle | last post by:
Here are three network-related exceptions. These were caught by "except" with no exception type, because none of the more specific exceptions matched. This is what a traceback produced: 1....
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.