473,574 Members | 2,389 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

new to python network programming is async_chat.push thread-safe?python3.0

I wrote this server to handle incoming messages in a process using
multiprocessing named "handler", and sending message in a Thread named
"sender", 'cause I think the async_chat object can not pass between
processes.

My project is a network gate server with many complex logic handler
behind, so I use multiprocessing to handle them separately and send
back the clients later when done.
To use the server multicore cpu I tried to separate the send and
receive function in different process but it seems can not be done :)

I just get questions about this design:
1. is async_chat.push thread-safe? 'Cause I found random errors
reporting push fifo queue out of index 0 sometimes
2. is the whole design odd in any way?
here is my code

import asyncore, asynchat
import os, socket, string
from multiprocessing import Process,Manager
import pickle
import _thread

PORT = 80

policyRequest = b"<policy-file-request/>"
policyReturn = b"""<cross-domain-policy>
<allow-access-from domain="*" to-ports="*" />
</cross-domain-policy\x00"""

def handler(taskLis t,msgList):
while 1:
print('getting task')
item = pickle.loads(ta skList.get())
print('item before handle ', item)
item['msg'] += b' hanlded done'
msgList.put(pic kle.dumps(item) )

def findClient(id):
for item in clients:
if item.idx == id:
return item

def sender():
global msgList
while 1:
item = pickle.loads(ms gList.get())
#print time()
c = findClient(item['cid'])
#print time()
c.push(item['msg'])
print('msg sent ',item['msg'])
#print time()

class HTTPChannel(asy nchat.async_cha t):

def __init__(self, server, sock, addr):
global cid;
asynchat.async_ chat.__init__(s elf, sock)
self.set_termin ator(b"\x00")
self.data = b""
cid += 1
self.idx = cid
if not self in clients:
clients.append( self)

def collect_incomin g_data(self, data):
self.data = self.data + data
print(data)

def found_terminato r(self):
global taskList
print("found",s elf.data)
if self.data == policyRequest:
self.push(polic yReturn)
else:
d = {'cid':self.idx ,'msg':self.dat a}
taskList.put(pi ckle.dumps(d))
self.data = b""

def handle_close(se lf):
if self in clients:
clients.remove( self)

class HTTPServer(asyn core.dispatcher ):

def __init__(self, port):
asyncore.dispat cher.__init__(s elf)
self.create_soc ket(socket.AF_I NET, socket.SOCK_STR EAM)
self.bind(("", port))
self.listen(5)

def handle_accept(s elf):
conn, addr = self.accept()
HTTPChannel(sel f, conn, addr)
#
# try it out
if __name__ == "__main__":
s = HTTPServer(PORT )
print ("serving at port", PORT, "...")

#clients sock obj list stored for further use
clients=[]

#client id auto increasement
cid = 0

manager = Manager()
taskList = manager.Queue()
msgList = manager.Queue()
h = Process(target= handler,args=(t askList,msgList ))
h.start()
_thread.start_n ew_thread(sende r,())
print('entering loop')

asyncore.loop()
Oct 24 '08 #1
0 1350

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

36
6345
by: Andrea Griffini | last post by:
I did it. I proposed python as the main language for our next CAD/CAM software because I think that it has all the potential needed for it. I'm not sure yet if the decision will get through, but something I'll need in this case is some experience-based set of rules about how to use python in this context. For example... is defining...
10
3672
by: Andrew Dalke | last post by:
Is there an author index for the new version of the Python cookbook? As a contributor I got my comp version delivered today and my ego wanted some gratification. I couldn't find my entries. Andrew dalke@dalkescientific.com
3
5059
by: Yan ZHANG | last post by:
1. In a network, there are a number of nodes. Each node has a queue and a thread, performing the discrete event simulation. class CNode {... queue q; CWinThread* pthread; void Start();//to execute the thread .... }; where CWinThread is the thread class in MFC.
0
1229
by: rix | last post by:
Hi there, I am writing an application that requires a client-server interaction. Looking up on the internet and working on it I came up with something like this: a class ADFSServer that takes inbound connections and dispatches them to ADFSReceiver. The connection is always initialized by ADFSSender. here is the code: import asynchat
1
1807
by: sajaisosa | last post by:
Hey there I've been following through the book "Beginning Python from Novice to Professional", and I have learned a great deal. At the moment, I am trying to expand upon the chatroom project near the end of this book. My problem, is that I can't get my head around how to allow the users to add new rooms. At the moment, there is only one room....
206
8231
by: WaterWalk | last post by:
I've just read an article "Building Robust System" by Gerald Jay Sussman. The article is here: http://swiss.csail.mit.edu/classes/symbolic/spring07/readings/robust-systems.pdf In it there is a footprint which says: "Indeed, one often hears arguments against building exibility into an engineered sys- tem. For example, in the philosophy of...
5
2454
by: ludvig.ericson | last post by:
Hello, My question concerns asynchat in particular. With the following half- pseudo code in mind: class Example(asynchat.async_chat): def readable(self): if foo: self.push_with_producer(ProducerA()) return asynchat.async_chat.readable(self)
1
124
by: Jean-Paul Calderone | last post by:
On Tue, 13 May 2008 16:44:11 -0700 (PDT), Giampaolo Rodola' <gnewsg@gmail.comwrote: Why? Isn't this why subtraction exists? If there is a call scheduled to happen at T1 and the current time is T2, then I know that after (T1 - T2) elapses, it will be time to run the call. Why do I have to do any checks at all? I just tell select() to wait...
0
837
by: Gabriel Genellina | last post by:
QOTW: "IIRC the idea was so that managers could write programs in English. It failed because nobody could write a parser that would handle something like 'The bottom line is that the stakeholder group requires the situation going forward to be such as to facilitate the variable known as x to provide the same outcome when stimulated by...
0
7720
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
8230
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
1
7810
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
8096
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
6451
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
0
5299
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3739
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
1
2240
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
0
1056
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.