473,689 Members | 2,890 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

asyncore, asynchat and threads

Ok.. so here is my situation.

I have an application that I would love to be able to use asynchat with for it's handle_close, found_terminato r etc abstraction on.

However, I must use a separate thread for each connection as the work resulting from the communication can result in blocking for an indefinate period of time.

What I would love to do for each thread that is created, is to spin off an asynchat subclass and manually run some kind of loop() function that will do the
polling//select etc for me, in a threadsafe manner, only on the one asynchat class/connection for that thread (e.g. I supply variables to keep track of the data)

I have come across some sparse documentation that mentioned asyncore.loop() can be supplied a map of some sorts to loop through.. However, I have no idea
how to create this map to pass to asyncore.loop, nor what thread-safe precautions (if any) I need to make.

A simple example of a thread-safe way to create an asynchat subclass and get it to run would be great...

even any pointers, help, ideas on how to make something like this work would be greatly appreciated as well.

Thanks, Joshua Moore-Oliva
Jul 18 '05 #1
4 3011
Joshua Moore-Oliva wrote:
However, I must use a separate thread for each connection as the work
resulting from the communication can result in blocking for an indefinate period of time. .... even any pointers, help, ideas on how to make something like this work
would be greatly appreciated as well.


The usual approach is to use something along the lines of a separate
thread for the *work* rather than for the connection, or maybe a
thread pool, and communicate with those threads from the asynchat
thingies using Queue.Queues...

I'm not sure we have enough information about what you really need
to say more. You've given lots of info about what you "would love
to do" but not enough about what you actually need to do. <wink>

-Peter
Jul 18 '05 #2
The usual approach is to use something along the lines of a separate
thread for the *work* rather than for the connection, or maybe a
thread pool, and communicate with those threads from the asynchat
thingies using Queue.Queues...

I'm not sure we have enough information about what you really need
to say more. You've given lots of info about what you "would love
to do" but not enough about what you actually need to do. <wink>

-Peter
Ok, I will give more info, and also, you touched on a subject that I thought was impossible for my problem domain, but here goes.

I have a very simple protocol setup as follows.. basically it is a method for retrieving data, however I must route data through a central
connection to avoid database locking.

A simplified version of my transaction looks like so..
'>' means data from the client, '<' means data from the server BATCH_GET

< 1

Right after the BATCH_GET I wait on a Queue object for data to send to the remote client.
Depending on whether there is data available for this specific connection, this call could never return, but other connections
could also be requesting data, and there could be data available to them.

I currently do have a separate thread doing all the work, and it puts each threads data into a separate queue.

The problem that I ran into with using a direct asyncore//asynchat interface is this... (hoping there is a way to work around it)

consider the very likely possiblity that asynchat retrieves the line BATCH_GET, but there is no data to return.

I have to not return any data until there is data to return for that specific connection. If there is some way to send data down a asynchat
socket from another thread (thread-safely), or to somehow wake up an asynchat class, that would be great.

The problem is that once BATCH_GET is received, unless i handle it there and then, no more events will be generated as the remote client will
be waiting for a response.

Hopefully this sufficiently answers your 'what I actually *need* to do' question ;)

I don't want to flood the email with a lot of irrelevant info, so if you would like more infomation, ask away, but I think this covers the problem sufficiently...

Joshua Moore-Oliva

Jul 18 '05 #3
Joshua Moore-Oliva wrote:
I have to not return any data until there is data to
return for that specific connection. If there is some
way to send data down a asynchat
socket from another thread (thread-safely), or to
somehow wake up an asynchat class, that would be great.


It's been years since I used asyncore (I find Twisted much
simpler to use) but at least in theory such a thing is
possible. Twisted has a callFromThread( ) method that can
be used to get a callback in another thread to be called
from the "reactor" (the equivalent of the asyncore core
loop), which would do what you want. Asyncore might have
similar facilities.

I don't _fully_ grasp what you are doing, and I don't work
well without pictures in such cases (got a whiteboard handy?
draw a picture and send me a digital photo :-), but so
far it doesn't sound like you have anything approaching
unusual, compared to many other applications I've seen doing
the sorts of things yours seems to be doing.

-Peter
Jul 18 '05 #4
Well, I took at look at twisted.. and it seems pretty scary ;)

What I've done though is made some very minor changes to asyncore and asynchat, and now it appears to be threadsafe//more controllable.

If all works out well for me I'll submit the patch to python, since it's fully backwards compatbile with the old behaviour.

Thanks for the help

Joshua Moore-Oliva

On August 4, 2004 08:10 am, Peter Hansen wrote:
Joshua Moore-Oliva wrote:
I have to not return any data until there is data to
> return for that specific connection. If there is some
> way to send data down a asynchat
socket from another thread (thread-safely), or to
> somehow wake up an asynchat class, that would be great.


It's been years since I used asyncore (I find Twisted much
simpler to use) but at least in theory such a thing is
possible. Twisted has a callFromThread( ) method that can
be used to get a callback in another thread to be called
from the "reactor" (the equivalent of the asyncore core
loop), which would do what you want. Asyncore might have
similar facilities.

I don't _fully_ grasp what you are doing, and I don't work
well without pictures in such cases (got a whiteboard handy?
draw a picture and send me a digital photo :-), but so
far it doesn't sound like you have anything approaching
unusual, compared to many other applications I've seen doing
the sorts of things yours seems to be doing.

-Peter

Jul 18 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
2696
by: Michael Welsh | last post by:
In order to learn sockets in Python I am trying to write a simple group chat server and client. I need a little nudge, please. My question contains some GUI but only as decoration. The root question is asyncore / asynchat. I have read that twisted makes this all simple, but, I wanted to get my hands dirty here for educational purposes. I've managed to build a small echo server with asyncore and asynchat that can handle any number of...
3
3663
by: Jos | last post by:
Hello. I'm using the asyncore and _chat modules to create a network server. I also have, running in a separate thread(s), a "producer" which needs to "push" data onto the network connection(s). (This is all for a multi-player game server, so the threads would be individual games, that need to update the players when new events happen in the game) I currently have a module level list of async_chat instances, which the thread then reads...
7
2652
by: Paul Kozik | last post by:
I am working on the networking code for a small Multiplayer RPG I'm working on. I currently have some basic code using threads, but it seems like asyncore would be far better suited for my needs. However, I have trouble finding a solid example for what I need. Python.org and other sites provide simple examples, but they appear more intended for servers that simply send one peice of data to the client. I want to have a server program that...
0
1096
by: Giampaolo Rodola' | last post by:
Hi, I post this message here in the hope someone using asyncore could review this. Since the thing I miss mostly in asyncore is a system for calling a function after a certain amount of time, I spent the last 3 days trying to implement this with the hopes that this could be included in asyncore in the the future.
0
8594
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
8528
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
8947
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8783
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8786
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7621
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
5810
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4321
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
2965
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.