473,320 Members | 1,799 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Asynchronous Messaging

Hello,

I'm getting my feet wet in Python and thought I'd try to see how well
Python works for asynchronous messaging. I've been using asynchronous
messaging for 5 years and find it advantageous for many things. In the
scheme I use communication is not only asynchronous but it is also non-
blocking and inherently serialized via a queue.

This provides two benefits, a sender is never directly effected by the
receiver and since the receiver handles only one message at a time
it generally never has to use mutexes or semaphores. This allows for
the
programmer to use multiple threads without having to contend with the
associated issues mutexes have in the area of deadly embraces or race
conditions.

Anyway, below is a link to the first pass at an implementation
in Python, comments are welcome:

http://www.saville.com/python/mproc.tgz

Regards,

Wink Saville

Sep 26 '07 #1
5 1996
wink wrote:
This provides two benefits, a sender is never directly effected by the
receiver and since the receiver handles only one message at a time
it generally never has to use mutexes or semaphores. This allows for
the programmer to use multiple threads without having to contend with the
associated issues mutexes have in the area of deadly embraces or race
conditions.
import Queue # ?

</F>

Sep 26 '07 #2
On 2007-09-26, wink <wi**@saville.comwrote:
Hello,

I'm getting my feet wet in Python and thought I'd try to see how well
Python works for asynchronous messaging. I've been using asynchronous
Have a look at Twisted (www.twistedmatrix.com)
Albert

Sep 26 '07 #3
wink wrote:
But its performance is poor if the number of items on a
Queue becomes large because it is implemented using a list.
One of the things I was thinking of was doing another implementation
using of Queue which was based on deque.
Updating from 2.3 to something newer will fix that, of course:

$ more Queue.py

....

from collections import deque

....

class Queue:

...

def _init(self, maxsize):
self.maxsize = maxsize
self.queue = deque()

</F>

Sep 26 '07 #4
Fredrik,

You are most correct, but Queue is slow compared to deque but
not for the reason I guessed. Apparently it's because deque is
implemented in C while Queue is in python. Using the program below
it looks there is about a 35:1 speed difference.

100 d.append 0.000011s 0.1097us per
100 d.popL 0.000011s 0.1097us per
100 q.put 0.000429s 4.2892us per
100 q.get 0.000391s 3.9077us per

So someday it _might_ warrant adding Queue to collections.
#!/usr/bin/python

import timeit
import Queue
from collections import deque

setupD = """
from collections import deque
d=deque()
cnt = %d
for x in xrange(cnt):
d.append(x)
"""

setupQ = """
import Queue
q=Queue.Queue()
cnt = %d
for x in xrange(cnt):
q.put(x)
"""

def main():
cnt = 100
t = timeit.Timer(setup=setupD % cnt, stmt="d.append(0)")
time = min(t.repeat(10000, cnt))
print " %9d d.append %fs %7.4fus per" % \
(cnt, time, (time/cnt) * 1000000.0)

t = timeit.Timer(setup=setupD % cnt, stmt="d.popleft()")
time = min(t.repeat(10000, cnt))
print " %9d d.popL %fs %7.4fus per" % \
(cnt, time, (time/cnt) * 1000000.0)

t = timeit.Timer(setup=setupQ % cnt, stmt="q.put(0)")
time = min(t.repeat(10000, cnt))
print " %9d q.put %fs %7.4fus per" % \
(cnt, time, (time/cnt) * 1000000.0)

t = timeit.Timer(setup=setupQ % cnt, stmt="q.get()")
time = min(t.repeat(10000, cnt))
print " %9d q.get %fs %7.4fus per" % \
(cnt, time, (time/cnt) * 1000000.0)
if __name__ == "__main__":
main()

Sep 27 '07 #5
En Thu, 27 Sep 2007 01:43:32 -0300, wink <wi**@saville.comescribi�:
You are most correct, but Queue is slow compared to deque but
not for the reason I guessed. Apparently it's because deque is
implemented in C while Queue is in python. Using the program below
it looks there is about a 35:1 speed difference.
That't not the reason. A Queue is built around a container, and it happens
to be a deque in the default implementation. But the important thing is
that a Queue is a synchronized object - it performs the necesary
synchronization to ensure proper operation even from multiple threads
attempting to use it at the same time.
So your comparison is meaningless, apart from telling that using mutexes
is not cheap.

--
Gabriel Genellina

Sep 27 '07 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Dennis Owens | last post by:
I want to develop a Client/Server messaging system that does not require Microsoft messaging. I want to use the .Net remoting, but I am having some real problems with the remoting. The remoting...
0
by: Dennis Owens | last post by:
Read below for previous conversation. We are developing an application that will some day run on anything from a computer down to a PDA (this is were the Lightweight comes in). The messaging...
1
by: ptdaw | last post by:
After spending a little time reading the documentation, I attempted to add some messaging code to my VB.Net project, only to discover that I couldn't get past my Imports System.Messaging ...
4
by: Michael C | last post by:
Hi all, Is there an easy way to get the parameters of an asynchronous delegate call from the callback function? Here's an example of what I'm trying to do: private delegate ArrayList...
1
by: John A Grandy | last post by:
in .NET 2.0 where is the type Message located ? in .NET 1.1 it was System.Messaging.Message
9
by: Sharon | last post by:
I have a remoting object, lets call it A, that fires an event to remotely registered handlers, lets call it B. The invocation is working OK and the remote handler B is handling the event like it...
4
by: John Grant | last post by:
If I build a web services today with VS 2005 does it support reliable messaging? If I use WSE 3.0 will it support reliable messaging? If I don’t have reliable messaging can I make a web method...
3
by: =?Utf-8?B?aGVyYmVydA==?= | last post by:
I need to build an event-based asynchronous pattern (around a send/receive messaging API). Is there a step-by-step guidance about how to write code for the EBAP ? Does any book cover this theme...
2
by: jparulan | last post by:
Hi All, I am developing web service that does not use IIS, I was successful on deploying a normal web service with System.Data and System.Web namespaces. BUT when I added the...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.