473,287 Members | 1,899 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

Async XMLRPC and job processing

I would like to set up a server that takes XMLRPC requests and
processes them asynchronously. The XMLRPC server part is trivial in
python. The job processing part is the part that I am having trouble
with. I have been looking at how to use threadpool, but I can't see
how to get that working. I would like to have the XMLRPC part of
things do something like:

def method1(a,b,c):
jobid=workRequest(long_method1,[a,b,c])
return(jobid)

def method2(a,b,c):
jobid=workRequest(long_method2,[a,b,c])
return(jobid)

def long_method1(a,b,c)
do lots of heavy computation, etc.
store results in files in a given directory, etc
return result

..... for any number of methods

Again, pretty straightforward. However, I run into problems with the
threadpool and xmlrpc server both waiting. In particular, if I do
something like:

server = SimpleXMLRPCServer.SimpleXMLRPCServer(.....)
server.serve_forever()

Where can tell the threadpool that I have set up to wait
indefinitely? Both are blocking.

Thanks,
Sean

Oct 17 '07 #1
1 3112
Sean Davis wrote:
I would like to set up a server that takes XMLRPC requests and
processes them asynchronously. The XMLRPC server part is trivial in
python. The job processing part is the part that I am having trouble
with. I have been looking at how to use threadpool, but I can't see
how to get that working. I would like to have the XMLRPC part of
things do something like:

def method1(a,b,c):
jobid=workRequest(long_method1,[a,b,c])
return(jobid)

def method2(a,b,c):
jobid=workRequest(long_method2,[a,b,c])
return(jobid)

def long_method1(a,b,c)
do lots of heavy computation, etc.
store results in files in a given directory, etc
return result

.... for any number of methods

Again, pretty straightforward. However, I run into problems with the
threadpool and xmlrpc server both waiting. In particular, if I do
something like:

server = SimpleXMLRPCServer.SimpleXMLRPCServer(.....)
server.serve_forever()

Where can tell the threadpool that I have set up to wait
indefinitely? Both are blocking.

Thanks,
Sean
This site shows how to make it multi-threaded:

http://aspn.activestate.com/ASPN/Coo.../Recipe/425043

But now you have to contend with the integrity of your data, provided
your going be storing data through these processes. You may want to look
into the Queue module to create a sort of message queue so your data can
be properly synchronized.

Hope this helps.

Adonis
Oct 17 '07 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Passynkov, Vadim | last post by:
I am using Asynchronous Query Processing interface from libpq library. And I got some strange results on Solaris My test select query is 'SELECT * from pg_user;' and I use select system...
8
by: TS | last post by:
Im in a web page and call an asynchronous method in business class. the call back method is in the web page. When page processes, it runs thru code begins invoking the method then the page...
2
by: Viet | last post by:
I have a couple of questions that hopefully someone could clarify for me. I have an app that uses the threading.timer to constantly poll a scanner to scan in documents. I understand that Async...
1
by: Simon Hart | last post by:
Hi, I thought I'd just open a thread in an attempt to get peoples feelers with regards to multithreading vs Async Web Service processing. Of course Web Services makes it easy to do Async...
0
by: Manfred Braun | last post by:
Hi All, I have a problem reading queue-messages async. My QueueReader has a Start() and a Stop() method and if my app starts, it calls Start(). The problem is, that there are possibly several...
0
by: David Hirschfield | last post by:
I have an xmlrpc client/server system that works fine, but I want to improve performance on the client side. Right now the system operates like this: client makes request from server (by...
6
by: half.italian | last post by:
Hi, I'm trying to serve up a simple XMLRPC server as a windows service. I got it to run properly, I'm just not sure how to stop it properly. Most of the documentation/examples I found for this...
10
by: Frankie | last post by:
It appears that System.Random would provide an acceptable means through which to generate a unique value used to identify multiple/concurrent asynchronous tasks. The usage of the value under...
0
by: Benjamin Grieshaber | last post by:
Hi, I´m on SuSE 9.3 with xmlrpc-c and xmlrpc-c-devel installed (ver. 0.9.10) I tried to compile php with xmlrpc support and got the following errors: ...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.