By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
445,909 Members | 2,008 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 445,909 IT Pros & Developers. It's quick & easy.

Async XMLRPC and job processing

P: n/a
I would like to set up a server that takes XMLRPC requests and
processes them asynchronously. The XMLRPC server part is trivial in
python. The job processing part is the part that I am having trouble
with. I have been looking at how to use threadpool, but I can't see
how to get that working. I would like to have the XMLRPC part of
things do something like:

def method1(a,b,c):
jobid=workRequest(long_method1,[a,b,c])
return(jobid)

def method2(a,b,c):
jobid=workRequest(long_method2,[a,b,c])
return(jobid)

def long_method1(a,b,c)
do lots of heavy computation, etc.
store results in files in a given directory, etc
return result

..... for any number of methods

Again, pretty straightforward. However, I run into problems with the
threadpool and xmlrpc server both waiting. In particular, if I do
something like:

server = SimpleXMLRPCServer.SimpleXMLRPCServer(.....)
server.serve_forever()

Where can tell the threadpool that I have set up to wait
indefinitely? Both are blocking.

Thanks,
Sean

Oct 17 '07 #1
Share this Question
Share on Google+
1 Reply


P: n/a
Sean Davis wrote:
I would like to set up a server that takes XMLRPC requests and
processes them asynchronously. The XMLRPC server part is trivial in
python. The job processing part is the part that I am having trouble
with. I have been looking at how to use threadpool, but I can't see
how to get that working. I would like to have the XMLRPC part of
things do something like:

def method1(a,b,c):
jobid=workRequest(long_method1,[a,b,c])
return(jobid)

def method2(a,b,c):
jobid=workRequest(long_method2,[a,b,c])
return(jobid)

def long_method1(a,b,c)
do lots of heavy computation, etc.
store results in files in a given directory, etc
return result

.... for any number of methods

Again, pretty straightforward. However, I run into problems with the
threadpool and xmlrpc server both waiting. In particular, if I do
something like:

server = SimpleXMLRPCServer.SimpleXMLRPCServer(.....)
server.serve_forever()

Where can tell the threadpool that I have set up to wait
indefinitely? Both are blocking.

Thanks,
Sean
This site shows how to make it multi-threaded:

http://aspn.activestate.com/ASPN/Coo.../Recipe/425043

But now you have to contend with the integrity of your data, provided
your going be storing data through these processes. You may want to look
into the Queue module to create a sort of message queue so your data can
be properly synchronized.

Hope this helps.

Adonis
Oct 17 '07 #2

This discussion thread is closed

Replies have been disabled for this discussion.