469,603 Members | 2,110 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,603 developers. It's quick & easy.

Approaches of interprocess communication

Hi all,

Supposing you have two separate processes running on the same box,
what approach would you suggest to communicate between those two
processes.

Let me list the ones I know of:

* Sockets
Advantage: Supported per se in nearly every programming language
without even the need to install additional packages
Disadvantage: Lot's of code to write, and it's kind of silly to
communicate via TCP/IP if the processes run on the same machine.

* Webservices
Advantage: Relatively easy to use, can work across different
languages
Disadvantage: Even more overhead on the TCP/IP side that simple
sockets, as really bulky SOAP messages need to be passed around.

* CORBA -- similar to webservices but more complicated to code.

* Shared memory
I don't know much about this subject.

Supposing both processes are written in Python, is there any other way
to achieve this? To me, shared memory sound the most suited approach.
But as said, I am still fuzzy in this area. Where can I find more
information on this subject?

Feb 16 '07 #1
22 2516
En Fri, 16 Feb 2007 07:11:36 -0300, exhuma.twn <ex****@gmail.comescribió:
Hi all,

Supposing you have two separate processes running on the same box,
what approach would you suggest to communicate between those two
processes.

Let me list the ones I know of:

* Sockets
Advantage: Supported per se in nearly every programming language
without even the need to install additional packages
Disadvantage: Lot's of code to write, and it's kind of silly to
communicate via TCP/IP if the processes run on the same machine.
Not so much code, really.
(And I would expect that making a connection to "localhost" actually does
*not* go down up to the network card hardware layer, but I don't know for
real if this is the case or not).
* Webservices
Advantage: Relatively easy to use, can work across different
languages
Disadvantage: Even more overhead on the TCP/IP side that simple
sockets, as really bulky SOAP messages need to be passed around.
You could use XMLRPC, wich is a lot simpler (but less powerful).
* CORBA -- similar to webservices but more complicated to code.
I would stay away as far as I could.
* Shared memory
I don't know much about this subject.
You forget the most basic one, stdio redirection. Easy, available on
almost any language, but limited to just a pair of processes.
You can add queues and messages.
Supposing both processes are written in Python, is there any other way
to achieve this? To me, shared memory sound the most suited approach.
But as said, I am still fuzzy in this area. Where can I find more
information on this subject?
Pyro appears to be a good alternative (altough I've never used it yet).

--
Gabriel Genellina

Feb 16 '07 #2
Supposing you have two separate processes running on the same box,
what approach would you suggest to communicate between those two
processes.

Let me list the ones I know of:

* Sockets
Advantage: Supported per se in nearly every programming language
without even the need to install additional packages
Disadvantage: Lot's of code to write, and it's kind of silly to
communicate via TCP/IP if the processes run on the same machine.

* Webservices
Advantage: Relatively easy to use, can work across different
languages
Disadvantage: Even more overhead on the TCP/IP side that simple
sockets, as really bulky SOAP messages need to be passed around.

* CORBA -- similar to webservices but more complicated to code.

* Shared memory
I don't know much about this subject.

Supposing both processes are written in Python, is there any other way
to achieve this? To me, shared memory sound the most suited approach.
But as said, I am still fuzzy in this area. Where can I find more
information on this subject?
Hi, if your requirements are sufficiently light then pylinda might be
an easy-to-use solution:

http://www-users.cs.york.ac.uk/~aw/pylinda/

A simple example is here:

http://www-users.cs.york.ac.uk/~aw/p.../beginner.html

HTH,
Daniel
Feb 16 '07 #3
"exhuma.twn" <ex****@gmail.comwrites:
Supposing you have two separate processes running on the same box,
what approach would you suggest to communicate between those two
processes.

Let me list the ones I know of:

* Sockets
Advantage: Supported per se in nearly every programming language
without even the need to install additional packages
This would be my choice. But first, set up a well-defined *protocol*
(preferably based on text commands) for the two processes to use for
communication; don't have each of them being intricately aware of each
others' implementation.
Disadvantage: Lot's of code to write, and it's kind of silly to
communicate via TCP/IP if the processes run on the same machine.
You can cut down on the amount of code by using the standard library
"cmd" module to handle a command interface, hooking the stdin and
stdout of the commandline handler to the socket.

If you're already thinking about cooperating processes, you should
make them network-neutral anyway from the start.

Here's what _The Art of Unix Programming_ has to say on the topic of
text protocols:

<URL:http://www.catb.org/~esr/writings/taoup/html/ch05s01.html>

and IPC tactics for peer processes:

<URL:http://www.catb.org/~esr/writings/taoup/html/ch07s07.html>

--
\ "There are only two ways to live your life. One is as though |
`\ nothing is a miracle. The other is as if everything is." -- |
_o__) Albert Einstein |
Ben Finney

Feb 16 '07 #4
"Gabriel Genellina" <ga******@yahoo.com.arwrites:
(And I would expect that making a connection to "localhost" actually
does *not* go down up to the network card hardware layer, but I
don't know for real if this is the case or not).
It damned well better. That's the entire point of the loopback
interface: to get all the network layer code involved, but not to talk
on a physical network interface.

If a programmer decides on behalf of the user that "localhost" should
be treated specially, that programmer is making an error.

--
\ "If you can't beat them, arrange to have them beaten." -- |
`\ George Carlin |
_o__) |
Ben Finney

Feb 16 '07 #5
In article <11**********************@m58g2000cwm.googlegroups .com>,
exhuma.twn <ex****@gmail.comwrote:
>Supposing you have two separate processes running on the same box,
what approach would you suggest to communicate between those two
processes.
[...]
>* Webservices
Advantage: Relatively easy to use, can work across different
languages
Disadvantage: Even more overhead on the TCP/IP side that simple
sockets, as really bulky SOAP messages need to be passed around.

* CORBA -- similar to webservices but more complicated to code.
Lots of people say that, but I don't think it's true. Obviously as the
maintainer of a CORBA implementation I'm biased, but take a look at
some examples of code that implement SOAP clients and servers and
compare it to similar CORBA code. Especially in Python, the SOAP code
tends to be incredibly verbose and complex, and the CORBA code really
small and simple.

My recommendation would be that for simple communications where
performance isn't important use XML-RPC; for more complex cases where
performance is a bit more important but you don't need anything except
Python, use Pyro; for cases where performance is particularly
important and/or you need cross-language communications, use CORBA.

Cheers,

Duncan.

--
-- Duncan Grisby --
-- du****@grisby.org --
-- http://www.grisby.org --

--
Posted via NewsDemon.com - Premium Uncensored Newsgroup Service
------->>>>>>http://www.NewsDem
Feb 16 '07 #6
On Feb 16, 1:33 pm, Duncan Grisby <duncan-n...@grisby.orgwrote:
In article <1171620696.577982.283...@m58g2000cwm.googlegroups .com>,

exhuma.twn <exh...@gmail.comwrote:
Supposing you have two separate processes running on the same box,
what approach would you suggest to communicate between those two
processes.

[...]
* Webservices
Advantage: Relatively easy to use, can work across different
languages
Disadvantage: Even more overhead on the TCP/IP side that simple
sockets, as really bulky SOAP messages need to be passed around.
* CORBA -- similar to webservices but more complicated to code.

Lots of people say that, but I don't think it's true. Obviously as the
maintainer of a CORBA implementation I'm biased, but take a look at
some examples of code that implement SOAP clients and servers and
compare it to similar CORBA code. Especially in Python, the SOAP code
tends to be incredibly verbose and complex, and the CORBA code really
small and simple.

My recommendation would be that for simple communications where
performance isn't important use XML-RPC; for more complex cases where
performance is a bit more important but you don't need anything except
Python, use Pyro; for cases where performance is particularly
important and/or you need cross-language communications, use CORBA.
Maybe this line of mine was a bit too condensed ;) I fully agree with
you on what you say about CORBA. It's just that for most people IDL
looks a bit out of place. Especially because it resembles C. But once
you actually wrote a few projects using CORBA, you actually begin to
see it's elegance ;)

I find Webservices "easier" as you can (although it's not
recommendable) leave out the WSDL part. And some implementations
actually generate the WSDL for you. But it's true, comparing WSDL and
IDL I'd rather write some IDL than WSDL ;)

But, returning back to topic, I first want to thank you all for your
input. I appreciate all the ideas. Especially the idea of using the
"cmd" module for sockets. That sound's very intriguing and I will very
likely play around with that. But also PYRO seems quite useful.

About "Linda": Am I right that it looks very similar to "JavaSpaces"?
If yes, are there any funcdamental differences between those two?

Feb 16 '07 #7
Maybe this line of mine was a bit too condensed ;) I fully agree with
you on what you say about CORBA. It's just that for most people IDL
looks a bit out of place. Especially because it resembles C. But once
you actually wrote a few projects using CORBA, you actually begin to
see it's elegance ;)
It looks out of the place in comparison to what exactly? A WSDL-document?
You certainly me laugh hard on that....
I find Webservices "easier" as you can (although it's not
recommendable) leave out the WSDL part. And some implementations
actually generate the WSDL for you.
You can't leave WSDL out of SOAP, you can leave it out of XMLRPC. Which is
nice and easy, but lacks any contract whatsoever. Nothing I as a Pythoneer
care to much about, but some people bother.

And generating the WSDL works from what exactly? ah, Java-code for example,
using java2wsdl. Which has a striking resemblance to.... C. Funny....
But it's true, comparing WSDL and
IDL I'd rather write some IDL than WSDL ;)
So - you say that yourself, but still you previously claimed IDL looks
strange to the eye?

Diez
Feb 16 '07 #8
On 16 Feb, 14:16, "Diez B. Roggisch" <d...@nospam.web.dewrote:
>
You can't leave WSDL out of SOAP
Yes you can, since they're two different things. What you probably
meant was that you can't leave WSDL out of "big architecture", W3C
standards-intensive Web services. Of course, RPC-style SOAP without
the contracts imposed by WSDL may remove some of the attractions for
some people (who really should consider CORBA, anyway), but then
there's always document-oriented SOAP, although if you don't want the
baggage associated with routing and other things, plain XML messaging
would be easier. And there's always XMPP if you want stuff like
routing and a standard that isn't undergoing apparently continuous
change.

Paul

Feb 16 '07 #9
Paul Boddie wrote:
On 16 Feb, 14:16, "Diez B. Roggisch" <d...@nospam.web.dewrote:
>>
You can't leave WSDL out of SOAP

Yes you can, since they're two different things. What you probably
meant was that you can't leave WSDL out of "big architecture", W3C
standards-intensive Web services. Of course, RPC-style SOAP without
the contracts imposed by WSDL may remove some of the attractions for
some people (who really should consider CORBA, anyway), but then
there's always document-oriented SOAP, although if you don't want the
baggage associated with routing and other things, plain XML messaging
would be easier. And there's always XMPP if you want stuff like
routing and a standard that isn't undergoing apparently continuous
change.
Didn't know that. Yet I presume it is pretty awful to manually decompose and
compose the method invocations and parameter sets.

I've got no idea what document-orientied SOAP means either. Is that just
pushing XML-files over a protocol?

Diez
Feb 16 '07 #10
On 16 Feb, 14:53, "Diez B. Roggisch" <d...@nospam.web.dewrote:
>
[XMPP, XML messaging]
Didn't know that. Yet I presume it is pretty awful to manually decompose and
compose the method invocations and parameter sets.
It depends on how well you like working with XML, I suppose.
I've got no idea what document-orientied SOAP means either. Is that just
pushing XML-files over a protocol?
As I understand it, yes, although you still have to put the payload
inside the usual SOAP boilerplate.

Paul

Feb 16 '07 #11
Ben Finney wrote:
"Gabriel Genellina" <ga******@yahoo.com.arwrites:
>(And I would expect that making a connection to "localhost" actually
does *not* go down up to the network card hardware layer, but I
don't know for real if this is the case or not).

It damned well better. That's the entire point of the loopback
interface: to get all the network layer code involved, but not to talk
on a physical network interface.

If a programmer decides on behalf of the user that "localhost" should
be treated specially, that programmer is making an error.
Inter-process TCP/IP communication between two processes on the same
host invariably uses the loopback interface (network 127.0.0.0).
According to standards, all addresses in that network space refer to the
local host, though 127.0.0.1 is conventionally used.

The transmit driver for the loopback interface receives a datagram from
the local network layer and immediately announces its reception back to
the local network layer.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Blog of Note: http://holdenweb.blogspot.com
See you at PyCon? http://us.pycon.org/TX2007

Feb 16 '07 #12
On 2/16/07, Shadab Sayani <sh**********@yahoo.comwrote:
Hi ,
We have a project where I need to read files store
them in database in the backend.We have done this in
python.Now we decided to use Ajax technique for user
interface.For that we found that GWT is one of the
best toolkits.Now I got a doubt can I interface GWT
with python.
http://pyjamas.pyworks.org/ is the way to go.

--
Roman Yakovenko
C++ Python language binding
http://www.language-binding.net/
Feb 16 '07 #13
About "Linda": Am I right that it looks very similar to "JavaSpaces"?
If yes, are there any funcdamental differences between those two?
Yes, they are both linda implementations, but I have no idea what so
ever how they compare. A local java expert maybe?
Feb 16 '07 #14
Shadab Sayani wrote:
Hi ,
We have a project where I need to read files store
them in database in the backend.We have done this in
python.Now we decided to use Ajax technique for user
interface.For that we found that GWT is one of the
best toolkits.Now I got a doubt can I interface GWT
with python.
Thanks ,
Shadab.

Send instant messages to your online friends http://uk.messenger.yahoo.com
Please note that you should not create a new conversation on a newsgroup
by editing a reply to an unrelated post - your comments are now threaded
along wiht "Approaches of Interprocess Communication", making them
unnecessarily hard to find and somewhat confusingly positioned.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Blog of Note: http://holdenweb.blogspot.com
See you at PyCon? http://us.pycon.org/TX2007
Feb 16 '07 #15
Shadab Sayani wrote:
Hi ,
We have a project where I need to read files store
them in database in the backend.We have done this in
python.Now we decided to use Ajax technique for user
interface.For that we found that GWT is one of the
best toolkits.Now I got a doubt can I interface GWT
with python.
Thanks ,
Shadab.

Send instant messages to your online friends http://uk.messenger.yahoo.com
Please note that you should not create a new conversation on a newsgroup
by editing a reply to an unrelated post - your comments are now threaded
along wiht "Approaches of Interprocess Communication", making them
unnecessarily hard to find and somewhat confusingly positioned.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Blog of Note: http://holdenweb.blogspot.com
See you at PyCon? http://us.pycon.org/TX2007

Feb 16 '07 #16
exhuma.twn wrote:
* Sockets
Advantage: Supported per se in nearly every programming
language without even the need to install additional packages
Disadvantage: Lot's of code to write,
Who's Lot? :)

No, seriously. Why would you think that it's much to write? It can,
especially using Python, be as easy as using a file.
and it's kind of silly to communicate via TCP/IP if the
processes run on the same machine.
Why?

- Do you worry about CPU cycles, memory, ...?
- Who forces you to use TCP?
Supposing both processes are written in Python, is there any other
way to achieve this?
It depends on what those processes need to share. What you need can
range from a simple unix socket connection to a full-fledged SQL
database server.

Regards,
Björn

--
BOFH excuse #301:

appears to be a Slow/Narrow SCSI-0 Interface problem

Feb 16 '07 #17
On Feb 16, 5:11 am, "exhuma.twn" <exh...@gmail.comwrote:
Hi all,

Supposing you have two separate processes running on the same box,
what approach would you suggest to communicate between those two
processes.
Spring Python makes it easy to get processes talking to each other.
You can write your code like you are talking locally, then when its
time to separate it either into another thread, another python
process, or on another node, it is just a reconfiguration issue.

http://springpython.python-hosting.c...ibutedRemoting

Feb 16 '07 #18
In article <11**********************@m58g2000cwm.googlegroups .com>,
"exhuma.twn" <ex****@gmail.comwrote:
Hi all,

Supposing you have two separate processes running on the same box,
what approach would you suggest to communicate between those two
processes.
Hi exhuma,
That would depend on what data I was exchanging between the processes.
For instance, if process A spawns work process B and wants to be able
monitor B's progress, a message-based protocol might be kind of chatty.
In this situation shared memory is probably a better fit because B can
write it's progress to a chunk of shared memory and A can read that at
its leisure. OTOH if the conversation is more event-driven, then a
messaging protocol makes good sense.

FYI there's a Python module (with sample code) for using shared memory
on most *nix systems here:
http://NikitaTheSpider.com/python/shm/

HTH

--
Philip
http://NikitaTheSpider.com/
Whole-site HTML validation, link checking and more
Feb 16 '07 #19
"exhuma.twn" <ex****@gmail.comwrote:
Hi all,

Supposing you have two separate processes running on the same box,
what approach would you suggest to communicate between those two
processes.
8< ------ sockets,webservices,CORBA,shared memory ---------------
Supposing both processes are written in Python, is there any other way
to achieve this? To me, shared memory sound the most suited approach.
But as said, I am still fuzzy in this area. Where can I find more
information on this subject?
Using named pipes works for me

Also look at Pyro, albeit not aimed at being in the same box.

- Hendrik

Feb 17 '07 #20
Quoth Steve Holden <st***@holdenweb.com>:
| Ben Finney wrote:
....
| If a programmer decides on behalf of the user that "localhost" should
| be treated specially, that programmer is making an error.
|
| Inter-process TCP/IP communication between two processes on the same
| host invariably uses the loopback interface (network 127.0.0.0).
| According to standards, all addresses in that network space refer to the
| local host, though 127.0.0.1 is conventionally used.
|
| The transmit driver for the loopback interface receives a datagram from
| the local network layer and immediately announces its reception back to
| the local network layer.

Are you saying, in that first paragraph, that if for example I telnet to
my local host's external IP address, to a service that bound explicitly to
the external interface -- I'll actually be going through the loopback
interface anyway, invariably regardless of host network implementation?

Donn Cave, do**@drizzle.com
Feb 17 '07 #21
| If a programmer decides on behalf of the user that "localhost" should
| be treated specially, that programmer is making an error.
|
| Inter-process TCP/IP communication between two processes on the same
| host invariably uses the loopback interface (network 127.0.0.0).
| According to standards, all addresses in that network space refer to the
| local host, though 127.0.0.1 is conventionally used.
|
| The transmit driver for the loopback interface receives a datagram from
| the local network layer and immediately announces its reception back to
| the local network layer.

Are you saying, in that first paragraph, that if for example I telnet to
my local host's external IP address, to a service that bound explicitly to
the external interface -- I'll actually be going through the loopback
interface anyway, invariably regardless of host network implementation?
On linux at least, this is true, yes.
--
damjan
Feb 17 '07 #22
Donn Cave wrote:
Quoth Steve Holden <st***@holdenweb.com>:
| Ben Finney wrote:
...
| If a programmer decides on behalf of the user that "localhost" should
| be treated specially, that programmer is making an error.
|
| Inter-process TCP/IP communication between two processes on the same
| host invariably uses the loopback interface (network 127.0.0.0).
| According to standards, all addresses in that network space refer to the
| local host, though 127.0.0.1 is conventionally used.
|
| The transmit driver for the loopback interface receives a datagram from
| the local network layer and immediately announces its reception back to
| the local network layer.

Are you saying, in that first paragraph, that if for example I telnet to
my local host's external IP address, to a service that bound explicitly to
the external interface -- I'll actually be going through the loopback
interface anyway, invariably regardless of host network implementation?
I wasn't specifically saying that, no. However on Solaris I have
observed local connections to an external interface actually increasing
the packet count on the loopback, but I can't confirm whether those
connections were to services specifically bound only to the external
interface.

Certainly on Windows XP there is a host-specific route via 127.0.0.1 to
the external interfaces as well as the network route via the external
interface. I wouldn't necessarily expect this to extend to other
platforms. This is demonstrated by the following output from "route
print". I have chopped the metric column to avoid line wrapping.

================================================== ===================
Active Routes:
Network Destination Netmask Gateway Interface
0.0.0.0 0.0.0.0 192.168.123.254 192.168.123.123
127.0.0.0 255.0.0.0 127.0.0.1 127.0.0.1
192.168.123.0 255.255.255.0 192.168.123.123 192.168.123.123
192.168.123.123 255.255.255.255 127.0.0.1 127.0.0.1
192.168.123.255 255.255.255.255 192.168.123.123 192.168.123.123
192.168.174.0 255.255.255.0 192.168.174.1 192.168.174.1
192.168.174.1 255.255.255.255 127.0.0.1 127.0.0.1
192.168.174.255 255.255.255.255 192.168.174.1 192.168.174.1
224.0.0.0 240.0.0.0 192.168.123.123 192.168.123.123
224.0.0.0 240.0.0.0 192.168.174.1 192.168.174.1
255.255.255.255 255.255.255.255 192.168.123.123 3
255.255.255.255 255.255.255.255 192.168.123.123 192.168.123.123
255.255.255.255 255.255.255.255 192.168.174.1 192.168.174.1
Default Gateway: 192.168.123.254
================================================== ===================

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
Blog of Note: http://holdenweb.blogspot.com
See you at PyCon? http://us.pycon.org/TX2007

Feb 18 '07 #23

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

7 posts views Thread by Daniel | last post: by
4 posts views Thread by Charles Packer | last post: by
3 posts views Thread by James Aguilar | last post: by
7 posts views Thread by Michael Butscher | last post: by
reply views Thread by Murali | last post: by
2 posts views Thread by Murali | last post: by
reply views Thread by devrayhaan | last post: by
reply views Thread by gheharukoh7 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.