473,388 Members | 1,264 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,388 software developers and data experts.

Inter-process communication, how?

Hi,
let's say I have two scripts: one does some computations and the other
one is a graphical front end for launching the first one. And both run
in separate processes (front end runs and that it spawns a subprocess
with the computation). Now, if the computation has a result I would
like to display it in the front end. In another words, I would like to
pass some data from one process to another. How to do that? I'm
affraid I can't use a pipe since the computation could print out some
logging (if I understant pipes correctly).
Thanks!
Dec 16 '07 #1
6 3678
On Dec 16, 2:42 am, Dennis Lee Bieber <wlfr...@ix.netcom.comwrote:
>
Have you perused and been perplexed by the "Is Python a Scripting
Language" thread? <G>
Oh, no!! Script as in "shell script".

>
Question: Are you creating both scripts from scratch?
Yes?
Yes.
Then you can define whatever protocol is needed for your usage and
is available on your OS.

If it is a one-shot (spawn sub, wait, retrieve results) you could
generate a temporary file name in the parent, pass that name to the sub
when invoking it, wait for the sub to complete (giving you a status
code) and then read the result the sub has written to the file.
Yes, it's once shot. But how do I pass "that name"?
From all the arguments of class Popen (http://docs.python.org/lib/
node529.html) perhaps I could use "env" only. Or am I wrong?
TCP/IP sounds good but isn't it a bit too heavy?
And what the other options goes I would prefer a cross platform
solution, if there is one.

PS: both with mmpam and temp file you probably meant that I should
hard code some as-weirdest-filename-as-possible two both programs but
what if I run the computation several times?

And thanks for reply!

>
Or, you could have parent and sub both mmap the same "file",
essentially passing data in memory unless it becomes too large and pages
out to swap disk. You might even be able to do bidirectional and dynamic
updates (rather than waiting for the sub to exit)... Define, say, an
epoch count for each process -- these would be the first two words of
the mmap file

|p-epoch|s-epoch|n-words of p-data (fixed constant known to both)|n-words of s-data|

periodically the parent would examine the value of s-epoch, and if it
has changed, read the s-data.. (both sides write the data first, then
update the epoch count). When the epoch stays the same and two
consecutive reads of the data match, you have stable data (so the reads
should occur more often than updates) and can process the data
transferred.

OR, you could have the parent open a TCP/IP socket as a server, and
pass the socket port number to the sub. The sub would then connect to
that port and write the data. For bidirectional you could pass the
parent port, and the sub's first action is to connect and pass a port
that it will be monitoring.

On a VMS system, the processes would connect to named "mailboxes"
and use QIO operations to pass data between them.

On an Amiga you'd use "message ports" (which operated somewhat
similar to VMS mailboxes except that mailboxes had an independent
existence, multiple processes can read or write to them -- message ports
were readable by the creating process, but could have messages sent from
anywhere; typically passing the message port [address of a linked list
of messages] for replies). Or a higher level message port: an ARexx
port.

On a Windows NT class system, the win32 extensions allow access to
Windows Named Pipes... Or maybe the Windows clipboard could be used...
--
Wulfraed Dennis Lee Bieber KD6MOG
wlfr...@ix.netcom.com wulfr...@bestiaria.com
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: web-a...@bestiaria.com)
HTTP://www.bestiaria.com/
Dec 16 '07 #2
On Dec 16, 2:34 pm, ecir.h...@gmail.com wrote:
If it is a one-shot (spawn sub, wait, retrieve results) you could
generate a temporary file name in the parent, pass that name to the sub
when invoking it, wait for the sub to complete (giving you a status
code) and then read the result the sub has written to the file.

Yes, it's once shot. But how do I pass "that name"?
From all the arguments of class Popen (http://docs.python.org/lib/
node529.html) perhaps I could use "env" only. Or am I wrong?
Yes. Consider this: If you were to run your calculation script from
the shell prompt [strongly recommended during testing], how would you
tell it the name of the file? Now look at the docs again.
>
PS: both with mmpam and temp file you probably meant that I should
hard code some as-weirdest-filename-as-possible two both programs but
what if I run the computation several times?
That's mmap, not mmpam. No, Dennis didn't mean that you should hard
code a filename. Have a look at the tempfile module.

>
And thanks for reply!
Or, you could have parent and sub both mmap the same "file",
essentially passing data in memory unless it becomes too large and pages
out to swap disk. You might even be able to do bidirectional and dynamic
updates (rather than waiting for the sub to exit)... Define, say, an
epoch count for each process -- these would be the first two words of
the mmap file
|p-epoch|s-epoch|n-words of p-data (fixed constant known to both)|n-words of s-data|
periodically the parent would examine the value of s-epoch, and if it
has changed, read the s-data.. (both sides write the data first, then
update the epoch count). When the epoch stays the same and two
consecutive reads of the data match, you have stable data (so the reads
should occur more often than updates) and can process the data
transferred.
OR, you could have the parent open a TCP/IP socket as a server, and
pass the socket port number to the sub. The sub would then connect to
that port and write the data. For bidirectional you could pass the
parent port, and the sub's first action is to connect and pass a port
that it will be monitoring.
On a VMS system, the processes would connect to named "mailboxes"
and use QIO operations to pass data between them.
On an Amiga you'd use "message ports" (which operated somewhat
similar to VMS mailboxes except that mailboxes had an independent
existence, multiple processes can read or write to them -- message ports
were readable by the creating process, but could have messages sent from
anywhere; typically passing the message port [address of a linked list
of messages] for replies). Or a higher level message port: an ARexx
port.
On a Windows NT class system, the win32 extensions allow access to
Windows Named Pipes... Or maybe the Windows clipboard could be used...
--
Wulfraed Dennis Lee Bieber KD6MOG
wlfr...@ix.netcom.com wulfr...@bestiaria.com
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: web-a...@bestiaria.com)
HTTP://www.bestiaria.com/
Dec 16 '07 #3
On Dec 16, 5:24 am, John Machin <sjmac...@lexicon.netwrote:
>
Yes. Consider this: If you were to run your calculation script from
the shell prompt [strongly recommended during testing], how would you
tell it the name of the file? Now look at the docs again.
File arguments! Of course, totally forgot about them! Thanks a lot!
>
PS: both with mmpam and temp file you probably meant that I should
hard code some as-weirdest-filename-as-possible two both programs but
what if I run the computation several times?

That's mmap, not mmpam. No, Dennis didn't mean that you should hard
code a filename. Have a look at the tempfile module.
Now I recall I read I somewhere that network communication is so much
faster than disk access (especially on the same machine, as steve
howell suggested) so instead of file names I probably should find out
which port is open to use.

Dec 16 '07 #4
On Dec 16, 6:38 am, Dennis Lee Bieber <wlfr...@ix.netcom.comwrote:
>

Read the details for subprocess.Popen() again...

"""
/args/ should be a string, or a sequence of program arguments. The
program to execute is normally the first item in the args sequence or
string, but can be explicitly set by using the executable argument.
"""

IOWs, passing it what you would enter on a command line

"subscript.py tempfilename"
["subscript.py", "tempfilename"]

should be sufficient.
>
There is a module that can generate temporary file names, though for
this usage you could even do something to obtain the parent program
process ID along with a timestamp and create a file name from all that.
What are the odds that your "several times" would have the same clock
time?
Quite small, I guess. However, perhaps I should better consider using
sockets.

Thanks!

ps: I really like how you format the paragraphs! :)
Dec 16 '07 #5
Just for the record:

http://www.amk.ca/python/howto/sockets/

"Of the various forms of IPC (Inter Process Communication), sockets
are by far the most popular. On any given platform, there are likely
to be other forms of IPC that are faster, but for cross-platform
communication, sockets are about the only game in town."
Dec 16 '07 #6
In article
<d9**********************************@e25g2000prg. googlegroups.com>,
ec*******@gmail.com wrote:
Hi,
let's say I have two scripts: one does some computations and the other
one is a graphical front end for launching the first one. And both run
in separate processes (front end runs and that it spawns a subprocess
with the computation). Now, if the computation has a result I would
like to display it in the front end. In another words, I would like to
pass some data from one process to another. How to do that? I'm
affraid I can't use a pipe since the computation could print out some
logging (if I understant pipes correctly).
Others have given you good suggestions; there's also this option which
may or may not be an appropriate tool for what you want to do:
http://NikitaTheSpider.com/python/shm/

--
Philip
http://NikitaTheSpider.com/
Whole-site HTML validation, link checking and more
Dec 22 '07 #7

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Tertius Cronje | last post by:
Hi All, Q: Is it possible for a thread on SocketServer.ThreadingTCPServer to get the socket info of *other* open thread/s and use that info to send data to the accepting client? I wrote a...
1
by: Thierry Marneffe | last post by:
Hello Suppose a database Db1 with tables tl1 and tl2 and a second database db2 with tables tl3 et tl4. Is it possible to make a join between tables of the two databases ? As for example,...
1
by: David M. Karr | last post by:
I've been asked to help debug a complex problem involving inter-frame references, so I just want to understand the elements involved with this. Apparently, there is a page with multiple frames,...
13
by: Bern McCarty | last post by:
I have run an experiment to try to learn some things about floating point performance in managed C++. I am using Visual Studio 2003. I was hoping to get a feel for whether or not it would make...
6
by: les | last post by:
Here's a class which uses 2.0 generics to implement an inter-thread message queue in C#. Any number of threads can post and read from the queue simultaneously, and the message object can be any...
7
by: Sumedh | last post by:
Hi everyone There is a C# project which calls C++/CLI dll to be able to call native C++ including templates. But the C++/CLI code itself also requires the C# dll to get the types. For example: ...
1
by: Laurence | last post by:
Hi folks, As I konw: database partition (aka data partition?), the database can span multiple machines; table partition, the data within a table can seperate by certain condition. How about...
1
by: halekio | last post by:
Hi all, Please bear with me as I've only started programming in C# 2 weeks ago and this is my first contact with OOP. I ran into a situation where I needed to catch an event in an object that...
0
by: rkprasad | last post by:
I am able to create BASIC-CLEAR-INTEGRATED sql http endpoint and consume it on a LAN. But i am not able to consume the created endpoint on inter domain network. If i create endpoint on a server...
1
by: rsennat | last post by:
Hi, what is the order of the libraries in the Makefile, for linking inter dependent libraries. i'm getting linker error for the following scenario. I have lib1.a and lib2.a, with which lib1.a...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.