Inter-process communication, how? Part 2

[ecir.hana]

Hello,

just to recap: last time I asked how to do an interprocess
communitation, between one Manager process (graphical beckend) and
some Worker processes.

I decided to go with sockets, thanks for replies, once more.

However, I would like to ask another thing: I would like to collect
everyting what the Workers print and display in Manager. Or, redirect
all Workers' stdout to stdio of Manager. If there was only one Worker
I could use a pipe, right? But if there are more than one Worker, what
to do? I found something called "named pipe" which seems rather
complicated. Then I thought I could somehow (how?) create a fake
(virtual) file object, redirect stdout of a Worket into it and from
there send the data to Manager via sockets. Please, what do you think?

Preferably, it should look like this:

--- Worker 1 ---
....some code...
print '123'

--- Manager ---
Worker 1: 123

--- Worker 2 ---
....some code...
print '456'

--- Manager ---
Worker 1: 123
Worker 2: 456

Thanks in advance!

[Guilherme Polo]

2007/12/22, ec*******@gmail.com <ec*******@gmail.com>:

Hello,

just to recap: last time I asked how to do an interprocess
communitation, between one Manager process (graphical beckend) and
some Worker processes.

I decided to go with sockets, thanks for replies, once more.

However, I would like to ask another thing: I would like to collect
everyting what the Workers print and display in Manager. Or, redirect
all Workers' stdout to stdio of Manager. If there was only one Worker
I could use a pipe, right? But if there are more than one Worker, what
to do? I found something called "named pipe" which seems rather
complicated.

Named pipe is called FIFO, but they are not that complicated.

Then I thought I could somehow (how?) create a fake
(virtual) file object,

That is why it is called named pipe, because you will be using a
(especial) file in your filesystem to use as a pipe.

redirect stdout of a Worket into it and from
there send the data to Manager via sockets. Please, what do you think?

Preferably, it should look like this:

--- Worker 1 ---
...some code...
print '123'

--- Manager ---
Worker 1: 123

--- Worker 2 ---
...some code...
print '456'

--- Manager ---
Worker 1: 123
Worker 2: 456

Thanks in advance!
--
http://mail.python.org/mailman/listinfo/python-list

Manager would create and open the FIFO, Workers would open this same
FIFO. So workers write to FIFO and the manager reads from FIFO.

--
-- Guilherme H. Polo Goncalves

[ecir.hana]

On Dec 23, 4:54 am, Dennis Lee Bieber <[e-mail]:

On Sat, 22 Dec 2007 17:56:05 -0800 (PST), [e-mail] declaimed
the following in comp.lang.python:

just to recap: last time I asked how to do an interprocess
communitation, between one Manager process (graphical beckend) and
some Worker processes.

Never considered a graphical control process as a "backend"
before... Backend processing, to me, implies some sort of server without
a user interface.

However, I would like to ask another thing: I would like to collect
everyting what the Workers print and display in Manager. Or, redirect
all Workers' stdout to stdio of Manager. If there was only one Worker
I could use a pipe, right? But if there are more than one Worker, what
to do? I found something called "named pipe" which seems rather
complicated. Then I thought I could somehow (how?) create a fake
(virtual) file object, redirect stdout of a Worket into it and from
there send the data to Manager via sockets. Please, what do you think?

I'd forget about stdout as a data communication means... The parent
should probably set up a socket that accepts messages from any worker...
or create a "reply" socket for each worker, and pass the worker the port
on which the master expects to retrieve its output.

Ok, but how to redirect "print" statement into a socket?

>
"Named pipes" are, I think, a M$ Windows creation (though I think
the Amiga supported disjoint pipes by using "run program >pipe:name" and
"program <pipe:name" instead of "program | program"http://stason.org/TULARC/pc/amiga/faq/2-5-1-Using-PIPE-in-a-standard-...
-- "run program" being ~ "program &" in most UNIX-based shells)
--
Wulfraed Dennis Lee Bieber KD6MOG
[e-mail] [e-mail]
HTTP://wlfraed.home.netcom.com/
(Bestiaria Support Staff: [e-mail])
HTTP://www.bestiaria.com/

[ecir.hana]

On Dec 23, 10:30 am, "Guilherme Polo" <[e-mail]:

2007/12/22, [e-mail] <[e-mail]>:

Hello,

just to recap: last time I asked how to do an interprocess
communitation, between one Manager process (graphical beckend) and
some Worker processes.

I decided to go with sockets, thanks for replies, once more.

However, I would like to ask another thing: I would like to collect
everyting what the Workers print and display in Manager. Or, redirect
all Workers' stdout to stdio of Manager. If there was only one Worker
I could use a pipe, right? But if there are more than one Worker, what
to do? I found something called "named pipe" which seems rather
complicated.

Named pipe is called FIFO, but they are not that complicated.

Then I thought I could somehow (how?) create a fake
(virtual) file object,

That is why it is called named pipe, because you will be using a
(especial) file in your filesystem to use as a pipe.

redirect stdout of a Worket into it and from
there send the data to Manager via sockets. Please, what do you think?

Preferably, it should look like this:

--- Worker 1 ---
...some code...
print '123'

--- Manager ---
Worker 1: 123

--- Worker 2 ---
...some code...
print '456'

--- Manager ---
Worker 1: 123
Worker 2: 456

Thanks in advance!
--
http://mail.python.org/mailman/listinfo/python-list

Manager would create and open the FIFO, Workers would open this same
FIFO. So workers write to FIFO and the manager reads from FIFO.

--
-- Guilherme H. Polo Goncalves

What I don't like about FIFO, is that on Unix they are persistent
files. So whatever happens to Manager they would stay there...
I was just wondering if there's another way of doing the above and if
not, I would probably go with FIFO. Thanks!

[Bjoern Schliessmann]

ec*******@gmail.com wrote:

What I don't like about FIFO, is that on Unix they are persistent
files. So whatever happens to Manager they would stay there...

Nope. /tmp exists. Many distributions delete /tmp contents on
reboot.

I was just wondering if there's another way of doing the above and
if not, I would probably go with FIFO. Thanks!

Didn't you write you went with sockets?

Regards,
Björn

--
BOFH excuse #354:

Chewing gum on /dev/sd3c

[Bjoern Schliessmann]

ec*******@gmail.com wrote:

Ok, but how to redirect "print" statement into a socket?

Create an object that has the socket's send-or-what-it's-called
method as a member called "write" and bind this object to
sys.stdout. Alternatively, you can use the "print "text" >object"
syntax (see language reference).

Regards,
Björn

--
BOFH excuse #116:

the real ttys became pseudo ttys and vice-versa.

[John Nagle]

ec*******@gmail.com wrote:

Hello,

just to recap: last time I asked how to do an interprocess
communitation, between one Manager process (graphical beckend) and
some Worker processes.

I decided to go with sockets, thanks for replies, once more.

However, I would like to ask another thing: I would like to collect
everyting what the Workers print and display in Manager. Or, redirect
all Workers' stdout to stdio of Manager. If there was only one Worker
I could use a pipe, right? But if there are more than one Worker, what
to do? I found something called "named pipe" which seems rather
complicated. Then I thought I could somehow (how?) create a fake
(virtual) file object, redirect stdout of a Worket into it and from
there send the data to Manager via sockets. Please, what do you think?

Take a look at

http://aspn.activestate.com/ASPN/Coo...n/Recipe/52296

which does this. I wouldn'd do it quite that way; read the notes
on the article for some criticism. But it's an approach that works.

One can get fancier. I wrote a system under QNX where each
real time process had its text output labeled, timestamped, and transmitted
using QNX interprocess communication to a lower priority logging process
on a different machine. Text output never blocked; if a queue
filled, "..." appeared in the log file. That's a special real-time situation.

John Nagle

[makkalot]

Sunday 23 December 2007 Tarihinde 03:56:05 yazmıştı:

Hello,

just to recap: last time I asked how to do an interprocess
communitation, between one Manager process (graphical beckend) and
some Worker processes.

Why just dont use dbus , and then have a drink :)