Debugging pipe IPC

[Jim B. Wilson]

I have the mother (of all) application(s, written in C++) that
occasionally outsources certain tasks to a child Python script. The
mother fork/execs (or equivalent) the child and then begins serving the
child's requests.

The child/client sends requests on its stdout and receives responses on
stdin. The interaction is facilitated by a Pyrex extension which handles
the lower-level aspects of the conversation and exposes a few functions
and classes to the script.

This part works peachy keen, but debugging has so far been via
print>>stderr-and-scratch-head.

On a contemplative bike ride one day, I imagined how neat it would be to
run interactively. On returning, I began to experiment with rebinding
sys.stdin and sys.stdout to /dev/tty, using the "-i" command-line switch,
etc. to see if I could devise a child that prompted me with ">>>" and
allowed me to compose/test small snippets on the terminal. So far, no
joy.

Is this possible? If so, can someone nudge me toward a solution or
better yet a recipe?

Jim Wilson
Gainesville, FL

[Ian Clark]

Jim B. Wilson wrote:

I have the mother (of all) application(s, written in C++) that
occasionally outsources certain tasks to a child Python script. The
mother fork/execs (or equivalent) the child and then begins serving the
child's requests.

The child/client sends requests on its stdout and receives responses on
stdin. The interaction is facilitated by a Pyrex extension which handles
the lower-level aspects of the conversation and exposes a few functions
and classes to the script.

This part works peachy keen, but debugging has so far been via
print>>stderr-and-scratch-head.

On a contemplative bike ride one day, I imagined how neat it would be to
run interactively. On returning, I began to experiment with rebinding
sys.stdin and sys.stdout to /dev/tty, using the "-i" command-line switch,
etc. to see if I could devise a child that prompted me with ">>>" and
allowed me to compose/test small snippets on the terminal. So far, no
joy.

Is this possible? If so, can someone nudge me toward a solution or
better yet a recipe?

Jim Wilson
Gainesville, FL

You're looking for the cmd module.
<http://docs.python.org/lib/module-cmd.html>

Ian Clark

[Jim B. Wilson]

Ian Clark pointed me to:

... the cmd module.
<http://docs.python.org/lib/module-cmd.html>

Yes, I found that, but I could only get it to print a nice interactive
prompt, "(Cmd)", read a line of input and discard it. Apparently, I'm
too stupid to figure out how to hook it into python.

Jim

[Ian Clark]

Jim B. Wilson wrote:
....

The child/client sends requests on its stdout and receives responses on
stdin.

So, why can't you just run this client on the command line and let the
shell handle stdin/stdout for you?

Ian Clark

[Jim B. Wilson]

Ian Clark wrote:

Jim B. Wilson wrote:
...

>The child/client sends requests on its stdout and receives responses
on stdin.

So, why can't you just run this client on the command line and let the
shell handle stdin/stdout for you?

I'm not sure I understand the topology of your proposal? And, it's
certainly possible the shell has mystical powers of which I am unaware.

Are you suggesting a run client that spews the mother's messages to the
terminal, and I somehow cut/paste them into the client, running with
stdin/stdout unconnected to anything but me? Or is there some
combination of tee, etal., I can lash together so that my commands (to
the client) don't get mixed up with mother's messages?

Some such arrangement would surely help. I often forget the ">>stderr,"
on my scratch-head debugging prints, and mother gets quite upset when
the incoming message doesn't fit the mold :)

Jim

[Ian Clark]

Jim B. Wilson wrote:

Ian Clark wrote:

>Jim B. Wilson wrote:
...

>>The child/client sends requests on its stdout and receives responses
on stdin.

So, why can't you just run this client on the command line and let the
shell handle stdin/stdout for you?

I'm not sure I understand the topology of your proposal? And, it's
certainly possible the shell has mystical powers of which I am unaware.

Are you suggesting a run client that spews the mother's messages to the
terminal, and I somehow cut/paste them into the client, running with
stdin/stdout unconnected to anything but me? Or is there some
combination of tee, etal., I can lash together so that my commands (to
the client) don't get mixed up with mother's messages?

Some such arrangement would surely help. I often forget the ">>stderr,"
on my scratch-head debugging prints, and mother gets quite upset when
the incoming message doesn't fit the mold :)

Jim

What I'm suggesting is to run the following from the shell: python
internal_python_program.py. Then you type into the terminal whatever
'mother' was sending it ("Clean your room!" most like) and see what it
spits back. If the script is simply reading from stdin and writing to
stdout there shouldn't be much of a problem. The hardest part might be
figuring out what 'mother' actually sends it.

If it's very verbose you can type it out once to a file and redirect
that file to the python process: python internal_python_program.py <
file_with_mother_commands.txt.

Another idea is to open a file somewhere and write your debug output to
that. I would also suggest the logging[1] module as well. It's very
handy for his sort of thing.

Ian

[1] http://docs.python.org/lib/module-logging.html

[Jim B. Wilson]

Ian Clark wrote:

... whatever 'mother' was sending it ("Clean your room!" most like)

:)

If it's very verbose ...

Alas, it is quite verbose. Constructing a single instance of a class
(from the Pyrex extension acting as the child's two-way radio) could
involve tens of thousands of more-or-less random exchanges between
mother and child. A subsequent method call could trigger a similar
volume of traffic.

And it isn't this communication I'm worried about. All that seems to go
quite well. My fondest wish is to play the role of the child at the
good old ">>>" prompt. Such play would allow me to test tiny snippets
of code that I could later incorporate into an actual child.

My wish may not be possible. If so, it's no great tragedy. A simple
test goes from:

>>foo = askmommy()
foo.why()

to:

1. Typing similar statements into my editor.
2. Saving these statements to a file, say "test.py".
3. Making clicky-clicky in the mother application to run "test.py".
4. Noticing that nothing appears on my terminal.
5. Inserting ">>sys.stderr," into "print foo.why()" :)
6. Repeating steps 2 and 3.

Steps 4, 5 and 6 are required because the mother has absolutely no
compunction against infanticide when her child asks her a question she
cannot answer.

If my wish isn't possible, it will be the first time. Normally, when I
wonder if Python can do something, I imagine what the syntax would be if
it could, type it in, and sure enough it works.

Jim

[Ian Clark]

Jim B. Wilson wrote:

... My fondest wish is to play the role of the child at the
good old ">>>" prompt. ...

Jim

Okay, I misunderstood the direction you wanted to go. I thought that you
wanted to play the part of the mother, giving commands to the child and
not the other way around.

Assuming you're on a *nix make child.py as:

import os
os.system("netcat -l -p 1234 localhost")

Run it though mother and on another terminal type: netcat localhost 1234

HTH,

Ian

[Jim B. Wilson]

Ian Clark wrote:

import os
os.system("netcat -l -p 1234 localhost")

HTH,

Nope, but the network theme got me thinking about how one might run
Python on a remote host. After a few false starts, Googling "remote
python shell" led me to Guido's "ripshell.py" (not *that* Guido, a
different one). Peeking inside, I discovered the module I'm looking for
was "code". Remember, in my case, communication on stdin and stdout
was all handled by a Pyrex extension. I'm unsure the code below will
work in a pure Python application.

For posterity, here is itest.py, the solution to my problem:

### itest.py - allows interactive debugging of a "filter"
#
# If stdin and stdout are dedicated to your filter script, but you
# want to interactively check a few things, try something like this:

from code import interact
import sys

sys.stdout = open("/dev/tty", "w") # Rebind Python's lips
sys.stdin = open("/dev/tty", "r") # and Python's ears.

foo = "This is a test" # Lay down some history
interact( # Convenience function from code.
"At your service, Sir!", # BUG: could be "Madam" or "Miss"
local = locals()) # Teach interpreter some history
print "Thank you, Sir!" # BUG: ibid.

To check this works:

$ python itest.py </dev/null >/dev/full # mother is a deafmute
At your service, Sir!

>>foo

'This is a test'

>>^D (not shown)

Thank you, Sir!
$