Python Interactive Shell - outputting to stdout?

[Avi Berkovich]

Hello,

I was unable to use popen2.popen4 to grab python.exe's (2.3) output, for
starts, it doesn't show the version information at the beginning and
won't return anything when writing to the stdin pipe, it seems that if I
give it some error nous expression, the pipe would return the exception
data, though nothing else comes through.

A friend of mine also tried this using win32api on delphi, and got the
same result.

I also tried "python.exe > data.txt" and was amazed to find out that it
won't redirect the output to the file.

So, may someone please shed some light over this?

Avi.

[Steve Holden]

Avi Berkovich wrote:

Hello,

I was unable to use popen2.popen4 to grab python.exe's (2.3) output, for
starts, it doesn't show the version information at the beginning and
won't return anything when writing to the stdin pipe, it seems that if I
give it some error nous expression, the pipe would return the exception
data, though nothing else comes through.

A friend of mine also tried this using win32api on delphi, and got the
same result.

I also tried "python.exe > data.txt" and was amazed to find out that it
won't redirect the output to the file.

So, may someone please shed some light over this?

Avi.

Well, your first misunderstanding appears to be the omission of the
standard error file. Interactive messages and prompts are written to
stderr, output to stdout:

C:\Steve>python.exe > python.txt
Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.

123
345
^Z

C:\Steve>type python.txt
123
345

When you try to write a program that communicates with a client that's
intended for interactive use (such as the Python interpreter when called
with no arguments), you have the problem of divining exactly when the
client is waiting for input.

Rather than blame the Python interpreter, first write a program that
interacts with (say) the Windows command-line interpreter.

If your experience is substantially different then it *might* be a
problem with the interpreter. If your experience is more or less the
same then the problem is probably to do with the nature of the
interactions, which are bad enough when only considering stdin and
stdout. When you add the stderr stream into the picture it's sometimes
impossible to know when the interactive client is waiting for input.

One final comment: you haven't so far said *why* you want to interact
with Python in this way. Given the availability of exec and eval(), just
what is it that you are trying to do that they won't let you?

regards
Steve
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/
Holden Web LLC [phone] [phone]

[Avi Berkovich]

Hey Steve,

I did write a program to deal with the windows command interpreter, and
it works.

I don't need to do this, but a friend of mine needed to issue commands
to the interpreter via pipes from a non python program, and he has a
fully functional component for using pipes, and has done it with other
interpreters before.

I guess I'll have to separate the std's and continue from there.

Thanks for your input.

Avi.

[Avi Berkovich]

Hey,

I can't make it work, I don't get any data from either stdout nor stderr.
If I send lines and then close the stdin pipe, I may get an exception
message from several lines up.

I tried manually reading from the stdout pipe, but it just blocks and
hangs no matter what I send over via the stdin pipe.

This behavior isn't presented by the command line interpreter by any chance.

Any suggestions?

[Steve Holden]

Avi Berkovich wrote:

Hey,

I can't make it work, I don't get any data from either stdout nor stderr.
If I send lines and then close the stdin pipe, I may get an exception
message from several lines up.

I tried manually reading from the stdout pipe, but it just blocks and
hangs no matter what I send over via the stdin pipe.

This behavior isn't presented by the command line interpreter by any
chance.

Any suggestions?

Yes: post your code along with messages (if any)!

regards
Steve
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/
Holden Web LLC [phone] [phone]

[Steve Holden]

Steve Holden wrote:

Avi Berkovich wrote:

Hey,

I can't make it work, I don't get any data from either stdout nor stderr.
If I send lines and then close the stdin pipe, I may get an exception
message from several lines up.

I tried manually reading from the stdout pipe, but it just blocks and
hangs no matter what I send over via the stdin pipe.

This behavior isn't presented by the command line interpreter by any
chance.

Any suggestions?

Yes: post your code along with messages (if any)!

regards
Steve

Sorry, for some reason I can no longer find the message you posted the
code in -- it didn't get correctly threaded in Mozilla. Anyway, I can't
say that I exactly understand what's going on, but here are a couple of
observations:

1. The interpreter performs buffering even when running in interactive
mode unless it can see a terminal (which here it can't). Hence adding a
"-u" to the interpreter command line is useful.

2. There's a problem with your stop condition, which is why the attached
modified version sleeps before closing the pipe to the interpreter.

import threading
import sys
import popen2

class Piper(threading.Thread):

def __init__(self, readPipe, output):
threading.Thread.__init__(self)

if not isinstance(readPipe, file):
raise TypeError, "readPipe parameter must be of File type"
#if not isinstance(output, file):
# raise TypeError, "output parameter must be of File type"

self.readPipe = readPipe
self.output = output
self.toStop = False

def run(self):
print "Running"
while not self.toStop and not self.readPipe.closed:
read = self.readPipe.readline()
self.output.write(read)
self.output.flush()
print "Stopped"

def stop(self):
self.toStop = True
if __name__ == "__main__":
r, w = popen2.popen4('c:\\python24\\python.exe -u')
piper = Piper(r, sys.stdout)
piper.start()
w.write("print 'Hello!'\r\n")
w.flush()
w.write("print 'Goodbye!'\r\n")
w.flush()
import time; time.sleep(2)
w.close()
#import time; time.sleep(3)
#r.close()
piper.stop()

Various other bits and pieces of messing about didn't really yield any
useful conclusions, so I pass this on only in the hope that you may be
more motivated to follow up now you can actually see some interpreter
output. I would have thought the flush() calls would have made output
appear one line at a time, but sadly they do not.

regards
Steve
--
Steve Holden http://www.holdenweb.com/
Python Web Programming http://pydish.holdenweb.com/
Holden Web LLC [phone] [phone]

[Avi Berkovich]

Hey Steve,

Well, I've tried flush() before, but I didn't know about the "-u" switch.

Thank you for tinkering on this, I shall post again if I make any progress.

Avi
Steve Holden wrote:

Steve Holden wrote:

Avi Berkovich wrote:

Hey,

I can't make it work, I don't get any data from either stdout nor
stderr.
If I send lines and then close the stdin pipe, I may get an exception
message from several lines up.

I tried manually reading from the stdout pipe, but it just blocks and
hangs no matter what I send over via the stdin pipe.

This behavior isn't presented by the command line interpreter by any
chance.

Any suggestions?

Yes: post your code along with messages (if any)!

regards
Steve

Sorry, for some reason I can no longer find the message you posted the
code in -- it didn't get correctly threaded in Mozilla. Anyway, I can't
say that I exactly understand what's going on, but here are a couple of
observations:

1. The interpreter performs buffering even when running in interactive
mode unless it can see a terminal (which here it can't). Hence adding a
"-u" to the interpreter command line is useful.

2. There's a problem with your stop condition, which is why the attached
modified version sleeps before closing the pipe to the interpreter.

import threading
import sys
import popen2

class Piper(threading.Thread):

def __init__(self, readPipe, output):
threading.Thread.__init__(self)

if not isinstance(readPipe, file):
raise TypeError, "readPipe parameter must be of File type"
#if not isinstance(output, file):
# raise TypeError, "output parameter must be of File type"

self.readPipe = readPipe
self.output = output
self.toStop = False

def run(self):
print "Running"
while not self.toStop and not self.readPipe.closed:
read = self.readPipe.readline()
self.output.write(read)
self.output.flush()
print "Stopped"

def stop(self):
self.toStop = True
if __name__ == "__main__":
r, w = popen2.popen4('c:\\python24\\python.exe -u')
piper = Piper(r, sys.stdout)
piper.start()
w.write("print 'Hello!'\r\n")
w.flush()
w.write("print 'Goodbye!'\r\n")
w.flush()
import time; time.sleep(2)
w.close()
#import time; time.sleep(3)
#r.close()
piper.stop()

Various other bits and pieces of messing about didn't really yield any
useful conclusions, so I pass this on only in the hope that you may be
more motivated to follow up now you can actually see some interpreter
output. I would have thought the flush() calls would have made output
appear one line at a time, but sadly they do not.

regards
Steve