Python wrapper, problem with subprocess read/write

[NeoGregorian]

Hello, I am writing a wrapper to a basic Input/Output programs (where
you type a one line command at a time and then get 0 or more lines of
output before you can input the next command).

I'm sorry if this problem description is a bit long, but I wanted to
make the problem clear.

Example run of the original program:
C:\home\>start
Starting up program
[Welcome to program v.X.Y.Z]
blahblahblah
and some more lines

>input command

program response...
more program response...
etc.

>another command

....

This is what the wrapper is expected to do...

1: Start up the program.
2: Forward the startup printouts by the program until the line where
it first asks for a command.
3: When waiting for input, input a command from a sequential list of
strings (or other source of strings).
4: Forward the programs printouts until all lines are read and the
program prompts for new command.
5: Repeat 3-4 until list is depleted or program is terminated and then
close the program.
Now, to the problem:
In step 2/4, how to read all lines except the one which is unfinished
(in the example, the lines beginning with >) and waiting for input?

My attempts use something like this:

proc = Popen(['programname'], stdout = PIPE, stdin = PIPE )
for string_element in string_source :
proc.stdin.write(string_element)
lines = proc.stdout.readlines()
method_that_processes_output(lines)

The problem with this is that stdout.readlines() doesn't return since
it reads until EOF...
I tried instead to use:

lines = []
line = proc.stdout.readline()
while line :
lines.append(line)
line = proc.stdout.readline()

This prints out everything except the ">" line, which is good. But
then freezes while waiting for input, which is bad.

Any suggestions on how to solve this in a good way?

[A.T.Hofkamp]

On 2007-09-07, NeoGregorian <ne**********@gmail.comwrote:

I tried instead to use:

lines = []
line = proc.stdout.readline()
while line :
lines.append(line)
line = proc.stdout.readline()

This prints out everything except the ">" line, which is good. But
then freezes while waiting for input, which is bad.

Any suggestions on how to solve this in a good way?

'readline()' reads a line, that is, some text ending with a new-line. Since
your last line, the ">" prompt has no ending new-line, the call blocks, waiting
for the new-line character.

So the simple anser is "don't use readline()".

You have to fall back to reading characters, such as "read(1)" (which block
until it receives a character).
In addition, you will have to do analysis on whether the line you are currently
reading is a prompt, and if so, stop reading to prevent blocking.
(and instead, give the program a command by writing to proc.stdin).
In case you don't know, pexpect (Python expect) does all (and more) that you
are trying to do.
Sincerely,
Albert