473,376 Members | 1,029 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,376 software developers and data experts.

Python wrapper, problem with subprocess read/write

Hello, I am writing a wrapper to a basic Input/Output programs (where
you type a one line command at a time and then get 0 or more lines of
output before you can input the next command).

I'm sorry if this problem description is a bit long, but I wanted to
make the problem clear.

Example run of the original program:
C:\home\>start
Starting up program
[Welcome to program v.X.Y.Z]
blahblahblah
and some more lines
>input command
program response...
more program response...
etc.
>another command
....

This is what the wrapper is expected to do...

1: Start up the program.
2: Forward the startup printouts by the program until the line where
it first asks for a command.
3: When waiting for input, input a command from a sequential list of
strings (or other source of strings).
4: Forward the programs printouts until all lines are read and the
program prompts for new command.
5: Repeat 3-4 until list is depleted or program is terminated and then
close the program.
Now, to the problem:
In step 2/4, how to read all lines except the one which is unfinished
(in the example, the lines beginning with >) and waiting for input?

My attempts use something like this:

proc = Popen(['programname'], stdout = PIPE, stdin = PIPE )
for string_element in string_source :
proc.stdin.write(string_element)
lines = proc.stdout.readlines()
method_that_processes_output(lines)

The problem with this is that stdout.readlines() doesn't return since
it reads until EOF...
I tried instead to use:

lines = []
line = proc.stdout.readline()
while line :
lines.append(line)
line = proc.stdout.readline()

This prints out everything except the ">" line, which is good. But
then freezes while waiting for input, which is bad.

Any suggestions on how to solve this in a good way?

Sep 7 '07 #1
1 2422
On 2007-09-07, NeoGregorian <ne**********@gmail.comwrote:
I tried instead to use:

lines = []
line = proc.stdout.readline()
while line :
lines.append(line)
line = proc.stdout.readline()

This prints out everything except the ">" line, which is good. But
then freezes while waiting for input, which is bad.

Any suggestions on how to solve this in a good way?
'readline()' reads a line, that is, some text ending with a new-line. Since
your last line, the ">" prompt has no ending new-line, the call blocks, waiting
for the new-line character.

So the simple anser is "don't use readline()".

You have to fall back to reading characters, such as "read(1)" (which block
until it receives a character).
In addition, you will have to do analysis on whether the line you are currently
reading is a prompt, and if so, stop reading to prevent blocking.
(and instead, give the program a command by writing to proc.stdin).
In case you don't know, pexpect (Python expect) does all (and more) that you
are trying to do.
Sincerely,
Albert
Sep 10 '07 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Kurt B. Kaiser | last post by:
Patch / Bug Summary ___________________ Patches : 378 open ( +3) / 3298 closed (+34) / 3676 total (+37) Bugs : 886 open (-24) / 5926 closed (+75) / 6812 total (+51) RFE : 224 open...
1
by: Calvin Spealman | last post by:
No matter what I do I cant get the following code to do what I expect. I hadn't used subprocess t o read and write to pipes of a still-running app, and I just can't seem to get it right. What...
7
by: Dave Sampson | last post by:
hey folks, A simple question hopefully. despite all my searching I have not found a satisfactory response. The goal. Interact with a command line program. Simple enough, but the key is...
23
by: Harishankar | last post by:
Hi, Sorry to start off on a negative note in the list, but I feel that the Python subprocess module is sorely deficient because it lacks a mechanism to: 1. Create non-blocking pipes which can...
1
by: Michael Torrie | last post by:
Recently a post that mentioned a recipe that extended subprocess to allow killable processes caused me to do some thinking. Some of my larger bash scripts are starting to become a bit unwieldy...
4
by: Aidan | last post by:
Hi, I'm having a bit of trouble with a python script I wrote, though I'm not sure if it's related directly to python, or one of the other software packages... The situation is that I'm trying...
0
by: Gabriel Genellina | last post by:
En Thu, 25 Sep 2008 09:49:31 -0300, Almar Klein <almar.klein@gmail.com> escribió: Use subprocess.PIPE Usually the tricky part is to figure out exactly whether there is more input or not. With...
1
by: replysonika | last post by:
Hello, I run a Java app with subprocess from Python script. This python script is called from another Python Wrapper. python = subprocess.Popen(, stdout=subprocess.PIPE,...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.