472,374 Members | 1,542 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,374 software developers and data experts.

Starting a child process and getting its stdout?

This has been driving me insane for the last hour or so. I have search
everywhere, and nothing works. I am trying to use the subprocess module
to run a program and get its output line by line. But, it always waits
for the process to terminate and then return the output all at once.

Can someone please show me some code that actually works for this sort
of thing? It doesn't even have to use the subprocess module. Don't
worry if the code isn't compatible with Windows. My program is targeted
at Linux/UNIX users.

Thanks!

Dec 29 '06 #1
8 6408
cypher543 wrote:
This has been driving me insane for the last hour or so. I have search
everywhere, and nothing works. I am trying to use the subprocess module
to run a program and get its output line by line. But, it always waits
for the process to terminate and then return the output all at once.

Can someone please show me some code that actually works for this sort
of thing? It doesn't even have to use the subprocess module. Don't
worry if the code isn't compatible with Windows. My program is targeted
at Linux/UNIX users.

Thanks!
try:

Python 2.5c1 (r25c1:51305, Aug 17 2006, 17:07:04)
[GCC 3.3.6] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>import subprocess
cmd = "ls"
process = subprocess.Popen(cmd, stdout=subprocess.PIPE)
print process.stdout.read()
For more info on how to do stdin and other things check out:
http://docs.python.org/lib/module-subprocess.html

Hope this helps.

Adonis
Dec 29 '06 #2
cypher543 wrote:
This has been driving me insane for the last hour or so. I have search
everywhere, and nothing works. I am trying to use the subprocess module
to run a program and get its output line by line. But, it always waits
for the process to terminate and then return the output all at once.
Right, you need to use subprocess.PIPE and polling of the process and
the pipe to get it to do what you want. Hopefully this makes sense, I
also hope it'll work on Linux but can't imagine why it wouldn't.

Save this code to a file, name unimportant. Executing the file will
fire the top clause of the if statement, the bit that runs the "parent"
part. That starts the script again as a subprocess, which triggers the
else clause. Hope it makes sense.

Depending on your environment, you might need 'shell=True' in your Popen
args. On Windows, that is required if you want to prevent a console
window from popping up if you're running in a GUI app.

-tom!

--

import subprocess
import sys
import time

if len(sys.argv) == 1:
# no command line arg means, "we're the parent process."
print 'starting parent process.'
myName = sys.argv[0]

# launch the subprocess with an additional parameter, to trigger
# else clause below.
p = subprocess.Popen(
( 'python', '-u', myName, 'x' ),
stdout=subprocess.PIPE
)

while p.poll() == None:
data = p.stdout.readline()
if data:
print data,

print 'process ended with return code', p.returncode

else:
# assume the command-line arg means "run the background process"
print 'starting subprocess'

for i in range(10):
time.sleep(.25)
print i

sys.exit(14)
Dec 29 '06 #3
Tom Plunket wrote:
while p.poll() == None:
data = p.stdout.readline()
if data:
print data,
If you change that print to something more decorated, like,

print 'process said:', data,

Then it might be more obvious where/how the print statements are getting
routed.

Keep in mind that readline() will return one line at a time, and that
line will end with a newline, although the last line you get (at EOF)
may not have one.

again, good luck. Hope this helps,
-tom!

--
Dec 29 '06 #4
Thank you for the examples, but I have tried all of that before. No
matter what I do, my program always hangs while it waits for the
process to exit and then it prints all of the output at once. I've
tried read(), readline(), readlines(), communicate(), etc and it is
always the same.

self.buildPID = subprocess.Popen(["python", "tobeforked.py"], stdout =
subprocess.PIPE)
while self.buildPID.poll() == None:
output = self.buildPID.stdout.readline()
self.consoleLogBuffer.insert(self.consoleLogBuffer .get_end_iter(),
output)
self.consoleLog.scroll_to_mark(self.consoleLogBuff er.get_insert(), 0)

Keep in mind that I'm not required to use subprocess. But I have also
tried os.fork and the pty module. They both produce the exact same
results.

On Dec 29, 3:38 am, Tom Plunket <t...@fancy.orgwrote:
Tom Plunket wrote:
while p.poll() == None:
data = p.stdout.readline()
if data:
print data,If you change that print to something more decorated, like,

print 'process said:', data,

Then it might be more obvious where/how the print statements are getting
routed.

Keep in mind that readline() will return one line at a time, and that
line will end with a newline, although the last line you get (at EOF)
may not have one.

again, good luck. Hope this helps,
-tom!

--
Dec 29 '06 #5
cypher543 wrote:
Thank you for the examples, but I have tried all of that before.
Did you try my example specifically?
No matter what I do, my program always hangs while it waits for the
process to exit and then it prints all of the output at once.

self.buildPID = subprocess.Popen(["python", "tobeforked.py"], ...
By default, python will execute in buffered mode if it's attached to a
pipe. Start it with the '-u' command line option and you may be good to
go. (`python -h` for more info on this, man pages may also discuss it.)

-tom!

--
Dec 29 '06 #6
At Friday 29/12/2006 12:22, cypher543 wrote:
>Thank you for the examples, but I have tried all of that before. No
matter what I do, my program always hangs while it waits for the
process to exit and then it prints all of the output at once. I've
tried read(), readline(), readlines(), communicate(), etc and it is
always the same.
Did you *actually* tried what Tom Plunket posted? Two tiny chars make
a difference.
--
Gabriel Genellina
Softlab SRL


__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

Dec 30 '06 #7
Gabriel Genellina wrote:
Did you *actually* tried what Tom Plunket posted? Two tiny chars make
a difference.
The sad irony is that before taking off for vacation I was struggling at
work with the same problem in some sense. I couldn't figure out why for
some processes I got all of the output right away and for others it all
got queued up 'til the process ended. Working out a simple example for
this thread showed the light: my calling of my_process.stdout.read() was
blocking 'til EOF. Oops.

-tom!

--
Dec 30 '06 #8
Yes, I did try your example. But, after talking on the #python IRC
channel, I realized that it wasn't the process that was blocking, it
was my GUI. I had to fire an event using gobject.io_add_watch()
whenever data was received from the child process. The event then read
from the process and added a line to my gtk.TextView.

Thank you for your suggestions, though.

On Dec 30, 12:12 am, Tom Plunket <t...@fancy.orgwrote:
Gabriel Genellina wrote:
Did you *actually* tried what Tom Plunket posted? Two tiny chars make
a difference.The sad irony is that before taking off for vacation I was struggling at
work with the same problem in some sense. I couldn't figure out why for
some processes I got all of the output right away and for others it all
got queued up 'til the process ended. Working out a simple example for
this thread showed the light: my calling of my_process.stdout.read() was
blocking 'til EOF. Oops.

-tom!

--
Dec 30 '06 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Fraser Gordon | last post by:
Hello, Hopefully someone can help me out with this issue... I have a python script that needs to run a shell command and be able to react to output from that command as it occurs (as opposed...
1
by: Tung Wai Yip | last post by:
I've build the following utility. It works so far but I want to make sure I'm doing the right thing. I am running Python 2.3 on windows 2000. def execute(cmd): """ execute cmd in sub-process,...
1
by: Dave Sellars | last post by:
Is there really no way to run a sub-process, gather its stdout/stderr, and collect the return-code, on Win32??? But that's what the docs say... > These methods do not make it possible to...
1
by: Mike Finister | last post by:
Hi there At the moment I am writing a GUI front-end that in the background is to call some scripts on a Unix box. The GUI has to be written in Visual Basic (yes I said the VB word...sorry! :-) )...
0
by: Jonathan Dodds | last post by:
I need to create a child process and read the child's stdout and stderr in the parent process. This seems to be surprisingly difficult to do in .NET. The documentation for the...
1
by: SG | last post by:
Hello, I'm using _execvpe (Win32 console project in VS.NET) to spawn a child process. I would like to know if you have any pointers on how to capture the child's STDOUT and STDERR? Any help is...
7
by: newgoat | last post by:
I just wrote a short program to see process switch when sleep() is invoked within a process. The code is as follows: #include<stdio.h> #include<unistd.h> #include<sys/types.h>...
0
by: danebarney | last post by:
I'm developing a Windows CE 5.0 app using C# and I need to call a child process and then capture its stdout into a window. The solutions I've read online for doing this involve either using the...
2
by: Greg Ercolano | last post by:
When I use os.popen(cmd,'w'), I find that under windows, the stdout of the child process disappears, instead of appearing in the DOS window the script is invoked from. eg: C:\type foo.py import...
2
by: Kemmylinns12 | last post by:
Blockchain technology has emerged as a transformative force in the business world, offering unprecedented opportunities for innovation and efficiency. While initially associated with cryptocurrencies...
0
by: Naresh1 | last post by:
What is WebLogic Admin Training? WebLogic Admin Training is a specialized program designed to equip individuals with the skills and knowledge required to effectively administer and manage Oracle...
0
by: antdb | last post by:
Ⅰ. Advantage of AntDB: hyper-convergence + streaming processing engine In the overall architecture, a new "hyper-convergence" concept was proposed, which integrated multiple engines and...
1
by: Matthew3360 | last post by:
Hi, I have been trying to connect to a local host using php curl. But I am finding it hard to do this. I am doing the curl get request from my web server and have made sure to enable curl. I get a...
0
Oralloy
by: Oralloy | last post by:
Hello Folks, I am trying to hook up a CPU which I designed using SystemC to I/O pins on an FPGA. My problem (spelled failure) is with the synthesis of my design into a bitstream, not the C++...
0
by: Rahul1995seven | last post by:
Introduction: In the realm of programming languages, Python has emerged as a powerhouse. With its simplicity, versatility, and robustness, Python has gained popularity among beginners and experts...
1
by: ezappsrUS | last post by:
Hi, I wonder if someone knows where I am going wrong below. I have a continuous form and two labels where only one would be visible depending on the checkbox being checked or not. Below is the...
0
by: jack2019x | last post by:
hello, Is there code or static lib for hook swapchain present? I wanna hook dxgi swapchain present for dx11 and dx9.
0
DizelArs
by: DizelArs | last post by:
Hi all) Faced with a problem, element.click() event doesn't work in Safari browser. Tried various tricks like emulating touch event through a function: let clickEvent = new Event('click', {...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.