473,322 Members | 1,494 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

Capturing stdout without waiting for the process end

Hi to all!

I'd like to execute an external program capturing the stdout/stderr
messages at "real-time". I mean that I don't want to wait for the end
of the process. If I write a code like this:

import os
import sys

class Runner:

def run(self, arg):
try:
fin, fout = os.popen4(arg)
self.outData = fout.readlines()
self.outStatus = fout.close()

except Exception, err:
print err

def printOut(self):
print "###"
print self.outStatus
print "###"
for line in self.outData:
print line
r = Runner()
r.run("ls /tmp")
r.printOut()

I can print out (in this case in the os.stdout, but it may be
elsewhere) the whole external program output only once it ends. How can
I do to intercept the external program output during the processing?

Thank you in advance

Luigi

Apr 3 '06 #1
5 2188
What OS are you doing this on?

I had an issue similar to this and it was due to default buffering
behavior of my tty's. If I recall correctly I executed a bunch of
settty to eliminate the buffering behavior of stdout. The set the
terminal back to it's original setting when my program was done.

Anand

Apr 3 '06 #2
I use Linux.

So, if I modify my sdtout behaviour, when I execute:

fin, fout = os.popen4(arg)

this is executed asyncronously? And how can I intercept a fout.write()
event?

The question is that I have a C program (by third part) that streams
the logs into the stderr and stdout devices. I need to create an
envelopment that captures the outputs and puts them in a file
generating log events (for a real-time view).

Thanks

Luigi

Apr 3 '06 #3
> fin, fout = os.popen4(arg)

this is executed asyncronously? And how can I intercept a fout.write()
event?


You don't intercept anything, you read from the child process stdin, which
from your POV is a readable stream.

Diez
Apr 3 '06 #4
Diez is correct, the C program is writting to stdout, you are reading
from stdout. Default bahavior in most modern Unix like systems is to
buffer std out. I stumbled on this a long time ago, so I am trying to
remember the details.

What I think is happening here, you call the child process, it does
it's thing and puts thing into stdout.

How long does the child process run typically?

What also is the actual behavior? How is it wrong, exactly? Are you
getting the full output from the C program? Part of it? I still
suspect a buffering problem of some sort, but I am not clear on what is
going wrong.

Apr 3 '06 #5
In article <11**********************@z34g2000cwc.googlegroups .com>,
"Luigi" <lu*********@libero.it> wrote:
The question is that I have a C program (by third part) that streams
the logs into the stderr and stdout devices. I need to create an
envelopment that captures the outputs and puts them in a file
generating log events (for a real-time view).


As suggested in another followup, the C program's
output will probably be "block buffered" when its
output is a pipe. In this case, you'll get output
only when the buffer is full, or when the program
exits.

If the program can be modified, it only needs to
flush stdout after each output event. Otherwise,
you need to use a device that looks like a tty,
just so the C stdio library will switch to line
buffering as it generally does with terminals.

This is called a pseudotty. It's a little more
difficult to use than a pipe, but you can probably
get something going with openpty or forkpty from
the os/posix module, or there may still be 3rd
party packages for this.

Donn Cave, do**@u.washington.edu
Apr 3 '06 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
by: Moosebumps | last post by:
I have a large set of Python scripts that interface with command line utilities (primarily Perforce). I am currently capturing ALL the text output in order to get results and such. I am using the...
2
by: Graham Ashton | last post by:
Hi. I'm having trouble flushing sys.stdout. I've written a small example to illustrate my problem (see below). In short, I expect it to ping "received hello", sleep for 5 seconds and then print...
1
by: Mike Finister | last post by:
Hi there At the moment I am writing a GUI front-end that in the background is to call some scripts on a Unix box. The GUI has to be written in Visual Basic (yes I said the VB word...sorry! :-) )...
1
by: Oz | last post by:
This is long. Bear with me, as I will really go through all the convoluted stuff that shows there is a problem with streams (at least when used to redirect stdout). The basic idea is that my...
2
by: sergio | last post by:
i have a huge database that contains large amounts of html that i need to translate to ascii.. i have tried using html2text.py: http://www.aaronsw.com/2002/html2text/ but i could not figure...
0
by: Christoph Haas | last post by:
Evening, I'm having trouble with running a process through Python 2.4's subprocess module. Example code: ======================================================== def run(command): run =...
6
by: Ed Leafe | last post by:
I've been approached by a local business that has been advised that they need to start capturing and archiving their instant messaging in order to comply with Sarbanes-Oxley. The company is largely...
0
by: kreismaler | last post by:
I have some problems to understand the difference of using the STDOUT and using "anonymous pipes" as shown below: using System; using System.Diagnostics; using System.IO; namespace...
1
by: James McGill | last post by:
Hi All, I'm using subprocess.Popen to run a C++ compiler and have set stdout = PIPE. The exact line of code that I am using is: process = Popen(command, stdout=PIPE) status = process.wait() ...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.