By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
439,941 Members | 1,753 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 439,941 IT Pros & Developers. It's quick & easy.

Capturing stderr and stdout of a subprocess as a single stream

P: n/a
Hello all,

Before I ask the question a couple of notes :

* This question is for implementing a script inside the Wing IDE. For
some reason using the subprocess module doesn't work so I need a
solution that doesn't use this module.
* The platform is Windows and I'm happy with a Windoze only solution.
:-)

I would like to execute subprocesses asynchronously and capture stdout
/ stderr as a single stream.

If I use "os.popen3(executable)" it gives me separate pipes for stdout
and stderr, but reads from them are blocking.

The output on stdout and stderr may be interleaved and I would like to
display them *as* they arrive. That means I can't just read from them
and output the results.

The only solution I can think of is to read from both a character at a
time on two separate threads, putting the data into a queue. A separate
thread could pull characters off the queue and display them. (I don't
need to differentiate between stdout and stderr when I display.)

Can anyone think of a better solution ?

My current code works, but *doesn't* capture stderr :

from threading import Thread

pipe = os.popen(executable)

def DisplayOutput():
while True:
output = pipe.read(1)
if not output:
break
display(output)

Thread(target=DisplayOutput).start()

All the best,
Fuzzyman
http://www.voidspace.org.uk/python/articles.shtml

Jan 7 '07 #1
Share this Question
Share on Google+
3 Replies


P: n/a


On 7 ene, 16:33, "Fuzzyman" <fuzzy...@gmail.comwrote:
Hello all,

Before I ask the question a couple of notes :

* This question is for implementing a script inside the Wing IDE. For
some reason using the subprocess module doesn't work so I need a
solution that doesn't use this module.
* The platform is Windows and I'm happy with a Windoze only solution.
:-)

I would like to execute subprocesses asynchronously and capture stdout
/ stderr as a single stream.

If I use "os.popen3(executable)" it gives me separate pipes for stdout
and stderr, but reads from them are blocking.

The output on stdout and stderr may be interleaved and I would like to
display them *as* they arrive. That means I can't just read from them
and output the results.

The only solution I can think of is to read from both a character at a
time on two separate threads, putting the data into a queue. A separate
thread could pull characters off the queue and display them. (I don't
need to differentiate between stdout and stderr when I display.)

Can anyone think of a better solution ?

My current code works, but *doesn't* capture stderr :

from threading import Thread

pipe = os.popen(executable)

def DisplayOutput():
while True:
output = pipe.read(1)
if not output:
break
display(output)

Thread(target=DisplayOutput).start()

All the best,

Fuzzymanhttp://www.voidspace.org.uk/python/articles.shtml
Try using popen4 instead. But since you already have a thread, it may
be better to use popen2 and two threads to read from stdout and stderr
to avoid a potential deadlock; they can put read lines into a Queue,
and DisplayOutput just get these lines in order. (See the warnings in
the popen2 module documentation).

--
Gabriel Genellina

Jan 7 '07 #2

P: n/a

Gabriel Genellina wrote:
On 7 ene, 16:33, "Fuzzyman" <fuzzy...@gmail.comwrote:
[snip..]
My current code works, but *doesn't* capture stderr :

from threading import Thread

pipe = os.popen(executable)

def DisplayOutput():
while True:
output = pipe.read(1)
if not output:
break
display(output)

Thread(target=DisplayOutput).start()

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/articles.shtml

Try using popen4 instead. But since you already have a thread, it may
be better to use popen2 and two threads to read from stdout and stderr
to avoid a potential deadlock; they can put read lines into a Queue,
and DisplayOutput just get these lines in order. (See the warnings in
the popen2 module documentation).
popen4 works great, I didn't even know it existed.

Two threads and a queue sounds horrible.

Thanks

Fuzzyman
http://www.voidspace.org.uk/python/articles.shtml
--
Gabriel Genellina
Jan 7 '07 #3

P: n/a
On 7 ene, 17:37, "Fuzzyman" <fuzzy...@gmail.comwrote:
Two threads and a queue sounds horrible.
But unfortunately it's the only way if you don't control how the child
process behaves.
(It's not soooo ugly afterwards... certainly would be worse if you had
to syncronize both reading threads and the display thread using
semaphores by hand.)

--
Gabriel Genellina

Jan 7 '07 #4

This discussion thread is closed

Replies have been disabled for this discussion.