467,136 Members | 1,281 Online
Bytes | Developer Community
Ask Question

Home New Posts Topics Members FAQ

Post your question to a community of 467,136 developers. It's quick & easy.

Re: Popen pipe hang

On Mon, 12 May 2008 17:35:44 -0700 (PDT), schickb <sc*****@gmail.comwrote:
>I'm trying to pipe data that starts life in an array('B') object
through several processes. The code below is a simplified example. The
data makes it through, but the wait() always hangs. Is there a better
way to indicate src.stdin has reach EOF?

from subprocess import Popen, PIPE
from array import array

arr = array('B')
arr.fromstring("hello\n")

src = Popen( ["cat"], stdin=PIPE, stdout=PIPE)
dst = Popen( ["cat"], stdin=src.stdout)
arr.tofile(src.stdin)
src.stdin.close()
dst.wait()
Alas, you haven't actually closed src's standard input. Though you did
close src.stdin, you didn't close the copy of it which was inherited by
dst! So though the file descriptor is no longer open in your main process,
it remains open due to the reference dst has to it. You can fix this by
having Popen close all file descriptors except 0, 1, and 2 before it execs
cat - pass close_fds=True to the 2nd Popen call and you should get the
behavior you want.

Jean-Paul
Jun 27 '08 #1
  • viewed: 1619
Share:
1 Reply
On May 12, 7:35 pm, Jean-Paul Calderone <exar...@divmod.comwrote:
from subprocess import Popen, PIPE
from array import array
arr = array('B')
arr.fromstring("hello\n")
src = Popen( ["cat"], stdin=PIPE, stdout=PIPE)
dst = Popen( ["cat"], stdin=src.stdout)
arr.tofile(src.stdin)
src.stdin.close()
dst.wait()

Alas, you haven't actually closed src's standard input. Though you did
close src.stdin, you didn't close the copy of it which was inherited by
dst! So though the file descriptor is no longer open in your main process,
it remains open due to the reference dst has to it. You can fix this by
having Popen close all file descriptors except 0, 1, and 2 before it execs
cat - pass close_fds=True to the 2nd Popen call and you should get the
behavior you want.
Thanks, that did the trick. Although I assume that by passing
close_fds=True the second Popen is actually closing src.stdout (rather
than src.stdin as mentioned)? With close_fds=True, is Python buffering
the data even though bufsize=0 is the default? Otherwise I don't see
how it could close the fd before executing the process.
Jun 27 '08 #2

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

17 posts views Thread by bastiaannaber@gmail.com | last post: by
5 posts views Thread by damacy | last post: by
9 posts views Thread by Clodoaldo Pinto Neto | last post: by
3 posts views Thread by Jesse | last post: by
7 posts views Thread by skunkwerk | last post: by
8 posts views Thread by clyfish@gmail.com | last post: by
reply views Thread by schickb | last post: by
25 posts views Thread by Jeremy Banks | last post: by
1 post views Thread by Mark Shewfelt | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.