By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
443,922 Members | 1,689 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 443,922 IT Pros & Developers. It's quick & easy.

Doing readline in a thread from a popen4('rsync ...') stream blocks when the stream ends.

P: n/a

Hi,

I have a problem doing I/O in a python thread.

I want a thread to execute a command using
os.popen, read its input until the end, and
then print 'DONE' and terminate.

However, the thread hangs when it reaches
the end of the stream.

The command that I want to do is 'rsync'
which is used to transfer files between hosts.
Rsync uses ssh to transfer the data securely.

Here is a small program that demonstrates
the problem:

------START-------
#!/usr/bin/python2.3
# -*- python -*-

# This program demonstrates a case when Python locks up.
# It runs the program 'rsync' and reads its output.
# When this is done inside a thread, the program locks up
# while doing the last i.readline()

# To try the program, make sure that you can ssh into localhost
# without password prompt, and that /etc/hosts exists
#

fileToGet = '/etc/hosts'
cmd='rsync -v -e ssh localhost:%s file.txt' % fileToGet

def do():
import os
(o,i)=os.popen4(cmd)
readAll(i)

def readAll(i):
x=i.readline()
while x:
print x
x=i.readline() # this is where the program locks up
print 'DONE'

from threading import Thread

class RunInThread(Thread):
def __init__(self,run,daemon=True):
Thread.__init__(self)
self.setDaemon(daemon)
self.run = run
self.start()

print '\nFirst we run the command outside of a thread. This works fine.\n'

do()

print '\nNow we will run the command inside a python thread. '
print 'The program will never print DONE, nor finish.\n'

RunInThread(do,daemon=False)

-------END-------
This problem occurs with rsync, so one guess is that it
is related to the interaction with python's threads and
and rsync or ssh.

One message I google'd up mentioned that non-standard
sighandlers could be a problem when running rsync from perl,
but I have not had any luck there.

I have also tried pexpect and pty.fork as alternatives to popen,
but with no luck.

I have the same problem on both Windows and Linux platforms.
It occurs on many versions of python.
I'd be very grateful for any suggestions/hints/solutions
to this problem!
Thanks!
Lars

Jul 18 '05 #1
Share this Question
Share on Google+
1 Reply


P: n/a
"Rasmusson, Lars" <la************@hp.com> wrote in message news:<ma*************************************@pyth on.org>...
Hi,

I have a problem doing I/O in a python thread.

I want a thread to execute a command using
os.popen, read its input until the end, and
then print 'DONE' and terminate.

However, the thread hangs when it reaches
the end of the stream.


You can have a look here:
http://sourceforge.net/tracker/?grou...70&atid=105470, looking for
'popen'.
requestID #566037 may be of interest, for eg.
Jul 18 '05 #2

This discussion thread is closed

Replies have been disabled for this discussion.