Running long script in the background

[wattersmt]

Hello,

I am trying to write a python cgi that calls a script over ssh, the
problem is the script takes a very long time to execute so Apache
makes the CGI time out and I never see any output. The script is set
to print a progress report to stdout every 3 seconds but I never see
any output until the child process is killed.

Here's what I have in my python script:

command = "ssh -l root %s /scripts/xen/xen-create-win-vps1.sh %s" %
(host, domuname)
output = os.popen(comman d)
for line in output:
print line.strip()

Here's a copy of the bash script.

http://watters.ws/script.txt

I also tried using os.spawnv to run ssh in the background and nothing
happens.

Does anybody know a way to make output show in real time?

[petercable]

On Feb 6, 11:13 pm, "peterca...@gma il.com" <peterca...@gma il.com>
wrote:

output = os.popen(comman d, 'r', 1)

OOPS... I imagine the ridiculous buffer size is unnecessary... I was
trying to get it to work with the original for loop iterating on
output, it should work fine without it.

Pete

[Jordan]

On Feb 6, 8:26 am, "watter...@gmai l.com" <watter...@gmai l.comwrote:

Hello,

I am trying to write a python cgi that calls a script over ssh, the
problem is the script takes a very long time to execute so Apache
makes the CGI time out and I never see any output. The script is set
to print a progress report to stdout every 3 seconds but I never see
any output until the child process is killed.

Here's what I have in my python script:

command = "ssh -l root %s /scripts/xen/xen-create-win-vps1.sh %s" %
(host, domuname)
output = os.popen(comman d)
for line in output:
print line.strip()

Here's a copy of the bash script.

http://watters.ws/script.txt

I also tried using os.spawnv to run ssh in the background and nothing
happens.

Does anybody know a way to make output show in real time?

Just a little note: os.popen has been replaced by the subprocess
module. ;D

[Karthik Gurusamy]

On Feb 6, 5:26 am, "watter...@gmai l.com" <watter...@gmai l.comwrote:

Hello,

I am trying to write a python cgi that calls a script over ssh, the
problem is the script takes a very long time to execute so Apache
makes the CGI time out and I never see any output. The script is set
to print a progress report to stdout every 3 seconds but I never see
any output until the child process is killed.

Here's what I have in my python script:

command = "ssh -l root %s /scripts/xen/xen-create-win-vps1.sh %s" %
(host, domuname)
output = os.popen(comman d)

Apart from other buffering issues, it could be very well that ssh
returns all the output in one single big chunk. Try running the ssh
command (with the trailing 'command') from your shell and see if it
generates output immediately.

There may be some option to make ssh not buffer the data it reads from
the remove command execution. If there is no such option, most likely
you are out of luck. In this case, even if you making your remote
script unbufferred, ssh may be buffering it.

If both the machines have any shared filesystem, you can do a trick.
Make your script write it's output unbuffered to a file. Since the
file is mounted and available on both the machines.. start reading the
file from this main python script (note that you may need a thread to
do it, as your script will anyway be stuck waiting for the ssh to
complete).

Karthik

for line in output:
print line.strip()

Here's a copy of the bash script.

http://watters.ws/script.txt

I also tried using os.spawnv to run ssh in the background and nothing
happens.

Does anybody know a way to make output show in real time?

[tleeuwenburg]

On Feb 8, 10:42 am, "Karthik Gurusamy" <kar1...@gmail. comwrote:

On Feb 6, 5:26 am, "watter...@gmai l.com" <watter...@gmai l.comwrote:

Hello,

I am trying to write a python cgi that calls a script over ssh, the
problem is the script takes a very long time to execute so Apache
makes the CGI time out and I never see any output. The script is set
to print a progress report to stdout every 3 seconds but I never see
any output until the child process is killed.

Here's what I have in my python script:

command = "ssh -l root %s /scripts/xen/xen-create-win-vps1.sh %s" %
(host, domuname)
output = os.popen(comman d)

Apart from other buffering issues, it could be very well that ssh
returns all the output in one single big chunk. Try running the ssh
command (with the trailing 'command') from your shell and see if it
generates output immediately.

There may be some option to make ssh not buffer the data it reads from
the remove command execution. If there is no such option, most likely
you are out of luck. In this case, even if you making your remote
script unbufferred, ssh may be buffering it.

If both the machines have any shared filesystem, you can do a trick.
Make your script write it's output unbuffered to a file. Since the
file is mounted and available on both the machines.. start reading the
file from this main python script (note that you may need a thread to
do it, as your script will anyway be stuck waiting for the ssh to
complete).

Karthik

for line in output:
print line.strip()

Here's a copy of the bash script.

http://watters.ws/script.txt

I also tried using os.spawnv to run ssh in the background and nothing
happens.

Does anybody know a way to make output show in real time?

You could also try flushing the buffer after each status message