By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
444,034 Members | 1,325 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 444,034 IT Pros & Developers. It's quick & easy.

Popen and wget, problems

P: n/a
Hi all, I have a problem using wget and Popen. I hope someone can help.
-- Problem --
I want to use the command:
wget -nv -O "dir/cpan.txt" "http://search.cpan.org"
and capture all it's stdout+stderr.
(Note that option -O requires 'dir' to be existing before wget is executed)

Popen doesn't work, while os.system and shell do. Popen will give the error:
dir/cpan.txt: No such file or directory

While os.system and shell will give the correct result:
06:52:40 URL:http://search.cpan.org/ [3657/3657] -"dir1/cpan.txt" [1]

-- Background info about wget --
-Option -nv: -nv, --no-verbose turn off verboseness, without
being quiet.
-Option -O: -O, --output-document=FILE write documents to FILE.

Note that wget requires any directories in the file-path of option -O to be
existing before the wget command is executed.
-- Python Code using Popen with cmd arg list --
# imports
import os
from subprocess import Popen, PIPE

# vars and create dir
cmd_set = ['wget', '-nv', '-O dir/cpan.txt', 'http://search.span.org']
cmd = ' '.join(cmd_set)
print "cmd: " + cmd
try:
os.makedirs('dir')
except:
print 'dir already exists'
# execute using Popen (does NOT work)
proc = Popen(cmd_set, stdout=PIPE, stderr=PIPE)
return_code = proc.wait()
if return_code == 0:
print "Success:\n%s" % (proc.stdout.read() + proc.stderr.read())
else:
print "Failure %s:\n%s" % (return_code, proc.stderr.read() +
proc.stdout.read())
# execute using os.system (does work)
os.system(cmd)
-- Python code output of Popen --
Failure 1:
dir/cpan.txt: No such file or directory
-- Question --
Why is Popen unable to correctly execute the wget, while os.system can?

May 12 '07 #1
Share this Question
Share on Google+
3 Replies


P: n/a
"Jesse" <ha**@spam.comwrites:
Hi all, I have a problem using wget and Popen. I hope someone can help.
-- Problem --
I want to use the command:
wget -nv -O "dir/cpan.txt" "http://search.cpan.org"
and capture all it's stdout+stderr.
(Note that option -O requires 'dir' to be existing before wget is executed)

Popen doesn't work, while os.system and shell do. Popen will give the error:
dir/cpan.txt: No such file or directory

While os.system and shell will give the correct result:
06:52:40 URL:http://search.cpan.org/ [3657/3657] -"dir1/cpan.txt" [1]
[...]
-- Python Code using Popen with cmd arg list --
# imports
import os
from subprocess import Popen, PIPE

# vars and create dir
cmd_set = ['wget', '-nv', '-O dir/cpan.txt', 'http://search.span.org']
cmd = ' '.join(cmd_set)
print "cmd: " + cmd
try:
os.makedirs('dir')
except:
print 'dir already exists'
# execute using Popen (does NOT work)
proc = Popen(cmd_set, stdout=PIPE, stderr=PIPE)
return_code = proc.wait()
if return_code == 0:
print "Success:\n%s" % (proc.stdout.read() + proc.stderr.read())
else:
print "Failure %s:\n%s" % (return_code, proc.stderr.read() +
proc.stdout.read())
# execute using os.system (does work)
os.system(cmd)
-- Python code output of Popen --
Failure 1:
dir/cpan.txt: No such file or directory
-- Question --
Why is Popen unable to correctly execute the wget, while os.system can?
I don't know exactly why in this case Popen doesn't work,
but the counterpart of os.system is Popen with option shell=True
and the first parameter should be a string instead of list.
That seems to work:
proc = Popen("wget -nv -O dir/cpan.txt http://search.span.org",
shell=True, stdout=PIPE, stderr=PIPE)

and this variant seems to work too:
cmd_set = ['wget', '-nv', '-O', 'dir/cpan.txt', 'http://search.span.org']

--
HTH,
Rob
May 12 '07 #2

P: n/a
Thx Rob!

Your solution works perfect!
"Rob Wolfe" <rw@smsnet.plwrote in message
news:87************@merkury.smsnet.pl...
"Jesse" <ha**@spam.comwrites:
>Hi all, I have a problem using wget and Popen. I hope someone can help.
-- Problem --
I want to use the command:
wget -nv -O "dir/cpan.txt" "http://search.cpan.org"
and capture all it's stdout+stderr.
(Note that option -O requires 'dir' to be existing before wget is
executed)

Popen doesn't work, while os.system and shell do. Popen will give the
error:
dir/cpan.txt: No such file or directory

While os.system and shell will give the correct result:
06:52:40 URL:http://search.cpan.org/ [3657/3657] -"dir1/cpan.txt" [1]

[...]
>-- Python Code using Popen with cmd arg list --
# imports
import os
from subprocess import Popen, PIPE

# vars and create dir
cmd_set = ['wget', '-nv', '-O dir/cpan.txt', 'http://search.span.org']
cmd = ' '.join(cmd_set)
print "cmd: " + cmd
try:
os.makedirs('dir')
except:
print 'dir already exists'
# execute using Popen (does NOT work)
proc = Popen(cmd_set, stdout=PIPE, stderr=PIPE)
return_code = proc.wait()
if return_code == 0:
print "Success:\n%s" % (proc.stdout.read() + proc.stderr.read())
else:
print "Failure %s:\n%s" % (return_code, proc.stderr.read() +
proc.stdout.read())
# execute using os.system (does work)
os.system(cmd)
-- Python code output of Popen --
Failure 1:
dir/cpan.txt: No such file or directory
-- Question --
Why is Popen unable to correctly execute the wget, while os.system can?

I don't know exactly why in this case Popen doesn't work,
but the counterpart of os.system is Popen with option shell=True
and the first parameter should be a string instead of list.
That seems to work:
proc = Popen("wget -nv -O dir/cpan.txt http://search.span.org",
shell=True, stdout=PIPE, stderr=PIPE)

and this variant seems to work too:
cmd_set = ['wget', '-nv', '-O', 'dir/cpan.txt', 'http://search.span.org']

--
HTH,
Rob
May 13 '07 #3

P: n/a
js
Hi Jesse.
cmd_set = ['wget', '-nv', '-O dir/cpan.txt', 'http://search.span.org']
[snip]
>proc = Popen(cmd_set, stdout=PIPE, stderr=PIPE)
wget will treat this as
$ wget -nv '-O dir/cpan.txt' "http://search.cpan.org"

And will emit the following error because there's no pathname ' dir/cpan.txt'.
(Note the pathname has a trailing space.)

dir/cpan.txt: No such file or directory

Replace '-O dir/cpan.txt" with '-Odir/cpan.txt' or '-O', 'dir/cpan.txt'
and it should work.

Hope this helps
May 13 '07 #4

This discussion thread is closed

Replies have been disabled for this discussion.