By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
444,050 Members | 1,019 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 444,050 IT Pros & Developers. It's quick & easy.

Output from subprocess.Popen()

P: n/a
Output from the shell:

[cpn@s0 teste]$ set | grep IFS
IFS=$' \t\n'

Output from subprocess.Popen():
>>import subprocess as sub
p = sub.Popen('set | grep IFS', shell=True, stdout=sub.PIPE)
p.stdout.readlines()[1]
"IFS=' \t\n"

Both outputs for comparison:
IFS=$' \t\n'
"IFS=' \t\n"

The subprocess.Popen() output is missing the $ and the last '

How to get the raw shell output from subprocess.Popen()?

Regards, Clodoaldo Pinto Neto

Oct 15 '06 #1
Share this Question
Share on Google+
9 Replies


P: n/a
Clodoaldo Pinto Neto wrote:
Output from the shell:

[cpn@s0 teste]$ set | grep IFS
IFS=$' \t\n'

Output from subprocess.Popen():
>>>import subprocess as sub
p = sub.Popen('set | grep IFS', shell=True, stdout=sub.PIPE)
p.stdout.readlines()[1]
"IFS=' \t\n"

Both outputs for comparison:stdout=subprocess.PIPE)
IFS=$' \t\n'
"IFS=' \t\n"

The subprocess.Popen() output is missing the $ and the last '
if you expect one line of output, why are you doing readlines()[1] ?
>>f = subprocess.Popen("set | grep IFS", shell=True,
f.stdout.readlines()[1]
Traceback (most recent call last):
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: list index out of range

this works for me:
>>f = subprocess.Popen("set | grep IFS", shell=True,
stdout=subprocess.PIPE)
>>f.stdout.readlines()
["IFS=$' \\t\\n'\n"]

what does the above return on your machine?

</F>

Oct 15 '06 #2

P: n/a

Fredrik Lundh wrote:
this works for me:
>>f = subprocess.Popen("set | grep IFS", shell=True,
stdout=subprocess.PIPE)
>>f.stdout.readlines()
["IFS=$' \\t\\n'\n"]

what does the above return on your machine?
>>f = subprocess.Popen("set | grep IFS", shell=True, stdout=subprocess.PIPE)
f.stdout.readlines()
["BASH_EXECUTION_STRING='set | grep IFS'\n", "IFS=' \t\n"]

I'm on FC5

Oct 15 '06 #3

P: n/a
Clodoaldo Pinto Neto <cl*************@gmail.comwrote:
>Fredrik Lundh wrote:
>this works for me:
> >>f = subprocess.Popen("set | grep IFS", shell=True,
stdout=subprocess.PIPE)
> >>f.stdout.readlines()
["IFS=$' \\t\\n'\n"]

what does the above return on your machine?
>>>f = subprocess.Popen("set | grep IFS", shell=True, stdout=subprocess.PIPE)
f.stdout.readlines()
["BASH_EXECUTION_STRING='set | grep IFS'\n", "IFS=' \t\n"]

I'm on FC5
OK, there's something going on here:
$ set | grep IFS
IFS=$' \t\n'
$ python
Python 2.4.1 (#2, May 5 2005, 11:32:06)
[GCC 3.3.5 (Debian 1:3.3.5-12)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>import subprocess
f = subprocess.Popen("set | grep IFS", shell=True, stdout=subprocess.PIPE)
f.stdout.readlines()
["IFS=' \t\n"]
>>>
(Hmph, I don't have any other non-Windows boxes with >=2.4 to hand.)

--
\S -- si***@chiark.greenend.org.uk -- http://www.chaos.org.uk/~sion/
___ | "Frankly I have no feelings towards penguins one way or the other"
\X/ | -- Arthur C. Clarke
her nu becomež se bera eadward ofdun hlęddre heafdes bęce bump bump bump
Oct 16 '06 #4

P: n/a
Now we have 3 different outputs from 3 people to the command:
>>f = subprocess.Popen("set | grep IFS", shell=True,
stdout=subprocess.PIPE)
>>f.stdout.readlines()
>From me on FC5:
["BASH_EXECUTION_STRING='set | grep IFS'\n", "IFS=' \t\n"]
>From Fredrik Lundh on unknown OS:
["IFS=$' \\t\\n'\n"]
>From Sion Arrowsmith on Debian 1:3.3.5-12:
["IFS=' \t\n"]

I found this problem while developing a CGI shell:
http://code.google.com/p/cgpy-shell/

Although the cgi-shell is working quite well, this issue is a show
stopper for that project. I need a stable/reliable output from Popen().
It looks like each bash or sh version outputs things differently, with
different escaping.

It is not a problem if there is only one or more than one line in the
returned list as i will always get the last one. The problem is the
escaping.

Oct 16 '06 #5

P: n/a
Clodoaldo Pinto Neto wrote:
Output from the shell:

[cpn@s0 teste]$ set | grep IFS
IFS=$' \t\n'

Output from subprocess.Popen():
>import subprocess as sub
p = sub.Popen('set | grep IFS', shell=True, stdout=sub.PIPE)
p.stdout.readlines()[1]
"IFS=' \t\n"

Both outputs for comparison:
IFS=$' \t\n'
"IFS=' \t\n"

The subprocess.Popen() output is missing the $ and the last '

How to get the raw shell output from subprocess.Popen()?
You are getting the raw shell output, it's just that subprocess uses
the default shell (/bin/sh) and not your personal shell. Try running
/bin/sh and run the set|grep command.

I can't see any obvious way to ask subprocess to use a shell other than
the default.
If you need a particular shell, you could use a shell script, e.g.
myscript.sh:
#!/bin/bash
set|grep IFS

and call that with shell=False
p = sub.Popen('/path/to/myscript.sh', stdout=sub.PIPE)

Oct 16 '06 #6

P: n/a
sj*******@yahoo.com wrote:
I can't see any obvious way to ask subprocess to use a shell other than
the default.
-c ?
>>f = Popen(["/bin/bash", "-c", "set|grep IFS"], stdout=PIPE)
f.stdout.read()
"IFS=$' \\t\\n'\n"
>>f = Popen(["/bin/sh", "-c", "set|grep IFS"], stdout=PIPE)
f.stdout.read()
"IFS=' \t\n"

</F>

Oct 16 '06 #7

P: n/a
Fredrik Lundh wrote:
sj*******@yahoo.com wrote:
I can't see any obvious way to ask subprocess to use a shell other than
the default.

-c ?
>>f = Popen(["/bin/bash", "-c", "set|grep IFS"], stdout=PIPE)
>>f.stdout.read()
"IFS=$' \\t\\n'\n"
>>f = Popen(["/bin/sh", "-c", "set|grep IFS"], stdout=PIPE)
>>f.stdout.read()
"IFS=' \t\n"
It solves my problem:
>>f = sub.Popen(['/bin/sh', '-c', 'set|grep IFS'], stdout=sub.PIPE)
f.stdout.read()
"BASH_EXECUTION_STRING='set|grep IFS'\nIFS=' \t\n"
>>f = sub.Popen(['/bin/bash', '-c', 'set|grep IFS'], stdout=sub.PIPE)
f.stdout.read()
"BASH_EXECUTION_STRING='set|grep IFS'\nIFS=$' \\t\\n'\n"

But I still don't understand what is happening. The manual says that
when shell=True the executable argument specifies which shell to use:
>>f = sub.Popen('set|grep IFS', shell=True, executable='/bin/sh', stdout=sub.PIPE)
f.stdout.read()
"BASH_EXECUTION_STRING='set|grep IFS'\nIFS=' \t\n"
>>f = sub.Popen('set|grep IFS', shell=True, executable='/bin/bash', stdout=sub.PIPE)
f.stdout.read()
"BASH_EXECUTION_STRING='set|grep IFS'\nIFS=' \t\n"

To make my confusion bigger, in Fedora sh is just a link to bash:
$ ll /bin/*sh
-rwxr-xr-x 1 root root 720888 Feb 11 2006 /bin/bash
lrwxrwxrwx 1 root root 4 Aug 28 22:53 /bin/csh -tcsh
-rwxr-xr-x 1 root root 1175496 Aug 17 13:19 /bin/ksh
lrwxrwxrwx 1 root root 4 Feb 24 2006 /bin/sh -bash
-rwxr-xr-x 1 root root 349312 Aug 17 15:20 /bin/tcsh
-rwxr-xr-x 1 root root 514668 Feb 12 2006 /bin/zsh

Oct 17 '06 #8

P: n/a
Clodoaldo Pinto Neto wrote:
But I still don't understand what is happening. The manual says that
when shell=True the executable argument specifies which shell to use:
no, it says that when shell=True, it runs the command "through" the
default shell. that is, it hands it over to the shell for execution,
pretty much as if you'd typed it in yourself.
To make my confusion bigger, in Fedora sh is just a link to bash:
that only means that the programs share the same binary, not that they
necessarily have exactly the same behaviour:

$ man bash

...

INVOCATION

...

If bash is invoked with the name sh, it tries to mimic the
startup behavior of historical versions of sh as closely as
possible, while conforming to the POSIX standard as well.

...

</F>

Oct 17 '06 #9

P: n/a
Fredrik Lundh wrote:
Clodoaldo Pinto Neto wrote:
But I still don't understand what is happening. The manual says that
when shell=True the executable argument specifies which shell to use:

no, it says that when shell=True, it runs the command "through" the
default shell. that is, it hands it over to the shell for execution,
pretty much as if you'd typed it in yourself.
"If shell=True, the executable argument specifies which shell to use."

Taken from 6.8.1 v2.4:

"The executable argument specifies the program to execute. It is very
seldom needed: Usually, the program to execute is defined by the args
argument. If shell=True, the executable argument specifies which shell
to use. On Unix, the default shell is /bin/sh. On Windows, the default
shell is specified by the COMSPEC environment variable."

Oct 17 '06 #10

This discussion thread is closed

Replies have been disabled for this discussion.