473,372 Members | 858 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,372 software developers and data experts.

Output from subprocess.Popen()

Output from the shell:

[cpn@s0 teste]$ set | grep IFS
IFS=$' \t\n'

Output from subprocess.Popen():
>>import subprocess as sub
p = sub.Popen('set | grep IFS', shell=True, stdout=sub.PIPE)
p.stdout.readlines()[1]
"IFS=' \t\n"

Both outputs for comparison:
IFS=$' \t\n'
"IFS=' \t\n"

The subprocess.Popen() output is missing the $ and the last '

How to get the raw shell output from subprocess.Popen()?

Regards, Clodoaldo Pinto Neto

Oct 15 '06 #1
9 9340
Clodoaldo Pinto Neto wrote:
Output from the shell:

[cpn@s0 teste]$ set | grep IFS
IFS=$' \t\n'

Output from subprocess.Popen():
>>>import subprocess as sub
p = sub.Popen('set | grep IFS', shell=True, stdout=sub.PIPE)
p.stdout.readlines()[1]
"IFS=' \t\n"

Both outputs for comparison:stdout=subprocess.PIPE)
IFS=$' \t\n'
"IFS=' \t\n"

The subprocess.Popen() output is missing the $ and the last '
if you expect one line of output, why are you doing readlines()[1] ?
>>f = subprocess.Popen("set | grep IFS", shell=True,
f.stdout.readlines()[1]
Traceback (most recent call last):
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
IndexError: list index out of range

this works for me:
>>f = subprocess.Popen("set | grep IFS", shell=True,
stdout=subprocess.PIPE)
>>f.stdout.readlines()
["IFS=$' \\t\\n'\n"]

what does the above return on your machine?

</F>

Oct 15 '06 #2

Fredrik Lundh wrote:
this works for me:
>>f = subprocess.Popen("set | grep IFS", shell=True,
stdout=subprocess.PIPE)
>>f.stdout.readlines()
["IFS=$' \\t\\n'\n"]

what does the above return on your machine?
>>f = subprocess.Popen("set | grep IFS", shell=True, stdout=subprocess.PIPE)
f.stdout.readlines()
["BASH_EXECUTION_STRING='set | grep IFS'\n", "IFS=' \t\n"]

I'm on FC5

Oct 15 '06 #3
Clodoaldo Pinto Neto <cl*************@gmail.comwrote:
>Fredrik Lundh wrote:
>this works for me:
> >>f = subprocess.Popen("set | grep IFS", shell=True,
stdout=subprocess.PIPE)
> >>f.stdout.readlines()
["IFS=$' \\t\\n'\n"]

what does the above return on your machine?
>>>f = subprocess.Popen("set | grep IFS", shell=True, stdout=subprocess.PIPE)
f.stdout.readlines()
["BASH_EXECUTION_STRING='set | grep IFS'\n", "IFS=' \t\n"]

I'm on FC5
OK, there's something going on here:
$ set | grep IFS
IFS=$' \t\n'
$ python
Python 2.4.1 (#2, May 5 2005, 11:32:06)
[GCC 3.3.5 (Debian 1:3.3.5-12)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>import subprocess
f = subprocess.Popen("set | grep IFS", shell=True, stdout=subprocess.PIPE)
f.stdout.readlines()
["IFS=' \t\n"]
>>>
(Hmph, I don't have any other non-Windows boxes with >=2.4 to hand.)

--
\S -- si***@chiark.greenend.org.uk -- http://www.chaos.org.uk/~sion/
___ | "Frankly I have no feelings towards penguins one way or the other"
\X/ | -- Arthur C. Clarke
her nu becomež se bera eadward ofdun hlęddre heafdes bęce bump bump bump
Oct 16 '06 #4
Now we have 3 different outputs from 3 people to the command:
>>f = subprocess.Popen("set | grep IFS", shell=True,
stdout=subprocess.PIPE)
>>f.stdout.readlines()
>From me on FC5:
["BASH_EXECUTION_STRING='set | grep IFS'\n", "IFS=' \t\n"]
>From Fredrik Lundh on unknown OS:
["IFS=$' \\t\\n'\n"]
>From Sion Arrowsmith on Debian 1:3.3.5-12:
["IFS=' \t\n"]

I found this problem while developing a CGI shell:
http://code.google.com/p/cgpy-shell/

Although the cgi-shell is working quite well, this issue is a show
stopper for that project. I need a stable/reliable output from Popen().
It looks like each bash or sh version outputs things differently, with
different escaping.

It is not a problem if there is only one or more than one line in the
returned list as i will always get the last one. The problem is the
escaping.

Oct 16 '06 #5
Clodoaldo Pinto Neto wrote:
Output from the shell:

[cpn@s0 teste]$ set | grep IFS
IFS=$' \t\n'

Output from subprocess.Popen():
>import subprocess as sub
p = sub.Popen('set | grep IFS', shell=True, stdout=sub.PIPE)
p.stdout.readlines()[1]
"IFS=' \t\n"

Both outputs for comparison:
IFS=$' \t\n'
"IFS=' \t\n"

The subprocess.Popen() output is missing the $ and the last '

How to get the raw shell output from subprocess.Popen()?
You are getting the raw shell output, it's just that subprocess uses
the default shell (/bin/sh) and not your personal shell. Try running
/bin/sh and run the set|grep command.

I can't see any obvious way to ask subprocess to use a shell other than
the default.
If you need a particular shell, you could use a shell script, e.g.
myscript.sh:
#!/bin/bash
set|grep IFS

and call that with shell=False
p = sub.Popen('/path/to/myscript.sh', stdout=sub.PIPE)

Oct 16 '06 #6
sj*******@yahoo.com wrote:
I can't see any obvious way to ask subprocess to use a shell other than
the default.
-c ?
>>f = Popen(["/bin/bash", "-c", "set|grep IFS"], stdout=PIPE)
f.stdout.read()
"IFS=$' \\t\\n'\n"
>>f = Popen(["/bin/sh", "-c", "set|grep IFS"], stdout=PIPE)
f.stdout.read()
"IFS=' \t\n"

</F>

Oct 16 '06 #7
Fredrik Lundh wrote:
sj*******@yahoo.com wrote:
I can't see any obvious way to ask subprocess to use a shell other than
the default.

-c ?
>>f = Popen(["/bin/bash", "-c", "set|grep IFS"], stdout=PIPE)
>>f.stdout.read()
"IFS=$' \\t\\n'\n"
>>f = Popen(["/bin/sh", "-c", "set|grep IFS"], stdout=PIPE)
>>f.stdout.read()
"IFS=' \t\n"
It solves my problem:
>>f = sub.Popen(['/bin/sh', '-c', 'set|grep IFS'], stdout=sub.PIPE)
f.stdout.read()
"BASH_EXECUTION_STRING='set|grep IFS'\nIFS=' \t\n"
>>f = sub.Popen(['/bin/bash', '-c', 'set|grep IFS'], stdout=sub.PIPE)
f.stdout.read()
"BASH_EXECUTION_STRING='set|grep IFS'\nIFS=$' \\t\\n'\n"

But I still don't understand what is happening. The manual says that
when shell=True the executable argument specifies which shell to use:
>>f = sub.Popen('set|grep IFS', shell=True, executable='/bin/sh', stdout=sub.PIPE)
f.stdout.read()
"BASH_EXECUTION_STRING='set|grep IFS'\nIFS=' \t\n"
>>f = sub.Popen('set|grep IFS', shell=True, executable='/bin/bash', stdout=sub.PIPE)
f.stdout.read()
"BASH_EXECUTION_STRING='set|grep IFS'\nIFS=' \t\n"

To make my confusion bigger, in Fedora sh is just a link to bash:
$ ll /bin/*sh
-rwxr-xr-x 1 root root 720888 Feb 11 2006 /bin/bash
lrwxrwxrwx 1 root root 4 Aug 28 22:53 /bin/csh -tcsh
-rwxr-xr-x 1 root root 1175496 Aug 17 13:19 /bin/ksh
lrwxrwxrwx 1 root root 4 Feb 24 2006 /bin/sh -bash
-rwxr-xr-x 1 root root 349312 Aug 17 15:20 /bin/tcsh
-rwxr-xr-x 1 root root 514668 Feb 12 2006 /bin/zsh

Oct 17 '06 #8
Clodoaldo Pinto Neto wrote:
But I still don't understand what is happening. The manual says that
when shell=True the executable argument specifies which shell to use:
no, it says that when shell=True, it runs the command "through" the
default shell. that is, it hands it over to the shell for execution,
pretty much as if you'd typed it in yourself.
To make my confusion bigger, in Fedora sh is just a link to bash:
that only means that the programs share the same binary, not that they
necessarily have exactly the same behaviour:

$ man bash

...

INVOCATION

...

If bash is invoked with the name sh, it tries to mimic the
startup behavior of historical versions of sh as closely as
possible, while conforming to the POSIX standard as well.

...

</F>

Oct 17 '06 #9
Fredrik Lundh wrote:
Clodoaldo Pinto Neto wrote:
But I still don't understand what is happening. The manual says that
when shell=True the executable argument specifies which shell to use:

no, it says that when shell=True, it runs the command "through" the
default shell. that is, it hands it over to the shell for execution,
pretty much as if you'd typed it in yourself.
"If shell=True, the executable argument specifies which shell to use."

Taken from 6.8.1 v2.4:

"The executable argument specifies the program to execute. It is very
seldom needed: Usually, the program to execute is defined by the args
argument. If shell=True, the executable argument specifies which shell
to use. On Unix, the default shell is /bin/sh. On Windows, the default
shell is specified by the COMSPEC environment variable."

Oct 17 '06 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

14
by: jas | last post by:
I would like to redirect the output from os.system to a variable, but am having trouble. I tried using os.popen(..).read() ...but that doesn't give me exactly what i want. ...this is windows by...
3
by: Darren Dale | last post by:
I'm a developer on the matplotlib project, and I am having trouble with the subprocess module on windows (Python 2.4.2 on winXP). No trouble to report with linux. I need to use _subprocess instead...
6
by: Kkaa | last post by:
I'm using the os.system command in a python script on Windows to run a batch file like this: os.system('x.exe') The third-party program x.exe outputs some text to the console that I want to...
3
by: johnny | last post by:
How do I pipe the output, generated from os.system(some_command), to the logging file?
14
by: nathan.shair | last post by:
Hello, I searched on Google and in this Google Group, but did not find any solution to my problem. I'm looking for a way to output stdout/stderr (from a subprocess or spawn) to screen and to...
4
by: Tobiah | last post by:
I am not sure how to capture the output of a command using subprocess without creating a temp file. I was trying this: import StringIO import subprocess file = StringIO.StringIO() ...
2
by: rparimi | last post by:
I am trying to redirect stderr of a process to a temporary file and then read back the contents of the file, all in the same python script. As a simple exercise, I launched /bin/ls but this doesn't...
25
by: Jeremy Banks | last post by:
Hi. I wondered if anyone knew the rationale behind the naming of the Popen class in the subprocess module. Popen sounds like the a suitable name for a function that created a subprocess, but the...
5
by: thedsadude | last post by:
Hello, I'm launching a script as follows: <code> p = subprocess.Popen() p.wait() </code> If p.py writes to sys.stdout, then it is shown on the console.
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.