By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
457,865 Members | 1,365 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 457,865 IT Pros & Developers. It's quick & easy.

#!/usr/bin/python or #!/usr/bin/env python?

P: n/a
I understand the difference, but I'm just curious if anyone has any
strong feelings toward using one over the other? I was reading that a
disadvantage to the more general usage (i.e. env) is that it finds the
first python on the path, and that might not be the proper one to use. I
don't know if that's a real issue most of the time, but it's at least
something to consider.

And is Python found in directories other than /usr/bin often enough to
justify using the more general form?
Aug 8 '06 #1
Share this Question
Share on Google+
29 Replies


P: n/a
John Salerno wrote:
I understand the difference, but I'm just curious if anyone has any
strong feelings toward using one over the other? I was reading that a
disadvantage to the more general usage (i.e. env) is that it finds the
first python on the path, and that might not be the proper one to use.
I don't see how that's any different from when the Python at
/usr/bin/python isn't the right one to use. I know that's true at my
university:

$ /usr/bin/python -V
Python 2.2.3
$ /usr/local/python/bin/python -V
Python 2.4.3

With this setup, the one in /usr/bin is for the OS and is seldom
updated, and the one in /usr/local is for users and is updated frequently.

Sure, if someone doesn't put /usr/local/python/bin on their PATH,
they're not going to get the right Python, but I don't write code to
Python 2.2 anymore, so if I use ``#!/usr/bin/python`` my code will
probably just fail.

Obviously, in my own code, I only use ``#!/usr/bin/env python``.

STeVe
Aug 8 '06 #2

P: n/a
John Salerno schrieb:
I understand the difference, but I'm just curious if anyone has any
strong feelings toward using one over the other?
I use /usr/bin/env if I don't know what the operating system is;
some systems don't have Python in /usr/bin. I use /usr/bin/pythonX.Y
if I want a specific version on a specific operating system (typically
Linux). I use /usr/bin/python when I'm too lazy to think about it
thoroughly.

Regards,
Martin
Aug 8 '06 #3

P: n/a

JohnI understand the difference, but I'm just curious if anyone has
Johnany strong feelings toward using one over the other?

#!/usr/bin/python -never

#!/usr/bin/env python -always

Strong enough?
JohnI was reading that a disadvantage to the more general usage
John(i.e. env) is that it finds the first python on the path

That's generally the one you want. On most systems /usr/bin/python will be
the system-provided one (e.g. the one that came with your Linux
distribution). You might prefer a Python of more recent vintage though.
That's commonly installed in /usr/local/bin/python and /usr/local/bin is
generally ahead of /usr/bin in PATH.

JohnAnd is Python found in directories other than /usr/bin often
Johnenough to justify using the more general form?

On my Mac I install most stuff in ~/local simply so I don't need to be
root and don't disturb the system-provided software. In cases where other
users need to run such tools I generally use /usr/local.

Skip

Aug 8 '06 #4

P: n/a
sk**@pobox.com wrote:
On most systems /usr/bin/python will be
the system-provided one (e.g. the one that came with your Linux
distribution). You might prefer a Python of more recent vintage though.
That's commonly installed in /usr/local/bin/python and /usr/local/bin is
generally ahead of /usr/bin in PATH.
Ah, that answered my next question too! I guess I'll stick with the env
method, since I like the more general, abstract stuff over the absolute
anyway.
Aug 8 '06 #5

P: n/a
John Salerno wrote:
I understand the difference, but I'm just curious if anyone has any
strong feelings toward using one over the other? I was reading that a
disadvantage to the more general usage (i.e. env) is that it finds the
first python on the path, and that might not be the proper one to use. I
don't know if that's a real issue most of the time, but it's at least
something to consider.

And is Python found in directories other than /usr/bin often enough to
justify using the more general form?
Yes. If Python (or indeed any other well-behaving software) is
installed manually, it will _not_ be installed in /usr/bin. /usr is
where your primary OS/distribution installation goes. Additional
software goes elsewhere, such as /usr/local.

Always prefer to use env over a hardcoded path, because that hardcoded
path will invariably be wrong. (Yes, for those about to nitpick, it's
conceivable that env might be somewhere other than /usr/bin. However,
that is very rare and results in a no-win situations regardless of the
issue of where Python is installed.)

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Never contend with a man who has nothing to lose.
-- Baltasar Gracian, 1647
Aug 8 '06 #6

P: n/a
Martin v. L÷wis wrote:
I use /usr/bin/env if I don't know what the operating system is;
some systems don't have Python in /usr/bin. I use /usr/bin/pythonX.Y
if I want a specific version on a specific operating system (typically
Linux).
Even there, /usr/bin/env pythonX.Y would be a better choice. (Maybe
that's what you meant.)

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Never contend with a man who has nothing to lose.
-- Baltasar Gracian, 1647
Aug 8 '06 #7

P: n/a
Erik Max Francis schrieb:
>I use /usr/bin/env if I don't know what the operating system is;
some systems don't have Python in /usr/bin. I use /usr/bin/pythonX.Y
if I want a specific version on a specific operating system (typically
Linux).

Even there, /usr/bin/env pythonX.Y would be a better choice. (Maybe
that's what you meant.)
I'm uncertain. When I refer to /usr/bin/python2.3, I know I get the
system vendor's installation. When I use /usr/bin/env python2.3, I
may get something that somebody has build and which may not work
at all, or lack certain add-on packages that the script needs.
It's bad when the script works for some users but fails for others.

Regards,
Martin
Aug 8 '06 #8

P: n/a
Always prefer to use env over a hardcoded path, because that hardcoded
path will invariably be wrong. (Yes, for those about to nitpick, it's
conceivable that env might be somewhere other than /usr/bin. However,
that is very rare and results in a no-win situations regardless of the
issue of where Python is installed.)
Don't yell at me for bringing in another language, but I really like the
trick, Tcl does:
#!/bin/sh
# The next line starts Tcl \
exec tclsh "$0" "$@"
This works by the somewhat weird feature of Tcl, that allows comments to be
continued in the next line with "\" at the end of the comment-line. It
looks unfamiliar, but has several advantages, I think. First it's really
VERY unlikely, that there is no /bin/sh (while I found systems with
different places for env), and you can add some other features at or before
the actual call of the interpreter, i.e. finding the right or preferred
version... - This way I coded a complete software-installer, that runs
either as a Tcl/Tk-Script with GUI, or as bash-script, when no Tcl is
available. - I really would like to have something like that for python,
but I did not find a way to achieve that, yet.

Regards
Stephan

Aug 9 '06 #9

P: n/a
On 2006-08-09 at 08:02:03 (+0200), Stephan Kuhagen wrote:
Don't yell at me for bringing in another language, but I really like the
trick, Tcl does:
#!/bin/sh
# The next line starts Tcl \
exec tclsh "$0" "$@"

This works by the somewhat weird feature of Tcl, that allows comments to be
continued in the next line with "\" at the end of the comment-line. It
looks unfamiliar, but has several advantages, I think. First it's really
VERY unlikely, that there is no /bin/sh (while I found systems with
different places for env), and you can add some other features at or before
the actual call of the interpreter, i.e. finding the right or preferred
version... - This way I coded a complete software-installer, that runs
either as a Tcl/Tk-Script with GUI, or as bash-script, when no Tcl is
available. - I really would like to have something like that for python,
but I did not find a way to achieve that, yet.
You could use:
#!/bin/sh
"""exec" python "$0" "$@"""
:)

--
__ __________________________________________________ __________________
| \/ | Micha│ Bartoszkiewicz <em**@magma-net.pl | _ )
| |\/| | GG:2298240 | _ \
|_| |_|_For all resources, whatever it is, you need more. [RFC1925]_|___/
Aug 9 '06 #10

P: n/a
Michał Bartoszkiewicz wrote:
#!/bin/sh
"""exec" python "$0" "$@"""
Wow, cool... I like that!

Stephan
Aug 9 '06 #11

P: n/a
Stephan Kuhagen wrote:
Michał Bartoszkiewicz wrote:
>#!/bin/sh
"""exec" python "$0" "$@"""

Wow, cool... I like that!
Only someone genuinely fond of the Tcl hack could ...

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
No man who needs a monument ever ought to have one.
-- Nathaniel Hawthorne
Aug 9 '06 #12

P: n/a
Erik Max Francis wrote:
>>#!/bin/sh
"""exec" python "$0" "$@"""

Wow, cool... I like that!

Only someone genuinely fond of the Tcl hack could ...
True, I admit, I'm a Tcl-Addict... But I really love Python too for many
reasons. But I miss features and tricks in both languages that I have in
the other...

Interleaving the script-language with shell-scripting was one of them. So
I'm a little bit happier with Python now... ;-)

Stephan
Aug 9 '06 #13

P: n/a
ZeD
Stephan Kuhagen wrote:
>#!/bin/sh
"""exec" python "$0" "$@"""

Wow, cool... I like that!
yeah, but...

$ cat test.py
#!/bin/sh
"""exec" python "$0" "$@"""

print "Hello, world"
$ file test.py
test.py: Bourne shell script text executable

--
Under construction
Aug 9 '06 #14

P: n/a
ZeD wrote:
print "Hello, world"
$ file test.py
test.py: Bourne shell script text executable
Yes, the same happens with all Tcl-Scripts. I like to see this as a bug in
"file", not in the scripting...

Stephan
Aug 9 '06 #15

P: n/a
Stephan Kuhagen wrote:
Yes, the same happens with all Tcl-Scripts. I like to see this as a bug in
"file", not in the scripting...
How does that make sense? `file` cannot possibly understand the
semantics of files at that level, at least not without executing them.
And that's exactly what you _don't_ want to do when you're using `file` ...

The file _is_ a /bin/sh executable. You're just having that /bin/sh
executable run something else -- how could `file` figure that out
without a ridiculously complicated set of rules that rise to the level
of a sh interpreter -- thereby, defeating the purpose?

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Only those who dare to fail greatly can ever achieve greatly.
-- Robert F. Kennedy
Aug 9 '06 #16

P: n/a
[Erik Max Francis
How does that make sense? `file` cannot possibly understand the
semantics of files at that level, at least not without executing them.
And that's exactly what you _don't_ want to do when you're using `file` ...
This is very off-topic, but if it's fairly common to begin tcl-scripts
as a /bin/sh-file with "exec tcl" at one of the first lines, I think
"file" ought to be able to recognize it.

"""exec" python is clearly an obscure hack not used by many, so I
don't see why "file" should ever recognize that :-)

--
Tobias Brox, 69░42'N, 18░57'E
Aug 9 '06 #17

P: n/a
Tobias Brox wrote:
This is very off-topic, but if it's fairly common to begin tcl-scripts
as a /bin/sh-file with "exec tcl" at one of the first lines, I think
"file" ought to be able to recognize it.

"""exec" python is clearly an obscure hack not used by many, so I
don't see why "file" should ever recognize that :-)
The point is, they're all part of the same tactic -- the particulars of
sh. Special casing each one is a task without an end. People will come
up with variants that will do the right thing but foil `file`,
intentionally or unintentionally -- just as we've seen in this thread.

The right way to approach this with `file` is to acknowledge that such
tricks are inherently sh-specific and leave it identified as a sh file.
Because that is, of course, exactly what it is.

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Only those who dare to fail greatly can ever achieve greatly.
-- Robert F. Kennedy
Aug 9 '06 #18

P: n/a
Tobias Brox wrote:
This is very off-topic,
Sorry for starting that...
but if it's fairly common to begin tcl-scripts
as a /bin/sh-file with "exec tcl" at one of the first lines, I think
"file" ought to be able to recognize it.

"""exec" python is clearly an obscure hack not used by many, so I
don't see why "file" should ever recognize that :-)
That's what I meant. It is okay for a Python-script not to be recognized by
"file" that way, it is enough that I can do it, if I need to. But file
should recognize this for Tcl, because it is common there. And if it needs
to work for Tcl only, one can construct a simple mechanism for "file" to
check this.

Stephan
Aug 9 '06 #19

P: n/a
ZeD
Erik Max Francis wrote:
The file _is_ a /bin/sh executable. You're just having that /bin/sh
executable run something else -- how could `file` figure that out
without a ridiculously complicated set of rules that rise to the level
of a sh interpreter -- thereby, defeating the purpose?
but...

$ cat test.py
#!/usr/bin/env python

print "Hello, world"
$ file test.py
file.py: a python script text executable

following what you said, test.py is a /usr/bin/env script, not a python one.

--
Under construction
Aug 9 '06 #20

P: n/a
[Erik Max Francis]
The point is, they're all part of the same tactic -- the particulars of
sh. Special casing each one is a task without an end. People will come
up with variants that will do the right thing but foil `file`,
intentionally or unintentionally -- just as we've seen in this thread.
I have no clue about Tcl, but it was said some postings ago that "all
Tcl-scripts are recognized as sh". If this is true, I think it ought
to be fixed. Searching for one of the very first lines to begin with
#, end with \, and followed by "exec tcl" doesn't sound like
rocket-science to me, though I've neither studied magic(4) or Tcl
scripting conventions.

I do understand that it would still be many ways to start a Tcl-script
without getting recognized by the above logic, as well as ways to
trick file into reporting a real shell script to be a Tcl-script.
IMHO a 90% hit rate is far better than 0% hit rate.
The right way to approach this with `file` is to acknowledge that such
tricks are inherently sh-specific and leave it identified as a sh file.
Because that is, of course, exactly what it is.
A shell script containing some inline tcl is a shell script, though
when the only shell-command is "start up tcl" and the rest of the file
is tcl-code, I really don't think it can be defined as a "shell
script" anymore. Particularly not if almost all tcl-scripts are
started that way.

--
Tobias Brox, 69░42'N, 18░57'E
Aug 9 '06 #21

P: n/a
John Salerno wrote:
I understand the difference, but I'm just curious if anyone has any
strong feelings toward using one over the other?
I personally prefer being explicit over implicit, but then, that is in
the Zen of Python.

I work on machines with multiple interpreters installed. I find picking
the first one in the path to be very annoying. That means that it's
going to work differently for different people based on their PATH.

Mike

Aug 9 '06 #22

P: n/a
Stephan Kuhagen a Úcrit :
>Always prefer to use env over a hardcoded path, because that hardcoded
path will invariably be wrong. (Yes, for those about to nitpick, it's
conceivable that env might be somewhere other than /usr/bin. However,
that is very rare and results in a no-win situations regardless of the
issue of where Python is installed.)

Don't yell at me for bringing in another language, but I really like the
trick, Tcl does:
> #!/bin/sh
# The next line starts Tcl \
exec tclsh "$0" "$@"

This works by the somewhat weird feature of Tcl, that allows comments to be
continued in the next line with "\" at the end of the comment-line. It
looks unfamiliar, but has several advantages, I think. First it's really
VERY unlikely, that there is no /bin/sh (while I found systems with
different places for env), and you can add some other features at or before
the actual call of the interpreter, i.e. finding the right or preferred
version... - This way I coded a complete software-installer, that runs
either as a Tcl/Tk-Script with GUI, or as bash-script, when no Tcl is
available. - I really would like to have something like that for python,
but I did not find a way to achieve that, yet.

Regards
Stephan
What about that one ( windows .bat file ) :

@echo off
rem = """-*-Python-*- script
rem -------------------- DOS section --------------------
rem You could set PYTHONPATH or TK environment variables here
python -x %~f0 %*
goto exit

"""
# -------------------- Python section --------------------
import sys
print sys.argv
DosExitLabel = """
:exit
rem """
I'm sure it can be updated to work on Unix too :)
Aug 9 '06 #23

P: n/a
In article <fj******************@news.tufts.edu>,
John Salerno <jo******@NOSPAMgmail.comwrote:
>
I understand the difference, but I'm just curious if anyone has any
strong feelings toward using one over the other? I was reading that a
disadvantage to the more general usage (i.e. env) is that it finds the
first python on the path, and that might not be the proper one to use. I
don't know if that's a real issue most of the time, but it's at least
something to consider.
The main argument against the env method is that I've seen a fair number
of sysadmins claim that it's less secure. I'm not competent to judget
that claim myself, but I prefer to play safe and stay away from env.
--
Aahz (aa**@pythoncraft.com) <* http://www.pythoncraft.com/

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are, by
definition, not smart enough to debug it." --Brian W. Kernighan
Aug 9 '06 #24

P: n/a
ZeD wrote:
but...

$ cat test.py
#!/usr/bin/env python

print "Hello, world"
$ file test.py
file.py: a python script text executable

following what you said, test.py is a /usr/bin/env script, not a python one.
That's rather a silly interpretation of what I said, I think. After all::

% cat test.gargle
#!/usr/bin/env gargle

asdfasflsafhklsfhklsdfhsdf
max@oxygen:~/tmp% file test.gargle
test.gargle: a gargle script text executable

`file` knows how /usr/bin/env is used because it's not only a standard
Unix idiom, it's the primary purpose for which it exists. That's not
the case with endless variants of /bin/sh scripts whose first statements
involve running another interpreter in some clever way that's not
detected by _that_ interpreter's processing, which is what the
previously discussed tricks amount to.

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Punctuality is the virtue of the bored.
-- Evelyn Waugh
Aug 9 '06 #25

P: n/a
Tobias Brox wrote:
A shell script containing some inline tcl is a shell script, though
when the only shell-command is "start up tcl" and the rest of the file
is tcl-code, I really don't think it can be defined as a "shell
script" anymore. Particularly not if almost all tcl-scripts are
started that way.
The problem is that there are endless ways to do that, and figuring out
all the cases makes `file` an sh interpreter, not the magic number
detector it's supposed to be.

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
Punctuality is the virtue of the bored.
-- Evelyn Waugh
Aug 9 '06 #26

P: n/a

Aahz wrote:
In article <fj******************@news.tufts.edu>,
John Salerno <jo******@NOSPAMgmail.comwrote:

I understand the difference, but I'm just curious if anyone has any
strong feelings toward using one over the other? I was reading that a
disadvantage to the more general usage (i.e. env) is that it finds the
first python on the path, and that might not be the proper one to use. I
don't know if that's a real issue most of the time, but it's at least
something to consider.

The main argument against the env method is that I've seen a fair number
of sysadmins claim that it's less secure. I'm not competent to judget
that claim myself, but I prefer to play safe and stay away from env.
--
Aahz (aa**@pythoncraft.com) <* http://www.pythoncraft.com/
Basically, someone could inject an arbirtrary script called 'python'
into your path that does whatever (rm -fr /) under your user context
when you run the script. But the same thing would happen if you run
'python test.py' instead of '/usr/local/bin/python test.py' to run a
script that doesn't have a she-bang or hasn't been flagged as
executable. Some admins will use a fully-qualified path for every
command to guard against this; I think that can be overkill.

Aug 9 '06 #27

P: n/a
ol*****@verizon.net wrote:
Basically, someone could inject an arbirtrary script called 'python'
into your path that does whatever (rm -fr /) under your user context
when you run the script. But the same thing would happen if you run
'python test.py' instead of '/usr/local/bin/python test.py' to run a
script that doesn't have a she-bang or hasn't been flagged as
executable. Some admins will use a fully-qualified path for every
command to guard against this; I think that can be overkill.
The primary guard for this is not having publicly-writable things in
your PATH. In other words, this is the argument for not putting things
like /tmp or . (because you might cd to somewhere publicly writable like
/tmp) in your PATH, not really for avoiding /usr/bin/env in hash bangs.

--
Erik Max Francis && ma*@alcyone.com && http://www.alcyone.com/max/
San Jose, CA, USA && 37 20 N 121 53 W && AIM erikmaxfrancis
We must all hang together, or, most assuredly, we will all hang
separately. -- John Hancock
Aug 9 '06 #28

P: n/a

Erik Max Francis wrote:
ol*****@verizon.net wrote:
Basically, someone could inject an arbirtrary script called 'python'
into your path that does whatever (rm -fr /) under your user context
when you run the script. But the same thing would happen if you run
'python test.py' instead of '/usr/local/bin/python test.py' to run a
script that doesn't have a she-bang or hasn't been flagged as
executable. Some admins will use a fully-qualified path for every
command to guard against this; I think that can be overkill.

The primary guard for this is not having publicly-writable things in
your PATH. In other words, this is the argument for not putting things
like /tmp or . (because you might cd to somewhere publicly writable like
/tmp) in your PATH, not really for avoiding /usr/bin/env in hash bangs.
Yeah, I agree 100%. I guess the best way to put it is that if random
people are creating scripts called 'python' in your path, you've got
bigger problems than /usr/bin/env calling the wrong program or a plain
old 'python test.py' doing the same.

Aug 9 '06 #29

P: n/a
Erik Max Francis wrote:
The problem is that there are endless ways to do that, and figuring out
all the cases makes `file` an sh interpreter, not the magic number
detector it's supposed to be.
It makes it into a pattern-matcher, not an interpreter. But that it is
already.

But right, there are endless ways to do that, but only one or a very small
subset is common. The way I cited is the way mentioned in the
Tclsh-manpage, so it can be (not must, but can) as the standard-header of a
Tcl-Script on Unix-like systems.

Even if the construct sometimes differs a little bit, "file" should be able
to identify, since the manpage of "file" says, that it is not a pure
magic-number-detector. The first part explains how the magic-number-thing
works and then what is done, when that fails:

"If a file does not match any of the entries in the magic file, it is
examined to see if it seems to be a text file.
[...]
Once file has determined the character set used in a text-type file, it will
attempt to determine in what language the file is written. The language
tests look for particular strings (cf names.h) that can appear anywhere in
the first few blocks of a file. For example, the keyword .br indicates
that the file is most likely a troff(1) input file, just as the keyword
struct indicates a C program. These tests are less reliable than the
previous two groups, so they are performed last."

This is not the most reliable way, as the man-page says, but it should work:
if in the first some blocks you can find a statement with a continued
comment-line and the exec-trick, "file" can at least guess, that it is a
Tcl-file. So this would be a valid method to detect that files type,
because a troff-file is a troff file, even when there is no .br in the
first few blocks, but "file" tries to identify it anyway. "file" is not
meant to be to ignore troff files, just because they are sometimes hard to
detect. The same applies to Tcl-files, I think. Not perfectly reliable, but
worth a try, since it is, according to the Tclsh-manpage, the common
header-pattern for a Tcl-script.

So "file" can not be perfect and reliable in every case, but it should try
to take a good guess. If you do not care about the Tcl-headers (and why
should you, this is comp.lang.python... ;-), you are right with your
reasoning. But if you accept, that file can not be perfect anyway and want
it to be as good as possible, then it is some kind of bug or missing
feature in "file" that it recognizes (or tries to) some morphing file
formats but not another (which is fairly wide spread, even if Tcl is not a
modern buzz-word-language these days).

Stephan
Aug 10 '06 #30

This discussion thread is closed

Replies have been disabled for this discussion.