Is pyton for me?

[Mark de+la+Fuente]

I need to write simple scripts for executing command line functions. Up till now I've used C-Shell scripts for this, but I'm looking for a better alternative. And I keep reading about how “easy” it is to program with python.

Unfortunately after reading "diveintopython" and the python tutorial, I'm not sure how to do what I need to do.

Here is an example of the type of thing I would like to be able to do. Can I do this with python? How do I get python to execute command line functions?

From the command prompt (not from the python interpreter – running linux for example) I would type:

gensky 3 21 12 > sky.rad

This would call the non-python program “gensky” which creates a computer model of a sky on march 21st at 12:00 PM and puts this information into a file called sky.rad.

If I create a CSH script like the one below, I can automate tasks by building loops along the lines of:

#!/bin/csh -fv

# simple script to create multiple sky files.

foreach hour (10 12 14)
gensky 3 21 $hour > sky$hour.rad
end

If I try and do a gensky command from the python interpreter or within a python.py file, I get an error message:

NameError: name ‘gensky’ is not defined

So even if I get the python syntax correct, I can’t get these commands to execute.

If anyone has any suggestions on how to get python scripts to execute this sort of thing, what I should be looking at, or if there is something else I might consider besides python, please let me know.

Thanks for your help.

Mark

_______________________________________________
Join Excite! - http://www.excite.com
The most personalized portal on the Web!

[Dan Bishop]

Mark de+la+Fuente wrote:

I need to write simple scripts for executing command line functions. Up till
now I've used C-Shell scripts for this, but I'm looking for a better
alternative. And I keep reading about how "easy" it is to program with
python.

Unfortunately after reading "diveintopython" and the python tutorial, I'm not
sure how to do what I need to do.

Here is an example of the type of thing I would like to be able to do.
Can I do this with python? How do I get python to execute command line
functions?

import os
os.system('gensky 3 21 12 > sky.rad')

[Andrew Dalke]

Mark de la Fuente wrote:

Here is an example of the type of thing I would like to be able to do.
Can I do this with python? How do I get python to execute command line
functions? ... # simple script to create multiple sky files.

foreach hour (10 12 14)
gensky 3 21 $hour > sky$hour.rad
end
Dan Bishop gave one example using os.system. The important
thing to know is that in the shell all programs can be used
as commands while in Python there isn't a direct connection.
Instead you need to call a function which translates a
request into something which calls the command-line program.

There are several ways to do that. In Python before 2.4
the easiest way is with os.system(), which takes the command-line
text as a string. For example,

import os
os.system("gensky 3 21 10 > sky10.rad")

You could turn this into a Python function rather easily

import os

def gensky(hour):
os.system("gensky 3 21 %d > sky%d.rad" % (hour, hour))

for hour in (10, 12, 14):
gensky(hour)
Python 2.4 introduces the subprocess module which makes it
so much easier to avoid nearly all the mistakes that can
occur in using os.system(). You could replace the 'gensky'
python function with

import subprocess
def gensky(hour):
subprocess.check_call(["gensky", "3", "21", str(hour)],
stdout = open("sky%d.rad" % (hour,), "w"))
The main differences here are:
- the original code didn't check the return value of os.system().
It should do this because, for example, the gensky program might
not be on the path. The check_call does that test for me.

- I needed to do the redirection myself. (I wonder if the
subprocess module should allow

if isinstance(stdout, basestring):
stdout = open(stdout, "wb")

Hmmm....)

If I try and do a gensky command from the python interpreter or within a
python.py file, I get an error message:

NameError: name ‘gensky’ is not defined
That's because Python isn't set up to search the command path
for an executable. It only knows about variable names defined
in the given Python module or imported from another Python
module.
If anyone has any suggestions on how to get python scripts to execute
this sort of thing, what I should be looking at, or if there is
something else I might consider besides python, please let me know.

You'll have to remember that Python is not a shell programming
language. Though you might try IPython - it allows some of the
things you're looking for, though not all.

You should also read through the tutorial document on Python.org
and look at some of the Python Cookbook.. Actually, start with
http://wiki.python.org/moin/BeginnersGuide

Andrew
da***@dalkescientific.com

[Dennis Lee Bieber]

On Thu, 9 Jun 2005 22:50:20 -0400 (EDT), "Mark de+la+Fuente"
<de****@excite.com> declaimed the following in comp.lang.python:

foreach hour (10 12 14)
gensky 3 21 $hour > sky$hour.rad
end

import os

for hour in [10, 12, 14]:
os.system("gensky 3 21 %s > sky%s.rad" % (hour, hour))
Python is a stand-alone language. External programs have to be
explicitly invoked.
-- ================================================== ============ <
wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
wu******@dm.net | Bestiaria Support Staff <
================================================== ============ <
Home Page: <http://www.dm.net/~wulfraed/> <
Overflow Page: <http://wlfraed.home.netcom.com/> <

[Kent Johnson]

Andrew Dalke wrote:

import subprocess
def gensky(hour):
subprocess.check_call(["gensky", "3", "21", str(hour)],
stdout = open("sky%d.rad" % (hour,), "w"))
The main differences here are:
- the original code didn't check the return value of os.system().
It should do this because, for example, the gensky program might
not be on the path. The check_call does that test for me.

Where do you find check_call()? It's not in the docs and I get

import subprocess
subprocess.check_call

Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'module' object has no attribute 'check_call'

with Python 2.4.1.

Kent

[Andrew Dalke]

Kent Johnson wrote:

Where do you find check_call()? It's not in the docs and I get

>>> import subprocess
>>> subprocess.check_call

Traceback (most recent call last):
File "<stdin>", line 1, in ?
AttributeError: 'module' object has no attribute 'check_call'

with Python 2.4.1.

Interesting. I got my subprocess.py from CVS. The CVS log

revision 1.12
date: 2005/01/01 09:36:34; author: astrand; state: Exp; lines: +39 -1
New subprocess utility function: check_call. Closes #1071764.

The bug tracker is
http://sourceforge.net/tracker/index...70&atid=305470

which says it's a 2.5ism. Oops! Sorry about that post
from the future. I didn't realize it.

Andrew
da***@dalkescientific.com

[Mike Meyer]

"Mark de+la+Fuente" <de****@excite.com> writes:

I need to write simple scripts for executing command line functions.
Up till now I've used C-Shell scripts for this, but I'm looking for
a better alternative. And I keep reading about how “easy” it is to
program with python.

Lots of answers - but everyone concentrated on telling you how to do
what you were trying to do, and no one answered your question. Which
is actually typical of the c.l.python, but may not be what you want.

As others pointed out, Python isn't a shell, or even a shell scripting
language, so it doesn't handle what you're doing in a "natural"
way. Because of that, it may not be the language for you. Python has
features that work well for building large systems, but those tend to
cause extra work when you want to do things that other languages make
simple.

One thing you might consider is changing shells. See <URL:
http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/ > for reasons why
you might want to drop csh. There's a similar paper discussing the
problems with later versions of sh (bash, late ksh, etc.), but from
what I recall the problems mostly come from trying to use the extra
features that have been added to those shells. So when writing sh
scripts, I tend to stay with fundamental bourne features.

Other languages have less overhead in invoking external commands. Rexx
and Perl come to mind. You might want to give them a look.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

[Drazen Gemic]

Honestly, I don't think that Python is for you. Python is an object
programming language, and you are looking for simple automation tool,
in order to avoid programming. If shell scripting is no good enough, try
tcl.

DG

[Dennis Lee Bieber]

On Fri, 10 Jun 2005 17:50:15 -0500, Mike Meyer <mw*@mired.org> declaimed
the following in comp.lang.python:

Other languages have less overhead in invoking external commands. Rexx
and Perl come to mind. You might want to give them a look.
Especially Rexx -- since any "statement" that is not recognized
as a part of the Rexx language will be automatically kicked out to the
currently addressed command host (typically a shell). Though one may
need to be wary of Rexx's tendency to uppercase everything <G>
-- ================================================== ============ <
wl*****@ix.netcom.com | Wulfraed Dennis Lee Bieber KD6MOG <
wu******@dm.net | Bestiaria Support Staff <
================================================== ============ <
Home Page: <http://www.dm.net/~wulfraed/> <
Overflow Page: <http://wlfraed.home.netcom.com/> <

[Kirk Job Sluder]

Mike Meyer <mw*@mired.org> writes:

"Mark de+la+Fuente" <de****@excite.com> writes:

I need to write simple scripts for executing command line functions.
Up till now I've used C-Shell scripts for this, but I'm looking for
a better alternative. And I keep reading about how “easy” it is to
program with python.
As others pointed out, Python isn't a shell, or even a shell scripting
language, so it doesn't handle what you're doing in a "natural"
way. Because of that, it may not be the language for you. Python has
features that work well for building large systems, but those tend to
cause extra work when you want to do things that other languages make
simple.

I agree with this approach. For me, there has to be a certain level of
complexity before I reach for python as a tool. This usually involves
one or more of the following:

1: text processing involving multiple files. For example, running
statistics on file A, looking up values stored in file B.

2: cases where the overhead of repeatedly opening processes and pipes
becomes unacceptable.
For example:
for f in file/*; do
cp $f $f.bak
sed -e 'something' < $f.bak > $f
done

This works well with small numbers of files. But even though sed is
quicker than python, starting a new sed process with every iteration
quickly stacks up. Rewriting the entire thing to run as a single
process can dramatically improve performance. Although this might be a
case of premature optimization.

3: cases where figuring out how to do something using one of the POSIX
shell utilities makes my head hurt.

Personally, I hate popen and avoid using it when possible. There is
nothing wrong with sh as a glue language when all you need is something
like: grep text file | filter | filter > output_file.
<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.

--
Kirk Job-Sluder
"The square-jawed homunculi of Tommy Hilfinger ads make every day an
existential holocaust." --Scary Go Round

[Peter Hansen]

Kirk Job Sluder wrote:

I agree with this approach. For me, there has to be a certain level of
complexity before I reach for python as a tool.

For me as well. In my case, the key question is "is this bash script
going to be longer than two lines?". If it would be, the Python
approach will be done faster and will be more maintainable, since I
often don't look at those scripts again for months and would tend to
forget any bash syntax that I've learned in the meantime.

:-)

-Peter

[Mike Meyer]

Peter Hansen <pe***@engcorp.com> writes:

Kirk Job Sluder wrote:

I agree with this approach. For me, there has to be a certain level of
complexity before I reach for python as a tool.

For me as well. In my case, the key question is "is this bash script
going to be longer than two lines?". If it would be, the Python
approach will be done faster and will be more maintainable, since I
often don't look at those scripts again for months and would tend to
forget any bash syntax that I've learned in the meantime.

I tend to switch from sh to python when I start writing a loop. A long
list of repeated commands - even with conditionals - isn't to bad. But
when you start forking processes in a loop, it's time to use a more
powerful tool.

<mike
--
Mike Meyer <mw*@mired.org> http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.