470,855 Members | 1,158 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 470,855 developers. It's quick & easy.

stdin or optional fileinput

I was writing a small script the other day with the following CLI
prog [options] [file]*

I've used getopt to parse out the possible options, so we'll ignore
that part, and assume for the rest of the discussion that args is a
list of file names (if any provided).

I used this bit of code to detect wether i want stdinput or not.

if len(args)==0:
args = [ sys.stdin ]

Now in my main loop I've written:

for file in args:
for line in open( file ):
#do stuff

The probelm occurs when I pass no arguments and python trys to open(
sys.stdin ).
open() customarily accepts a pathname, and returns a file object.
But this code is so elegant I thought that maybe, if open were passed
file object it could re-open that file in a new mode (if a mode was
provided different from the current mode) or simply return the file
object it was passed.

Anyhow, what do you guys think? There might be something about my idea
that makes this proposed behavior of open() unexpected, and I just
haven't thought of it.

Mar 15 '06 #1
5 2348
the.theorist wrote:
I was writing a small script the other day with the following CLI
prog [options] [file]*

I've used getopt to parse out the possible options, so we'll ignore
that part, and assume for the rest of the discussion that args is a
list of file names (if any provided).

I used this bit of code to detect wether i want stdinput or not.

if len(args)==0:
args = [ sys.stdin ]

Now in my main loop I've written:

for file in args:
for line in open( file ):
#do stuff


You should probably write:

if not args: # note that len(args) == 0 is repetitively redundant, over
# and over again, in a reiterative manner
files = [sys.stdin]
else:
files = (open(filename) for filename in args)

....

for fileobj in files: # using the name 'file' is a bad idea since it
# shadows the builtin 'file'
for line in fileobj:
# do stuff
STeVe
Mar 15 '06 #2

Steven Bethard wrote:
the.theorist wrote:
I was writing a small script the other day with the following CLI
prog [options] [file]*

I've used getopt to parse out the possible options, so we'll ignore
that part, and assume for the rest of the discussion that args is a
list of file names (if any provided).

I used this bit of code to detect wether i want stdinput or not.

if len(args)==0:
args = [ sys.stdin ]

Now in my main loop I've written:

for file in args:
for line in open( file ):
#do stuff


You should probably write:

if not args: # note that len(args) == 0 is repetitively redundant, over
# and over again, in a reiterative manner
files = [sys.stdin]
else:
files = (open(filename) for filename in args)

...

for fileobj in files: # using the name 'file' is a bad idea since it
# shadows the builtin 'file'
for line in fileobj:
# do stuff
STeVe


I'll keep both those in mind for future programs.
my current fix has been

if not args:
args = [ sys.stdin ]
else:
map( open, args )

and then a modification to the main loop, as you proposed.

I thought that one day I might run into a problem opening too many
files though (esp if i used xargs in a pipe). And it just _feels_
better to open and close as I loop over the files list, rather than
open everything at the get go.

OH, i've noticed you used a generator that takes care of that. Except,
the machine I'm working on is currently stuck at Python 2.3. Still,
I'll keep your suggestions in mind for future coding.

Mar 15 '06 #3
the.theorist wrote:
Steven Bethard wrote:
the.theorist wrote:
I was writing a small script the other day with the following CLI
prog [options] [file]*

I've used getopt to parse out the possible options, so we'll ignore
that part, and assume for the rest of the discussion that args is a
list of file names (if any provided).

I used this bit of code to detect wether i want stdinput or not.

if len(args)==0:
args = [ sys.stdin ]

Now in my main loop I've written:

for file in args:
for line in open( file ):
#do stuff
You should probably write:

if not args: # note that len(args) == 0 is repetitively redundant, over
# and over again, in a reiterative manner
files = [sys.stdin]
else:
files = (open(filename) for filename in args)

...

for fileobj in files: # using the name 'file' is a bad idea since it
# shadows the builtin 'file'
for line in fileobj:
# do stuff
STeVe

I'll keep both those in mind for future programs.
my current fix has been

if not args:
args = [ sys.stdin ]
else:
map( open, args )

and then a modification to the main loop, as you proposed.

I thought that one day I might run into a problem opening too many
files though (esp if i used xargs in a pipe). And it just _feels_
better to open and close as I loop over the files list, rather than
open everything at the get go.

Yes, that feels most Pythonic.
OH, i've noticed you used a generator that takes care of that. Except,
the machine I'm working on is currently stuck at Python 2.3. Still,
I'll keep your suggestions in mind for future coding.

Well the nice thing about the generator expression is precisely its
deferral of applying open() to the filename argument until the iterable
element is extracted. But as you observe there's no use trying that in
Python 2.3, so you'll have to adopt a slightly less elegant solution for
now.

The important thing is that you seem to be well in touch with the
various issues, so there's a good chance you won't go wrong.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd www.holdenweb.com
Love me, love my blog holdenweb.blogspot.com

Mar 15 '06 #4
"the.theorist" wrote:
I used this bit of code to detect wether i want stdinput or not.

if len(args)==0:
args = [ sys.stdin ]

Now in my main loop I've written:

for file in args:
for line in open( file ):
#do stuff

The probelm occurs when I pass no arguments and python trys to open(
sys.stdin ).
open() customarily accepts a pathname, and returns a file object.
But this code is so elegant I thought that maybe, if open were passed
file object it could re-open that file in a new mode (if a mode was
provided different from the current mode) or simply return the file
object it was passed.


so use your own open function:

def myopen(file):
if isinstance(file, basestring):
return open(file)
return file

for file in args:
for line in myopen( file ):
#do stuff
or perhaps

def myopen(file):
if hasattr(file, "read"):
return file
return open(file)

...

or perhaps

if not args:
args = [ "-" ]

def myopen(file):
if file == "-":
return sys.stdin
return open(file)

for file in args:
for line in myopen( file ):
#do stuff

(the "-" = stdin convention is quite common)

</F>

Mar 15 '06 #5
the.theorist wrote:
I'll keep both those in mind for future programs.
my current fix has been

if not args:
****args*=*[*sys.stdin*]
else:
****map(*open,*args*)

and then a modification to the main loop, as you proposed.

I thought that one day I might run into a problem opening too many
files though (esp if i used xargs in a pipe). And it just feels
better to open and close as I loop over the files list, rather than
open everything at the get go.

OH, i've noticed you used a generator that takes care of that. Except,
the machine I'm working on is currently stuck at Python 2.3. Still,
I'll keep your suggestions in mind for future coding.


use

if args:
args = itertools.imap(open, args)
else:
args = [sys.stdin]

to get the same lazy-open behaviour as with the generator expression.
Or have a look at http://docs.python.org/lib/module-fileinput.html.

Peter
Mar 15 '06 #6

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

reply views Thread by Daniel Yoo | last post: by
2 posts views Thread by Jon Schull | last post: by
8 posts views Thread by max(01)* | last post: by
7 posts views Thread by Will McDonald | last post: by
10 posts views Thread by wo_shi_big_stomach | last post: by
4 posts views Thread by Adam Funk | last post: by
11 posts views Thread by jo3c | last post: by
3 posts views Thread by Robert | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.