470,604 Members | 2,049 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 470,604 developers. It's quick & easy.

Process multiple files

Say I have multiple text files in a single directory, for illustration
they are called "spam.txt" and "eggs.txt". All of these text files are
organized in exactly the same way. I have written a program that parses
each file one at a time. In other words, I need to run my program each
time I want to process one of these files.

However, because I have hundreds of these files I would like to be able
to process them all in one fell swoop. The current program is something
like this:

sample.py
new_file = open('filename.txt', 'w')
params = open('eggs.txt', 'r')
do all the python stuff here
new_file.close()

If these files followed a naming convention such as 1.txt and 2.txt I
can easily see how these could be parsed consecutively in a loop.
However, they are not and so is it possible to modify this code such
that I can tell python to parse all .txt files in a certain directory
and then to save them as separate files? For instance, using the example
above, python would parse both spam.txt and eggs.txt and then save 2
different files, say as spam_parsed.txt and eggs_parsed.txt.

Thanks


Jun 27 '08 #1
1 1955
"Doran, Harold" <HD****@air.orgwrote:
If these files followed a naming convention such as 1.txt and 2.txt I
can easily see how these could be parsed consecutively in a loop.
However, they are not and so is it possible to modify this code such
that I can tell python to parse all .txt files in a certain directory
and then to save them as separate files? For instance, using the example
above, python would parse both spam.txt and eggs.txt and then save 2
different files, say as spam_parsed.txt and eggs_parsed.txt.
This isn't exactly what you asked for, but have you looked at the fileinput
module: it will process a list of filenames pointing stdin at each file in
turn, optionally it can work inplace, but without that option I think it is
pretty close to what you want (warning, untested code follows):

output = None
for line in fileinput.input(n for n in glob.glob('*.txt')
if not n.endswith('_parsed.txt')):

if fileinput.isfirstline():
if output: output.close()
output = open(fileinput.filename().replace('.', '_parsed.', 1),
'w')

print >>output, "parsed:", line.strip()

if output:
output.close()
Jun 27 '08 #2

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

2 posts views Thread by Jeffrey Palermo, MCAD.Net | last post: by
reply views Thread by Paul Anton Letnes | last post: by
7 posts views Thread by dieter | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.