In my mind it also depends a great deal on the structure of the text. I do this task constantly and because of the setting I am in I deal primarily with two types of data files.
One is text or numeric data that is fairly structured, for example each line may contain a variety of numbers or strings delimited by a character or white space. For example to describe a point in space, a file may have hundreds of lines with
Name,X,Y,Z,R1,R2,R3,Desc
Where Name is the name of a point, x,y,z, are geographic coordinates and the R's are some kind of real numbers, and Desc is a text description. When working with these kinds of files I find it easiest to take the data from the input file and create a list of lists then just loop over the lists outputing the data in the new format. I usually grab the whole file using readlines() then run it through a function I call lines2lists which works on data separated by whitespace but could easily be modified to add a delimiter variable.
- def lines2lists(AListOfDataLines):
-
'''
-
Function readlines returns an entire file with each line as a string in a
-
list of data. This function will convert each string into a list of words,
-
then return a list of lists.
-
Example: lines2lists(['first line','the second line','or sentences'])
-
Would return: [['first','line'],['the','second','line'],['or','sentences]]
-
'''
-
DataList=[]
-
for Line in AListOfDataLines:
-
DataList.append(Line.split())
-
return DataList
-
I usually combine it with readlines(), for example
- Data=utilitymodule.lines2lists(AnOpenFile.readlines())
Where Data is the resulting list in which the example above would look something like the following.
[[Name,X,Y,Z,R1,R2,R3,Desc],[Name,X,Y,Z,R1,R2,R3,Desc],[Name,X,Y,Z,R1,R2,R3,Desc]]
The for each element in Data I format and output the elements of each coordinate (in this case).
The other type of file I work with a lot is one in which there may be multiple elements of a particular dataset but they are on different lines of the file. Using the example above it might look like.
Name1 aString
Name2 aString
X1 aNumber
X2 aNumber
Y1 aNumber
Y2 aNumber
and so on....
In this case I still read in the lines, build lists from them, then use something like the count() function to test for a value in a particular list, if the test value exists, I grab the other member of the list which is the actual data then append it to the actual list of data. In pseudocode it would be something like
If InputString.count(test value like name1) is true then datastring.append( the value associated with name1) then when all the input strings have been parsed. for each element of datastring, format and output the values.
There are probably easier ways to do this, but most of the time I may only need to write a program every couple of weeks, may only have 5 minutes notice and need to have it done very quickly. Since there is so much power in lists I tend to stick with the functions I know and love and can hack together quickly.
1
