469,362 Members | 2,441 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,362 developers. It's quick & easy.

Formatting text files

Im starting new project where I have to take data from one text file and toss it into another is a specified format ( i have a 27 page document outlining that format). I am new to python and was wondering if anyone had some tips on usefull ways to parse text files using python?
Jan 5 '07 #1
5 6052
bvdet
2,851 Expert Mod 2GB
Im starting new project where I have to take data from one text file and toss it into another is a specified format ( i have a 27 page document outlining that format). I am new to python and was wondering if anyone had some tips on usefull ways to parse text files using python?
I'm no expert on parsing data, but regular expressions (module re) is a powerful tool.
Jan 5 '07 #2
bartonc
6,596 Expert 4TB
Im starting new project where I have to take data from one text file and toss it into another is a specified format ( i have a 27 page document outlining that format). I am new to python and was wondering if anyone had some tips on usefull ways to parse text files using python?
Python has many powerful and easy to use tools for dealing with text. So many, in fact, that picking the tools is sometimes the hard part. After getting a grasp on the language syntax and structures, parsing text is fairly easy to implement.
Jan 6 '07 #3
dshimer
136 Expert 100+
In my mind it also depends a great deal on the structure of the text. I do this task constantly and because of the setting I am in I deal primarily with two types of data files.

One is text or numeric data that is fairly structured, for example each line may contain a variety of numbers or strings delimited by a character or white space. For example to describe a point in space, a file may have hundreds of lines with
Name,X,Y,Z,R1,R2,R3,Desc
Where Name is the name of a point, x,y,z, are geographic coordinates and the R's are some kind of real numbers, and Desc is a text description. When working with these kinds of files I find it easiest to take the data from the input file and create a list of lists then just loop over the lists outputing the data in the new format. I usually grab the whole file using readlines() then run it through a function I call lines2lists which works on data separated by whitespace but could easily be modified to add a delimiter variable.

Expand|Select|Wrap|Line Numbers
  1. def lines2lists(AListOfDataLines):
  2.     '''
  3.     Function readlines returns an entire file with each line as a string in a
  4.     list of data.  This function will convert each string into a list of words,
  5.     then return a list of lists.
  6.     Example:            lines2lists(['first line','the second line','or sentences'])
  7.     Would return:       [['first','line'],['the','second','line'],['or','sentences]]
  8.     '''
  9.     DataList=[]
  10.     for Line in AListOfDataLines:
  11.         DataList.append(Line.split())
  12.     return DataList
  13.  
I usually combine it with readlines(), for example
Expand|Select|Wrap|Line Numbers
  1. Data=utilitymodule.lines2lists(AnOpenFile.readlines())
Where Data is the resulting list in which the example above would look something like the following.
[[Name,X,Y,Z,R1,R2,R3,Desc],[Name,X,Y,Z,R1,R2,R3,Desc],[Name,X,Y,Z,R1,R2,R3,Desc]]

The for each element in Data I format and output the elements of each coordinate (in this case).

The other type of file I work with a lot is one in which there may be multiple elements of a particular dataset but they are on different lines of the file. Using the example above it might look like.
Name1 aString
Name2 aString
X1 aNumber
X2 aNumber
Y1 aNumber
Y2 aNumber
and so on....

In this case I still read in the lines, build lists from them, then use something like the count() function to test for a value in a particular list, if the test value exists, I grab the other member of the list which is the actual data then append it to the actual list of data. In pseudocode it would be something like

If InputString.count(test value like name1) is true then datastring.append( the value associated with name1) then when all the input strings have been parsed. for each element of datastring, format and output the values.

There are probably easier ways to do this, but most of the time I may only need to write a program every couple of weeks, may only have 5 minutes notice and need to have it done very quickly. Since there is so much power in lists I tend to stick with the functions I know and love and can hack together quickly.
Jan 9 '07 #5
bvdet
2,851 Expert Mod 2GB
dshimer,
Whatever works for you in a efficient manner IS the easiest way. Here are two things that I learned on this forum:
Expand|Select|Wrap|Line Numbers
  1. f = open(dlg1.import_file, "r")
  2. # Files can be used as iterators. Internally, for calls file next() method.
  3. for item in f:
  4.     ....do stuff....
Expand|Select|Wrap|Line Numbers
  1. # If a sub-string is in a string, evaluate True
  2. if "subject_text" in item.lower():
  3.     pt = re.split('[:;,]', item)
Maybe you can use these sometime.
Jan 9 '07 #6

Post your reply

Sign in to post your reply or Sign up for a free account.

Similar topics

4 posts views Thread by Bradley | last post: by
8 posts views Thread by coleenholley | last post: by
6 posts views Thread by Brad | last post: by
2 posts views Thread by D. Shane Fowlkes | last post: by
8 posts views Thread by alamb200 | last post: by
14 posts views Thread by Scott M. | last post: by
1 post views Thread by CARIGAR | last post: by
reply views Thread by zhoujie | last post: by
reply views Thread by suresh191 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.