By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
440,171 Members | 809 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 440,171 IT Pros & Developers. It's quick & easy.

How to average column data in txt files?

P: 11
80409 1110 16.77 19.2 216.5 -9.97 -13.48

My data resembles the above. I'm trying to average all columns except the first two, which are date columns. I need to average when the values in column 1 (the 1110 value) equal to 1100 or 1200, 1300, etc). I need to write the averaged columns to an output file. Thanks. Code sample below.
Expand|Select|Wrap|Line Numbers
  1. >>> fileIN = open('c:\TestColData.txt','r')
  2. >>> line = fileIN.readlines()
  3. >>> i = 0
  4. >>> while line:
  5.     line = fileIN.readlines()
  6.     i = i + 1
  7.     third_col = float(line[i].split( )[2])
  8.     print line
  9.     print third_col


Traceback (most recent call last):
File "<pyshell#496>", line 4, in <module>
third_col = float(line[i].split( )[2])
IndexError: list index out of range
Dec 15 '10 #1
Share this Question
Share on Google+
8 Replies


bvdet
Expert Mod 2.5K+
P: 2,851
Create a file object as you did, but iterate on the file object instead of calling readlines(). Untested:
Expand|Select|Wrap|Line Numbers
  1. for line in fileObj:
  2.     lineList = line.strip().split()
  3.     if lineList[1] in ['1100', '1200', '1300']:
  4.         list_to_average = [float(s) for s in lineList[2:]]
  5.         average = sum(list_to_average)/len(list_to_average)
Dec 16 '10 #2

P: 11
I tried your solution and this is what i got:


>>> fileObj = open('c:\TestColData.txt','r')
>>> for line in fileObj:
lineList = line.strip().split()
if lineList[1] in ['1200', '1300']:
list_to_average = [float(s) for s in lineList[2:]]
average = sum(list_to_average)/len(list_to_average)
print average



Traceback (most recent call last):
File "<pyshell#506>", line 5, in <module>
average = sum(list_to_average)/len(list_to_average)
TypeError: 'float' object is not callable
Dec 16 '10 #3

bvdet
Expert Mod 2.5K+
P: 2,851
Let's test it:
Expand|Select|Wrap|Line Numbers
  1. >>> line = "80409 1100 16.77 19.2 216.5 -9.97 -13.48\n"
  2. >>> lineList = line.strip().split()
  3. >>> if lineList[1] in ['1100', '1200', '1300']:
  4. ...     list_to_average = [float(s) for s in lineList[2:]]
  5. ...     average = sum(list_to_average)/len(list_to_average)
  6. ...     
  7. >>> print average
  8. 45.804
  9. >>> 
Maybe you used sum or len as an identifier for a float.
Dec 16 '10 #4

P: 11
hmmm...still getting an error and typed in exactly as you suggested.

Traceback (most recent call last):
File "<pyshell#522>", line 3, in <module>
average = sum(list_to_average)/len(list_to_average)
TypeError: 'float' object is not callable

Also, i need each column averaged or columns 2 thru 6 not just one average of all data in columns 2 thru 6. Could you insert comments? Thanks!!!
Dec 16 '10 #5

bvdet
Expert Mod 2.5K+
P: 2,851
This illustrates your error, I believe:
Expand|Select|Wrap|Line Numbers
  1. >>> len = 8.0
  2. >>> len([1,2,3,4,5])
  3. Traceback (most recent call last):
  4.   File "<interactive input>", line 1, in ?
  5. TypeError: 'float' object is not callable
  6. >>> del len
  7. >>> len([1,2,3,4,5])
  8. 5
  9. >>> 
Dec 16 '10 #6

bvdet
Expert Mod 2.5K+
P: 2,851
The code I posted will not average a column of numbers. You will need to compile a list of each column (maybe a list of lists), then you can average each column list after reaching the end of the file.
Dec 16 '10 #7

P: 11
Hi again. Would you be able to tell me what is "s" in your code:
list_to_average = [float(s) for s in lineList[2:]]

Thanks,
Randall
Dec 16 '10 #8

bvdet
Expert Mod 2.5K+
P: 2,851
Sure. "s" is a temporary variable used in the list comprehension. In the example, each element of lineList is in turn assigned to "s". Example:
Expand|Select|Wrap|Line Numbers
  1. >>> lineList = ['1', '2', '3', '4', '5']
  2. >>> [float(s) for s in lineList]
  3. [1.0, 2.0, 3.0, 4.0, 5.0]
  4. >>> s
  5. '5'
  6. >>> 
An equivalent for loop:
Expand|Select|Wrap|Line Numbers
  1. >>> newList = []
  2. >>> for s in lineList:
  3. ...     newList.append(float(s))
  4. ...     
  5. >>> print newList
  6. [1.0, 2.0, 3.0, 4.0, 5.0]
  7. >>> 
Dec 16 '10 #9

Post your reply

Sign in to post your reply or Sign up for a free account.