Reading from file to multiple lists

[Sushi]

Hello again all! (and happy easter if you are that way inclined)

I have this file called cameratrack.txt:

0  (0  ,0  ,0  )
5  (5  ,0  ,0  )
10 (10 ,0  ,0  )
15 (15 ,5  ,0  )
20 (20 ,10 ,0  )
25 (25 ,15 ,0  )
30 (30 ,20 ,5  )
35 (30 ,25 ,10 )
40 (30 ,30 ,15 )
45 (30 ,30 ,20 )
50 (30 ,30 ,25 )
 

And basically it is frame numbers and co-ordinates for a 3d object animation. I need to put each line's data into 4 different lists, which are frame, cordx, cordy and cordz. So for example line 2 would be frame[5], cordx[5], cordy[0], cordz[0]. I've got a solution with the following code:

#######camera tracking##########
import string
 
frame = []    # The frame number of animation
cordx = []    # The x position of object
cordy = []    # The y position of object
cordz = []    # The z position of object
 
f = open('C:\rest of filepath\cameratrack.txt', 'r')
 
data = f.readlines()        # Reading of the file
datalength = len(data[0])    # Length of each line
filelength = len(data)        # Length of file
 
print data
print datalength
print filelength
 
i = 0
 
while i != filelength:
    frameppoints = [data[i][0], data[i][1], data[i][2]] 
    framenum= float(string.join(frameppoints,''))    # Join to make frame number
    frame.insert(i,framenum)                  # num inserted into list
 
 
    xpoints = [data[i][4], data[i][5], data[i][6]]
    xcords = float(string.join(xpoints,''))
    cordx.insert(i,xcords)
 
    ypoints = [data[i][8],data[i][9], data[i][10]]
    ycords = float(string.join(ypoints,''))
    cordy.insert(i,ycords)
 
    zpoints = [data[i][12], data[i][13],data[i][14]]
    zcords = float(string.join(zpoints,''))
    cordz.insert(i,zcords)
 
    i+=1
 
print frame
print cordx
print cordy
print cordz
 

but was wondering if I've gone round a long way of doing this? I've got the string joining from this thread:
http://www.thescripts.com/forum/threadnav621769-2-10.html
but was wondering if anyone could see an obvious improvement on the code that Im missing? Any help would be appreciated, if not no worries!

Thanks!
Mel

[bartonc]

Hi, Mel. I'll let bvdet grab this one (he loves these kind of exercises).

I will point out that the string module is a depricated module:

import string
framenum= float(string.join(frameppoints,''))    # Join to make frame number
 

should be

framenum= float(''.join(frameppoints))    # Join to make frame number
 

The string module has some handy constants in it, but str type variables and literals have all the methods (defined as funcions in the string module) built into them.

Also, I prefer to iterate through a file object by line. And since you don't do anything fancy with the variable, i,

frame.insert(i,framenum)

is the same as

frame.append(framenum)

So this cleans up like this:

f = open('C:\rest of filepath\cameratrack.txt', 'r')
 
## data = f.readlines()        # Reading of the file
## datalength = len(data[0])    # Length of each line
f## ilelength = len(data)        # Length of file
 
## print data
## print datalength
## print filelength   ### this is actually the length of the list created by readlines()
 
## i = 0
 
for line in f:
    print line

[Sushi]

Ah I see, that makes it neater. Will incorporate that into my code. Thanks barton!

[bartonc]

Ah I see, that makes it neater. Will incorporate that into my code. Thanks barton!

Sorry, I was editing my post so you missed some.

[bvdet]

Here's a couple of ways.............
Data -

0  (0  ,0  ,0  )
5  (5  ,0  ,0  )
10 (10 ,0  ,0  )
15 (15 ,5  ,0  )
20 (20 ,10 ,0  )
25 (25 ,15 ,0  )
30 (30 ,20 ,5  )
35 (30 ,25 ,10 )
40 (30 ,30 ,15 )
45 (30 ,30 ,20 )
50 (30 ,30 ,25 )

import re
 
fn = r'H:\TEMP\temsys\cameratrack.txt'
patt = r'\d+'
fList = open(fn).readlines()
data = [re.findall(patt,item) for item in fList]
 
data1 = [[], [], [], []]
 
for i in range(4):
    for item in data:
        data1[i].append(int(item[i]))
 
for lst in data1:
    print lst
 
print
data2 = [[int(item[i]) for i in range(4) for item in data][j:j+len(data)] for j in range(0,len(data)*4,len(data))]
 
for lst in data2:
    print lst
 
'''
>>> [0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50]
[0, 5, 10, 15, 20, 25, 30, 30, 30, 30, 30]
[0, 0, 0, 5, 10, 15, 20, 25, 30, 30, 30]
[0, 0, 0, 0, 0, 0, 5, 10, 15, 20, 25]
 
[0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50]
[0, 5, 10, 15, 20, 25, 30, 30, 30, 30, 30]
[0, 0, 0, 5, 10, 15, 20, 25, 30, 30, 30]
[0, 0, 0, 0, 0, 0, 5, 10, 15, 20, 25]
>>>
'''

.....and to apply to your list names:

>>> varList = ['frame', 'coordx', 'coordy', 'coordz']
>>> for i, v in enumerate(varList):
...     exec '%s = %s' % (v, data2[i])
...     
>>> frame
[0, 5, 10, 15, 20, 25, 30, 35, 40, 45, 50]
>>> coordx
[0, 5, 10, 15, 20, 25, 30, 30, 30, 30, 30]
>>> coordy
[0, 0, 0, 5, 10, 15, 20, 25, 30, 30, 30]
>>> coordz
[0, 0, 0, 0, 0, 0, 5, 10, 15, 20, 25]
>>> 
 

HTH :)

[bvdet]

I actually prefer to compile data like this in a dictionary:

import re
 
fn = r'H:\TEMP\temsys\cameratrack.txt'
patt = r'\d+'
fList = open(fn).readlines()
data = [re.findall(patt,item) for item in fList]
 
dataDict = dict(zip(['frame', 'coordx', 'coordy', 'coordz'], [[],[],[],[]]))
for i, key in enumerate(dataDict):
    for item in data:
        dataDict[key].append(item[i])
 
for key in dataDict:
    print '%s = %s' % (key, dataDict[key])
 
'''
frame = ['0', '5', '10', '15', '20', '25', '30', '35', '40', '45', '50']
coordz = ['0', '5', '10', '15', '20', '25', '30', '30', '30', '30', '30']
coordy = ['0', '0', '0', '5', '10', '15', '20', '25', '30', '30', '30']
coordx = ['0', '0', '0', '0', '0', '0', '5', '10', '15', '20', '25']
'''

[ghostdog74]

From what I see at your output, you basically wants to "transpose" your data from column to row. here's another way,

store = [] #store results
for line in open("cameratrack.txt"):
    line = line.split(" ",1) 
    coord = map(float,eval(line[1]))
    frame = float(line[0])
    coord.insert(0,frame)
    store.append(coord)
for i in  zip(*store):
    print i
 

output:

# ./test.py
(0.0, 5.0, 10.0, 15.0, 20.0, 25.0, 30.0, 35.0, 40.0, 45.0, 50.0)
(0.0, 5.0, 10.0, 15.0, 20.0, 25.0, 30.0, 30.0, 30.0, 30.0, 30.0)
(0.0, 0.0, 0.0, 5.0, 10.0, 15.0, 20.0, 25.0, 30.0, 30.0, 30.0)
(0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 5.0, 10.0, 15.0, 20.0, 25.0)
 

[ghostdog74]

here's another way. assuming your data is all the same format. the open bracket is replaced with "," and the close bracket removed.

data = open("cameratrack").read().split("\n")
result = [ eval(i.replace("(",",").replace(")","")) for i in data]
for j in  zip(*result):
    print j
 

[bvdet]

Well, I have learned something new today. Thanks GD. :) Here are a couple more variations using zip(*arg):
Lists -

import re
 
fn = 'cameratrack.txt'
patt = r'\d+'
varList = ['frame', 'coordx', 'coordy', 'coordz']
 
data = zip(*[re.findall(patt,item) for item in open(fn).readlines()])
 
for i, v in enumerate(varList):
    exec '%s = %s' % (v, [float(i) for i in data[i]])

Dictionary -

data = zip(*[re.findall(patt,item) for item in open(fn).readlines()])
 
dataDict = {}
 
for i, key in enumerate(varList):
    dataDict[key] = [float(i) for i in data[i]]

[Sushi]

Thanks for all the input guys!

I will have to have a proper look at the codes (i dont fully understand some of them on first glance) but will pop back if I have any queries!
Thanks again!
Mel

[Sushi]

Right I've had some time to look at these codes.

data = open("C:\Documents and Settings\Mel\Desktop\Python2\Blender\cameratrack.txt").read().split("\n")
result = [ eval(i.replace("(",",").replace(")","")) for i in data]
for j in  zip(*result):
    print j

I think I understand the code, but cannot work out how to assign the list names so I can recall them later. I tried using some of bvdet code but ended up applying a line as a list:

data = open("C:\Documents and Settings\Mel\Desktop\Python2\Blender\cameratrack.txt").read().split("\n")
result = [ eval(i.replace("(",",").replace(")","")) for i in data]
for j in  zip(*result):
    print j
 
varList = ['frame', 'coordx', 'coordy', 'coordz']
for i, v in enumerate(varList):
    exec '%s = %s' % (v, result[i])
 
print frame
print coordx
print coordy
print coordz

I know its got something to do with the result[i] bit, but wasn't sure what to put there instead.

 
import re
 
fn = 'C:\Documents and Settings\Mel\Desktop\Python2\Blender\cameratrack.txt'
patt = r'\d+'
varList = ['frame', 'coordx', 'coordy', 'coordz']
 
data = zip(*[re.findall(patt,item) for item in open(fn).readlines()])
 
for i, v in enumerate(varList):
    exec '%s = %s' % (v, [float(i) for i in data[i]])
 
print frame
print coordx
print coordy
print coordz

This was helpful but doesn't allow for negative numbers in the list e.g 5 (1,-10,10) comes out as 5, 1, 10, 10. Though I cant quite work out why!

If anyone could help or explain to me where Im going wrong that would be great thanks!
Mel

[bartonc]

I'll try to get some time to take a look. In the mean time, I'm just bumping this thread.

See ya, Mel.

[bvdet]

Right I've had some time to look at these codes.



I think I understand the code, but cannot work out how to assign the list names so I can recall them later. I tried using some of bvdet code but ended up applying a line as a list:

data = open("C:\Documents and Settings\Mel\Desktop\Python2\Blender\cameratrack.txt").read().split("\n")
result = [ eval(i.replace("(",",").replace(")","")) for i in data]
for j in  zip(*result):
    print j
 
varList = ['frame', 'coordx', 'coordy', 'coordz']
for i, v in enumerate(varList):
    exec '%s = %s' % (v, result[i])
 
print frame
print coordx
print coordy
print coordz

I know its got something to do with the result[i] bit, but wasn't sure what to put there instead.


This was helpful but doesn't allow for negative numbers in the list e.g 5 (1,-10,10) comes out as 5, 1, 10, 10. Though I cant quite work out why!

If anyone could help or explain to me where Im going wrong that would be great thanks!
Mel

Replace the assignment to 'patt' with:

patt = r'-\d+|\d+'

This will look for negative numbers (numbers preceded with '-') and then look for positive numbers.

[Sushi]

Ah brilliant!
Thanks Bvdet!