How can I read a data file into a 2xN matrix using NumPy arrays?

csgirlie

I'm trying to store or arrange three sets of two-dimensional data into three 2xN matrices that are stored as NumPy arrays.

Expand|Select|Wrap|Line Numbers

 import os                # for file handling functions

import numpy as np            # for array/matrix processing

import matplotlib.pyplot as plt        # for general plotting

from mpl_toolkits.mplot3d import axes3d    # for 3d plotting
 
wordList=[]

numFiles = 0

numDataSets = numVectors = sizeVector = 0
 
print"\n\nBeginning data processing."

# traverse filePath folder

filePath = "Train"

for file in os.listdir(filePath):

    numDataSets +=1                # one data set per file

    print "Loading the file: " + file + "."

    tempFile = open(filePath+"/"+file, 'rU')

    for line in tempFile:

        numVectors +=1            # one vector per line

        for word in line.split():

            sizeVector +=1        # components in vector

            wordList.append(word)
 
sizeVector = sizeVector / numVectors

numVectors = numVectors / numDataSets
 
# print"\nHere's all the data:\n", wordList, "\n\n"

print "numDataSets: ",numDataSets

print "numVectors: ",numVectors

print "sizeVector: ",sizeVector

x = eval(wordList[5])
 
# Structure our arrays to hold our data.

# We'll use the three sample class data sets from the text.

# sizeVector of rows and numVectors of columns for column vector data.

# In other words, each feature vector will be a column vector for set array.

w1 = np.zeros((sizeVector, numVectors))

w2 = np.zeros((sizeVector, numVectors))

w3 = np.zeros((sizeVector, numVectors))
 
#   Load matrix array with class w1 numeric values,

# replacing elements indexed as in matrices row, col starting from 0.

#   Pull elements from wordList based on dataSet, sizeVector, and

# which vector we're on during the column, row iteration.
 
skip = numVectors*sizeVector
 
dataSet = 0    

for j in range(sizeVector):

    for i in range(numVectors):

        (do something - nothing I've tried is working)

print "\nClass w1 is as follows: \n",w1
 
dataSet = 1    

for j in range(sizeVector):

    for i in range(numVectors):

        (do something - nothing I've tried is working)

print "\nClass w2 is as follows: \n",w2
 
dataSet = 2    

for j in range(sizeVector):

    for i in range(numVectors):

        (do something - nothing I've tried is working)

print "\nClass w3 is as follows: \n",w3

I need advice on how to get my dataSet(s) set up in this 2xN format.

Glenton

Hi

This is a bit difficult to engage on, because of the level of detail. You might also get better responses if you say more generally what you're trying to achieve and why.

But, if I understand it, you have a list, called wordList which has (3 x numVectors x sizeVector) elements, and you want to change it into 3 separate numpy arrays each of which has 2 columns and (numVectors x sizeVector)/2 rows. Or, perhaps into arrays which have sizeVector columns and numVectors rows.

So the key lines might be something like
w1[i,j]=wordList[i+numVectors*j]
w2[i,j]=wordList[i+numVectors*j+size]
w2[i,j]=wordList[i+numVectors*j+2*size]

or maybe w1[i][j]

I haven't had a chance to check it yet. Is something like this what you've tried?

How can I read a data file into a 2xN matrix using NumPy arrays?

re: How can I read a data file into a 2xN matrix using NumPy arrays?

Similar Python bytes

/bytes/development

/bytes/it

/bytes/about