How to read a csv file from a python script?

[somialis]

Hi all,
I want to read the contents of a csv file thru my python script.
The csv file has say n rows and each row contains some numbers separated by commas. I have to access a specific row and then all the numbers separately. For ex: the csv file is something like this:

1 01,02,05,07,18
2 00.01,04
3 09,20,21

now the first number which is the number of the row will be provided as input thru command line args.
i want to get the numbers following the row number...i.e for row 1 i want to get 01,02,05,07,18 not as one complete line but as separate numbers.
plz help

[ghostdog74]

Hi all,
I want to read the contents of a csv file thru my python script.
The csv file has say n rows and each row contains some numbers separated by commas. I have to access a specific row and then all the numbers separately. For ex: the csv file is something like this:

1 01,02,05,07,18
2 00.01,04
3 09,20,21

now the first number which is the number of the row will be provided as input thru command line args.
i want to get the numbers following the row number...i.e for row 1 i want to get 01,02,05,07,18 not as one complete line but as separate numbers.
plz help

since your data has a significant blank space just after the row number and nowhere else, you can use split() on blank.
eg

>>> for line in open("file"):
...  line = line.strip().split()
...  print "row:  ", line[0] , line[1:][0].split(",")
...
row:   1 ['01', '02', '05', '07', '18']
row:   2 ['00.01', '04']
row:   3 ['09', '20', '21']
 

[bartonc]

Hi all,
I want to read the contents of a csv file thru my python script.
The csv file has say n rows and each row contains some numbers separated by commas. I have to access a specific row and then all the numbers separately. For ex: the csv file is something like this:

1 01,02,05,07,18
2 00.01,04
3 09,20,21

now the first number which is the number of the row will be provided as input thru command line args.
i want to get the numbers following the row number...i.e for row 1 i want to get 01,02,05,07,18 not as one complete line but as separate numbers.
plz help

My friend ghostdog74 has a very good solution if, in fact, rows start with row number and there are NO spaces between entries.
A true csv file would have a format more like:
1st_value, 2nd_value, etc
For such files, the csv module may be a good choice, too.

[somialis]

My friend ghostdog74 has a very good solution if, in fact, rows start with row number and there are NO spaces between entries.
A true csv file would have a format more like:
1st_value, 2nd_value, etc
For such files, the csv module may be a good choice, too.

Actually the situation is little bit different..
there are several rows in the csv file and each row starts with a unique number followed by another set of numbers. as i told in the previous post. I have to access a particular row and the numbers following it.

[bvdet]

Actually the situation is little bit different..
there are several rows in the csv file and each row starts with a unique number followed by another set of numbers. as i told in the previous post. I have to access a particular row and the numbers following it.

Barton is correct. This is not a true csv file. As long as the data is consistent, this should work:

'''
1 01,02,05,07,18
2 00,01,04
3 09,20,21
4 12,34,56,77,88,99
5 03,05,77,88,54
'''
 
def file_data1(f):
    dd = {}
    for line in f:
        dd[int(line.split()[0])] = [int(i) for i in line.split()[1].split(',')]
    return dd        
 
dd = file_data1(open('your_file').readlines())
for key in dd:
    print '%s = %s' % (key, dd[key])
 
'''
>>> 1 = [1, 2, 5, 7, 18]
2 = [0, 1, 4]
3 = [9, 20, 21]
4 = [12, 34, 56, 77, 88, 99]
5 = [3, 5, 77, 88, 54]
'''

>>> for i in dd[5]:
... print i,
...
3 5 77 88 54
>>>

[ghostdog74]

Actually the situation is little bit different..
there are several rows in the csv file and each row starts with a unique number followed by another set of numbers. as i told in the previous post. I have to access a particular row and the numbers following it.

so you want to get an input from the command line arguments, try something like this

import sys
choice = sys.argv[1]
for line in open("file"):
     line = line.strip().split()
     if choice == line[0]
          print line[1:][0].split(",")
 

[somialis]

so you want to get an input from the command line arguments, try something like this

import sys
choice = sys.argv[1]
for line in open("file"):
     line = line.strip().split()
     if choice == line[0]
          print line[1:][0].split(",")
 

Sry but this dint work...it did not print nething!! :(

[somialis]

Barton is correct. This is not a true csv file. As long as the data is consistent, this should work:

'''
1 01,02,05,07,18
2 00,01,04
3 09,20,21
4 12,34,56,77,88,99
5 03,05,77,88,54
'''
 
def file_data1(f):
    dd = {}
    for line in f:
        dd[int(line.split()[0])] = [int(i) for i in line.split()[1].split(',')]
    return dd        
 
dd = file_data1(open('your_file').readlines())
for key in dd:
    print '%s = %s' % (key, dd[key])
 
'''
>>> 1 = [1, 2, 5, 7, 18]
2 = [0, 1, 4]
3 = [9, 20, 21]
4 = [12, 34, 56, 77, 88, 99]
5 = [3, 5, 77, 88, 54]
'''

>>> for i in dd[5]:
... print i,
...
3 5 77 88 54
>>>

i executed the code but it showed the following error:

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 4, in file_data
ValueError: invalid literal for int(): Of

[bartonc]

i executed the code but it showed the following error:

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 4, in file_data
ValueError: invalid literal for int(): Of

It looks like part of the error message is missing...

You'll need to paste a copy of the actual data. BV's code is not the problem.

[somialis]

It looks like part of the error message is missing...

You'll need to paste a copy of the actual data. BV's code is not the problem.

i pasted the entire error message...
the actual csv file somwwhat looks like this:


List Of Individual Patches
1 00;01;
2 00;
3 01;
4 02;
5 00;01;02;
6 01;02;
7 01;02;03;04;05;
8 01;02;03;04;05;06;
9 00;01;02;03;05;
10 01;02;03;05;
11 02;03;05;06;
12 04;
13 01;02;03;05;08;09;
14 00;01;02;03;05;09;
15 00;01;02;03;05;09;10;
16 01;02;03;05;08;09;10;
17 01;02;03;05;08;10;
18 08;
19 00;01;02;03;05;08;09;10;
20 02;03;05;06;08;10;
21 11;
22 12;
23 11;12;
24 00;01;02;03;05;14;


i'll take the leftmost number as input and depending upon tht number..i'll have to get the list of numbers following it.

plz help

[bartonc]

i pasted the entire error message...
the actual csv file somwwhat looks like this:


List Of Individual Patches
1 00;01;
2 00;
3 01;
4 02;
5 00;01;02;
6 01;02;
7 01;02;03;04;05;
8 01;02;03;04;05;06;
9 00;01;02;03;05;
10 01;02;03;05;
11 02;03;05;06;
12 04;
13 01;02;03;05;08;09;
14 00;01;02;03;05;09;
15 00;01;02;03;05;09;10;
16 01;02;03;05;08;09;10;
17 01;02;03;05;08;10;
18 08;
19 00;01;02;03;05;08;09;10;
20 02;03;05;06;08;10;
21 11;
22 12;
23 11;12;
24 00;01;02;03;05;14;


i'll take the leftmost number as input and depending upon tht number..i'll have to get the list of numbers following it.

plz help

You originally stated that there are commas in your data, so people wrote functions that look for commas:

def file_data1(f):
    dd = {}
    for line in f:
        dd[int(line.split()[0])] = [int(i) for i in line.split()[1].split(',')]
    return dd        
 

Do you see the comma in the above function? Your data has semi-colun so you need to change that to:

def file_data1(f):
    dd = {}
    for line in f:
        dd[int(line.split()[0])] = [int(i) for i in line.split()[1].split(';')] # chaned here
    return dd        
 

Thank you, BV, for the original function.

[somialis]

You originally stated that there are commas in your data, so people wrote functions that look for commas:

def file_data1(f):
    dd = {}
    for line in f:
        dd[int(line.split()[0])] = [int(i) for i in line.split()[1].split(',')]
    return dd        
 

Do you see the comma in the above function? Your data has semi-colun so you need to change that to:

def file_data1(f):
    dd = {}
    for line in f:
        dd[int(line.split()[0])] = [int(i) for i in line.split()[1].split(';')] # chaned here
    return dd        
 

Thank you, BV, for the original function.

now as soon as i typed the line :

dd = file_data(open('my_file').readlines())

it showed the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "<stdin>", line 4, in file_dat
IndexError: list index out of range
 

[ghostdog74]

i pasted the entire error message...
the actual csv file somwwhat looks like this:


List Of Individual Patches
1 00;01;
2 00;
3 01;
4 02;
5 00;01;02;
6 01;02;
7 01;02;03;04;05;
8 01;02;03;04;05;06;
9 00;01;02;03;05;
10 01;02;03;05;
11 02;03;05;06;
12 04;
13 01;02;03;05;08;09;
14 00;01;02;03;05;09;
15 00;01;02;03;05;09;10;
16 01;02;03;05;08;09;10;
17 01;02;03;05;08;10;
18 08;
19 00;01;02;03;05;08;09;10;
20 02;03;05;06;08;10;
21 11;
22 12;
23 11;12;
24 00;01;02;03;05;14;


i'll take the leftmost number as input and depending upon tht number..i'll have to get the list of numbers following it.

plz help

it does work

import sys
choice = sys.argv[1]
for line in open("file"):
     line = line.strip().split()
     if choice == line[0]:
          print line[1:][0].split(";")
 

output :

# ./test.py 23
['11', '12', '']
 

Note that your csv file is now colon separated and not comma separated. So you have to change the split() statement.

[somialis]

now as soon as i typed the line :

dd = file_data(open('my_file').readlines())

it showed the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "<stdin>", line 4, in file_data
ValueError: invalid literal for int(): Of
 

now as soon as i typed the line :

dd = file_data(open('my_file').readlines())

it showed the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "<stdin>", line 4, in file_data
ValueError: invalid literal for int(): Of
 

[/quote]

[somialis]

now as soon as i typed the line :

dd = file_data(open('my_file').readlines())

it showed the following error:

Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "<stdin>", line 4, in file_dat
IndexError: list index out of range
 

plz ignore the above mentioned error. i had done some mistake in naming the variable. The error is actually the same as before as soon as i type the following line:

dd = file_data(open('my_file').readlines())

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "<stdin>", line 4, in file_data
ValueError: invalid literal for int(): Of
 

[somialis]

it does work

import sys
choice = sys.argv[1]
for line in open("file"):
     line = line.strip().split()
     if choice == line[0]:
          print line[1:][0].split(";")
 

output :

# ./test.py 23
['11', '12', '']
 

Note that your csv file is now colon separated and not comma separated. So you have to change the split() statement.

it dint work even now. Let me tell u wht i did:

1) created a file new.py
2) Wrote the following code:

import os
import sys
import csv
 
def main(argv):
 
    choice = sys.argv[1]
    fp = open("c:\pst\pst_modified\log_4_0.csv",'r')
    for line in fp:
     line = line.strip().split()
     if choice == line[0]:
          print line[1:][0].split(";")
 

3)i have python23 installed at my system
4) opened the command prompt
5)Moved to the directory where new.py is placed
6)executed the following command
python new.py '23'

whr am i wrong??
./new.py flasgged an error tht "." is not a valid command.

[ghostdog74]

it dint work even now. Let me tell u wht i did:

1) created a file new.py
2) Wrote the following code:

import os
import sys
import csv
 
def main(argv):
 
    choice = sys.argv[1]
    fp = open("c:\pst\pst_modified\log_4_0.csv",'r')
    for line in fp:
     line = line.strip().split()
     if choice == line[0]:
          print line[1:][0].split(";")
 

3)i have python23 installed at my system
4) opened the command prompt
5)Moved to the directory where new.py is placed
6)executed the following command
python new.py '23'

whr am i wrong??
./new.py flasgged an error tht "." is not a valid command.

try this:

import os
import sys
import csv ## you are not using this?
 
filepath = os.path.join("C:\\","pst","pst_modified","log_4_0.csv")
 
def main():
    choice = sys.argv[1]    
    for line in open(filepath):
     line = line.strip().split()
     if choice == line[0]:
          print line[1:][0].split(";")
 
main()  ## < --you did not call the main in your last posted script sample.
 

[somialis]

try this:

import os
import sys
import csv ## you are not using this?
 
filepath = os.path.join("C:\\","pst","pst_modified","log_4_0.csv")
 
def main():
    choice = sys.argv[1]    
    for line in open(filepath):
     line = line.strip().split()
     if choice == line[0]:
          print line[1:][0].split(";")
 
main()  ## < --you did not call the main in your last posted script sample.
 

Hi,
it's still not working....thr's something tht i wud like to tell.
If u put a " print line" statement after
line = line.strip().split()
u'll c that it prints the entire csv file and reaches the end. After tht when u compare choice with line[0] ...the condition does not satisfy becauseline[0] contains the last line of the csv file.

[ghostdog74]

Hi,
it's still not working....thr's something tht i wud like to tell.
If u put a " print line" statement after
line = line.strip().split()
u'll c that it prints the entire csv file and reaches the end. After tht when u compare choice with line[0] ...the condition does not satisfy becauseline[0] contains the last line of the csv file.

please try this debugging method:

def main():
    choice = sys.argv[1]    
    for line in open(filepath):
     line = line.strip().split()
     print line, type(line), line[0], line[1]
     raw_input("Press to continue: " )
     if choice == line[0]:
          print line[1:][0].split(";")
main()
 

please show the output of the above.

[somialis]

please try this debugging method:

def main():
    choice = sys.argv[1]    
    for line in open(filepath):
     line = line.strip().split()
     print line, type(line), line[0], line[1]
     raw_input("Press to continue: " )
     if choice == line[0]:
          print line[1:][0].split(";")
main()
 

please show the output of the above.

The output of your code is as follows:

[',List', 'Of', 'Individual', 'Patches'] <type 'list'> ,List Of
Press to continue:
['1,00;01;'] <type 'list'> 1,00;01;
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "my.py", line 12, in main
    print line, type(line), line[0], line[1]
IndexError: list index out of range
 

[ghostdog74]

The output of your code is as follows:

[',List', 'Of', 'Individual', 'Patches'] <type 'list'> ,List Of
Press to continue:
['1,00;01;'] <type 'list'> 1,00;01;
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "my.py", line 12, in main
    print line, type(line), line[0], line[1]
IndexError: list index out of range
 

hey...you original sample is like this:

List Of Individual Patches
1 00;01;
2 00;
3 01;
4 02;
5 00;01;02;
6 01;02;
7 01;02;03;04;05;
8 01;02;03;04;05;06;
9 00;01;02;03;05;
10 01;02;03;05;
11 02;03;05;06;
12 04;
13 01;02;03;05;08;09;
14 00;01;02;03;05;09;
15 00;01;02;03;05;09;10;
16 01;02;03;05;08;09;10;
17 01;02;03;05;08;10;
18 08;
19 00;01;02;03;05;08;09;10;
20 02;03;05;06;08;10;
21 11;
22 12;
23 11;12;
24 00;01;02;03;05;14;
 

so if you have changed your format to like this:

1,00;01;
2,00;
...
..
 

then surely it doesn't work. you have to modify the code. i give you a hint on where to change:

....
    for line in open(filepath):
     line = line.strip().split()  <---here.
....
 

[somialis]

hey...you original sample is like this:

List Of Individual Patches
1 00;01;
2 00;
3 01;
4 02;
5 00;01;02;
6 01;02;
7 01;02;03;04;05;
8 01;02;03;04;05;06;
9 00;01;02;03;05;
10 01;02;03;05;
11 02;03;05;06;
12 04;
13 01;02;03;05;08;09;
14 00;01;02;03;05;09;
15 00;01;02;03;05;09;10;
16 01;02;03;05;08;09;10;
17 01;02;03;05;08;10;
18 08;
19 00;01;02;03;05;08;09;10;
20 02;03;05;06;08;10;
21 11;
22 12;
23 11;12;
24 00;01;02;03;05;14;
 

so if you have changed your format to like this:

1,00;01;
2,00;
...
..
 

then surely it doesn't work. you have to modify the code. i give you a hint on where to change:

....
    for line in open(filepath):
     line = line.strip().split()  <---here.
....
 

i haven't changed the format. My csv file is the same as it was before.It's like this only:

List Of Individual Patches
1 00;01;
2 00;
3 01;
4 02;
5 00;01;02;
6 01;02;
7 01;02;03;04;05;
8 01;02;03;04;05;06;
9 00;01;02;03;05;
10 01;02;03;05;
11 02;03;05;06;
12 04;
13 01;02;03;05;08;09;
14 00;01;02;03;05;09;
15 00;01;02;03;05;09;10;
16 01;02;03;05;08;09;10;
17 01;02;03;05;08;10;
18 08;
19 00;01;02;03;05;08;09;10;
20 02;03;05;06;08;10;
21 11;
22 12;
23 11;12;
24 00;01;02;03;05;14;
25 00;01;02;03;05;09;14;
26 00;01;02;03;05;08;09;10;14;
27 00;01;02;03;05;17;
28 00;01;02;03;05;09;17;
29 00;01;02;03;05;08;09;10;17;
30 01;02;03;05;08;09;10;17;
31 00;01;02;03;05;09;18;
32 01;02;03;05;08;09;10;18;

and after executing ur code snippet i got the error message which i posted in the previous post. Dunno in the output where from the comma is coming.

[ghostdog74]

look at this error you had previously:

[',List', 'Of', 'Individual', 'Patches'] <type 'list'> ,List Of
Press to continue:
['1,00;01;'] <type 'list'> 1,00;01;
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "my.py", line 12, in main
    print line, type(line), line[0], line[1]
IndexError: list index out of range
 

the follwing output line results from "print line, type(line), line[0], line[1]" statement:

...
['1,00;01;'] <type 'list'> 1,00;01;
...
 

The "line" value is ['1,00;01;'] after the split. This means it did not split at all. Furthermore, line.strip().split() splits on blanks and since your data have a comma now, it does nothing.
notice the "," after the first "1" ?? that's why i deduced that you csv format is different. If otherwise, then i have no idea at all.

...
line = line.strip().split(",")
...
 

[somialis]

look at this error you had previously:

[',List', 'Of', 'Individual', 'Patches'] <type 'list'> ,List Of
Press to continue:
['1,00;01;'] <type 'list'> 1,00;01;
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
  File "my.py", line 12, in main
    print line, type(line), line[0], line[1]
IndexError: list index out of range
 

the follwing output line results from "print line, type(line), line[0], line[1]" statement:

 
...
['1,00;01;'] <type 'list'> 1,00;01;
...
 

The "line" value is ['1,00;01;'] after the split. This means it did not split at all. Furthermore, line.strip().split() splits on blanks and since your data have a comma now, it does nothing.
notice the "," after the first "1" ?? that's why i deduced that you csv format is different. If otherwise, then i have no idea at all.

...
line = line.strip().split(",")
...
 

hey it worked this time.
thanks a ton!! :)

[ghostdog74]

hey it worked this time.
thanks a ton!! :)

so it's your csv format after all?

[somialis]

so it's your csv format after all?

dint get ur question.
I just included this line

line = line.strip().split(",")
 

in place of this

line = line.strip().split()
 

[ghostdog74]

dint get ur question.
I just included this line

line = line.strip().split(",")
 

in place of this

line = line.strip().split()
 

if by doing a split on comma and it works, it means your csv file has commas, right? so that's why i ask is your csv format like this instead:

1,00;01;
2,00;
....
...
 

if its not, then a split on comma will not work.

[somialis]

if by doing a split on comma and it works, it means your csv file has commas, right? so that's why i ask is your csv format like this instead:

1,00;01;
2,00;
....
...
 

if its not, then a split on comma will not work.

ok...now i understood!!
Thanks a lot!!