convert a text file to csv

[olivercfc]

Hello All,

I have a text file in the following format

<
line 1
line 2
line 3
line < 4
line > 5
>
<
line 1
line < 2
line >3
>
 

and so on

I need to convert it to csv so it looks like this:

<,line 1,line 2,line 3,line < 4,line > 5,>
<,line 1,line < 2,line >3,>
 

So basically, in pseudo code, I need to do the following:

find opening marker ( start of line = < )
join each subsequent line with a comma delimiter
find ending marker ( start of line = > )
continue thru file

Any helpers would be gratefully received

Thanks

[bartonc]

Hello All,

I have a text file in the following format

<
line 1
line 2
line 3
line < 4
line > 5
>
<
line 1
line < 2
line >3
>
 

and so on

I need to convert it to csv so it looks like this:

<,line 1,line 2,line 3,line < 4,line > 5,>
<,line 1,line < 2,line >3,>
 

So basically, in pseudo code, I need to do the following:

find opening marker ( start of line = < )
join each subsequent line with a comma delimiter
find ending marker ( start of line = > )
continue thru file

Any helpers would be gratefully received

Thanks

I'm just going to wing this (I'm not at a computer with Python installed):

theData = open(r"the\file\path\filename.txt")
for line in theData:
    marker = line[0]
    if marker == "<":
        while marker != ">":
        nextLine = theData.next()
        print nextLine
        marker = nextLine[0]

Something like that, anyway. It shoud be enough to get you started.

[olivercfc]

Something like that, anyway. It shoud be enough to get you started.

thanks - I'll give it a whirl

[bvdet]

Hello All,

I have a text file in the following format

<
line 1
line 2
line 3
line < 4
line > 5
>
<
line 1
line < 2
line >3
>
 

and so on

I need to convert it to csv so it looks like this:

<,line 1,line 2,line 3,line < 4,line > 5,>
<,line 1,line < 2,line >3,>
 

So basically, in pseudo code, I need to do the following:

find opening marker ( start of line = < )
join each subsequent line with a comma delimiter
find ending marker ( start of line = > )
continue thru file

Any helpers would be gratefully received

Thanks

This uses string concatenation. It is not efficient, but it works:

fList = [s.strip() for s in open('text_to_csv.txt').readlines()]
 
outStr = ''
for i in fList:
    if i == '>':
        outStr += (i+'\n')
    else:
        outStr += (i+',')
 
f = open('text_to_csv.csv', 'w')
f.write(outStr.strip())
f.close()

[olivercfc]

This uses string concatenation. It is not efficient, but it works

works perfectly for me - thanks!

[bvdet]

works perfectly for me - thanks!

What I meant by inefficient: String concatenation is somewhat inefficient compared to s.join() for example. After thinking about it, reading a file and writing to another file takes far longer to process compared to the string addition, so it just does not matter.

[ghostdog74]

data=open("file").read()
pat = re.compile("^<$(.*?)^>$",re.M|re.DOTALL)
for item in pat.findall(data):
    item=item.replace("\n",",")
    print "<",item,">"
 

output:

# ./test.py
< ,line 1,line 2,line 3,line < 4,line > 5, >
< ,line 1,line < 2,line >3, >
 

[bvdet]

fList = open('text_data').read().split('>\n<')
f = open('csv_data', 'w')
f.write(',>\n'.join([','.join([i for i in item.split('\n') if i != '']) for item in fList]))
f.close()

So far we have three methods that do basically the same thing. Let's see what timeit shows us:

def test1(s):
    fList = [i.strip() for i in s.split('\n')]
    outStr = ''
    for i in fList:
        if i == '>':
            outStr += (i+'\n')
        else:
            outStr += (i+',')
    return outStr
 
def test4(s):
    import re
    pat = re.compile("^<$(.*?)^>$",re.M|re.DOTALL)
    return ''.join(['%s%s%s' % ("<",item.replace("\n",","),">\n") for item in pat.findall(s)])
 
def test5(s):
    fList = s.split('>\n<')
    return ',>\n'.join([','.join([i for i in item.split('\n') if i != '']) for item in fList])
 
if __name__ == '__main__':
    from timeit import Timer
    s = '<\nline 1\nline 2\nline 3\nline < 4\nline > 5\n>\n<\nline 1\nline < 2\nline >3\n>\n<\nline 1\nline < 2\nline >3\n>\n<\nline 1\nline < 2\nline >3\n>\n<\nline 1\nline 2\nline 3\nline < 4\nline > 5\n>\n<\nline 1\nline 2\nline 3\nline < 4\nline > 5\n>'
    t = Timer("test1(s)", "from __main__ import test1, s")
    print t.timeit(100000)
    t = Timer("test4(s)", "from __main__ import test4, s")
    print t.timeit(100000)
    t = Timer("test5(s)", "from __main__ import test5, s")
    print t.timeit(100000)

Output (executed three times):

>>> 5.90535333401
4.68323404232
4.66921242784
>>> 5.84924620308
4.71450198279
4.44999015238
>>> 5.91304453499
4.73270178193
4.4917451799
>>> 

Not much difference! If you leave out the import statement, the 're' solution (test4()) ends up slightly faster than test5().

[bvdet]

Correction -

fList = open('text_data').read().split('>\n<')
f = open('csv_data', 'w')
f.write(',>\n<'.join([','.join([i for i in item.split('\n') if i != '']) for item in fList]))
f.close()

def test5(s):
    fList = s.split('>\n<')
    return ',>\n<'.join([','.join([i for i in item.split('\n') if i != '']) for item in fList])

I left the character '<' out in my previous post.

[bvdet]

Here's a winner (judged by timeit):

def test6(s):
    return ',>\n<,'.join([item.strip().replace('\n', ',') for item in s.split('>\n<')])
 
if __name__ == '__main__':
    from timeit import Timer
    t = Timer("test6(s)", "from __main__ import test6, s")
    print t.timeit(100000)
 
>>> 1.79765081874