csv module

[Laurent Laporte]

hello,

I'm using cvs standard module under Python 2.3 / 2.4 to write a file
delimited with tabs. I use the "excel-tab" dialect to do that.

To read my CSV file, I choose to 'sniff' with a sample data in order to
get the dialect.
The problem I meet is that I get a wrong dialect: the sniffer return an
empty string delimiter. It is probably a bug in _guess_delimiter()
method.

The message I obtain is:
TypeError: bad argument type for built-in operation

Do you know a way to sniff tab-delimited data ?
Is it a known bug ?

Bye.

[Fredrik Lundh]

Laurent Laporte wrote:

I'm using cvs standard module under Python 2.3 / 2.4 to write a file
delimited with tabs. I use the "excel-tab" dialect to do that.

To read my CSV file, I choose to 'sniff' with a sample data in order to
get the dialect.
The problem I meet is that I get a wrong dialect: the sniffer return an
empty string delimiter. It is probably a bug in _guess_delimiter()
method.

The message I obtain is:
TypeError: bad argument type for built-in operation

Do you know a way to sniff tab-delimited data ?
Is it a known bug ?

http://www.python.org/sf/1157169

</F>

[skip]

Laurent> To read my CSV file, I choose to 'sniff' with a sample data in
Laurent> order to get the dialect. The problem I meet is that I get a
Laurent> wrong dialect: the sniffer return an empty string delimiter. It
Laurent> is probably a bug in _guess_delimiter() method.

Laurent> The message I obtain is:
Laurent> TypeError: bad argument type for built-in operation

Laurent> Do you know a way to sniff tab-delimited data ?
Laurent> Is it a known bug ?

Using a file with the following contents:

open("tabber.csv", "rb").read() '1\t2\tabc\n3\t4\tdef\n'

I get:
sniffer = csv.Sniffer()
d = sniffer.sniff(open("tabber.csv", "rb").read())
d.delimiter

'\t'

Can you provide a concrete example (preferably in a bug report on SF)?

Skip

[skip]

me> Using a file with the following contents:

me> >>> open("tabber.csv", "rb").read()
me> '1\t2\tabc\n3\t4\tdef\n'

me> I get:

me> >>> sniffer = csv.Sniffer()
me> >>> d = sniffer.sniff(open("tabber.csv", "rb").read())
me> >>> d.delimiter
me> '\t'

BTW, this also seems to work with a Mac-style EOL:

open("tabber.csv", "rb").read() '1\t2\tabc\r3\t4\tdef\r' d = sniffer.sniff(open("tabber.csv", "rb").read())
d.delimiter

'\t'

Perhaps this has been fixed in CVS.

Skip

[Laurent Laporte]

Sorry,

Here is my example:

Python 2.3.1 (#1, Sep 29 2003, 15:42:58)
[GCC 2.96 20000731 (Red Hat Linux 7.1 2.96-98)] on linux2
Type "help", "copyright", "credits" or "license" for more information.

import csv
t_sniffer = csv.Sniffer()
t_data = "aaa\tbbb\r\n\r\nAAA\tBBB\r\n"
t_dialect = t_sniffer.sniff(t_data)
t_dialect.delimiter

''

In fact, I found the pb (thanks to you): I add a newline '\r\n' to
separate the header from the records...

[Laurent Laporte]

In fact, there is another bug:

In my CVS file, all the records ends with a trailing tab '\t'
except the header because the last field is always empty.

For example, I get :

import csv
t_sniffer = csv.Sniffer()
t_data = "aaa\tbbb\r\nAAA\t\r\nBBB\t\r\n"
t_dialect = t_sniffer.sniff(t_data)
t_dialect.delimiter

''

It is done in the _guess_delimiter() method during the building of
frequency tables. A striping is done for each line (why??)
If I change:
freq = line.strip().count(char)
by:
freq = line.count(char)
It works fine.

Do you have a workaround for that?

------- Laurent.

[skip]

Laurent> If I change:
Laurent> freq = line.strip().count(char)
Laurent> by:
Laurent> freq = line.count(char)
Laurent> It works fine.

Laurent> Do you have a workaround for that?

Nope. I just checked in precisely your fix to the Python repository.

Skip