High level csv reader

[George Sakkis]

It occured to me that most times I read a csv file, I'm often doing
from scratch things like assigning labels to columns, mapping fields to
the appropriate type, ignoring some fields, changing their order, etc.
Before I go on and reinvent the wheel, is there a generic high level
wrapper around csv.reader that does all this ?

Thanks,
George

[James Stroud]

George Sakkis wrote:

It occured to me that most times I read a csv file, I'm often doing
from scratch things like assigning labels to columns, mapping fields to
the appropriate type, ignoring some fields, changing their order, etc.
Before I go on and reinvent the wheel, is there a generic high level
wrapper around csv.reader that does all this ?

Thanks,
George

There is a csv in the standard library. Though many of us don't mind
answering questions like this, you can get a lot of answers quicker by
(1) looking to see what's in the standard library and (2) using google.

http://docs.python.org/lib/module-csv.html

James

[Fredrik Lundh]

James Stroud wrote:

>Before I go on and reinvent the wheel, is there a generic high level
wrapper around csv.reader that does all this ?

There is a csv in the standard library.

I'm not sure the "csv" module qualifies as a high-level wrapper around
itself, though.

</F>

[John Machin]

James Stroud wrote:

George Sakkis wrote:

It occured to me that most times I read a csv file, I'm often doing
from scratch things like assigning labels to columns, mapping fields to
the appropriate type, ignoring some fields, changing their order, etc.
Before I go on and reinvent the wheel, is there a generic high level
wrapper around csv.reader that does all this ?

Thanks,
George

There is a csv in the standard library. Though many of us don't mind
answering questions like this, you can get a lot of answers quicker by
(1) looking to see what's in the standard library and (2) using google.

http://docs.python.org/lib/module-csv.html

James

The OP mentioned "csv.reader". This indicates to me that he *has* read
the csv docs (have you?), and is *already* using the csv module. The
tasks he says he does often are *not* covered by the standard library.
He appears to be asking if there is a higher-level wrapper around the
standard library.

Please consider reading his question carefully.

[John Machin]

George Sakkis wrote:

It occured to me that most times I read a csv file, I'm often doing
from scratch things like assigning labels to columns, mapping fields to
the appropriate type, ignoring some fields, changing their order, etc.
Before I go on and reinvent the wheel, is there a generic high level
wrapper around csv.reader that does all this ?

Hi George,

Firstly one thing to remember: if you are going to reinvent the wheel,
don't forget to also reinvent the axle :-)

AFAIK there is no such animal.

I would need a lot of persuasion that a wrapper or toolbox wasn't just
overhead with little benefit. For example, ignoring some fields and/or
changing the order of others can be done rather simply:

| >>inrow
| ['a', 'b', 'c', 'd', 'e', 'f']
Suppose we want to ignore the "vowels" and reverse the order of the
others:
| >>outrow = list(inrow[k] for k in (5, 3, 2, 1))
| >>outrow
| ['f', 'd', 'c', 'b']

I don't see the value in creating (and documenting!) a one-line
function

def gather_list_items(input_list, item_map):
return list(input_list[k] for k in item_map)

NB something like this is already in the mx kit somewhere IIRC.

Cheers,
John

[skip]

GeorgeIt occured to me that most times I read a csv file, I'm often
Georgedoing from scratch things like assigning labels to columns,
Georgemapping fields to the appropriate type, ignoring some fields,
Georgechanging their order, etc. Before I go on and reinvent the
Georgewheel, is there a generic high level wrapper around csv.reader
Georgethat does all this ?

I'm not aware of anything that specifically addresses these ideas. Here are
some thoughts though:

* If you use the DictReader class you can ignore fields you aren't
interested in more easily since you access the fields of interest by
name and with the fieldnames parameter to the constructor can assign
column names to csv data which lacks it (use similar functionality in
DictWriter to create column labels on output). You can also specify
the restkey parameter to the constructor and thus only specify the
fields of interest in the fieldnames parameter. (I think. I've never
actually used that capability, but that's what the documentation
suggests.)

* There was a thread earlier this month with this subject:

paseline(my favorite simple script): does something similar exist?

Check it out for a number of different solutions to formatting the
fields in a line of text. The idea can easily be extended to a list
or dict of values instead, perhaps in a subclass of DictReader.

Skip

[George Sakkis]

sk**@pobox.com wrote:

GeorgeIt occured to me that most times I read a csv file, I'm often
Georgedoing from scratch things like assigning labels to columns,
Georgemapping fields to the appropriate type, ignoring some fields,
Georgechanging their order, etc. Before I go on and reinvent the
Georgewheel, is there a generic high level wrapper around csv.reader
Georgethat does all this ?

I'm not aware of anything that specifically addresses these ideas. Here are
some thoughts though:

* If you use the DictReader class you can ignore fields you aren't
interested in more easily since you access the fields of interest by
name and with the fieldnames parameter to the constructor can assign
column names to csv data which lacks it (use similar functionality in
DictWriter to create column labels on output). You can also specify
the restkey parameter to the constructor and thus only specify the
fields of interest in the fieldnames parameter. (I think. I've never
actually used that capability, but that's what the documentation
suggests.)

* There was a thread earlier this month with this subject:

paseline(my favorite simple script): does something similar exist?

Check it out for a number of different solutions to formatting the
fields in a line of text. The idea can easily be extended to a list
or dict of values instead, perhaps in a subclass of DictReader.

Skip

Indeed, they are both relevant; actually Fredrik's suggestion in that
thread was my starting point. Here's my my current API for the (most
typical) case of fixed-size rows, addressing my most common
requirement, field conversions (no named columns for now):
http://rafb.net/paste/results/4UgvSD50.html.

It looks somewhat involved but it's more along the lines of "making
easy things easy and hard things possible". Comments and suggestions
are most welcome.

George