473,322 Members | 1,562 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

Request for a change in the csv module.

Hi.

I have just seen that csv module, more exactly the Dialect class,
does not have any variable to specify the "floating point" character!
In portuguese this is ','. Not '.'. 3.1415 -3,1415.
I think this is also the case of other languages/countries.
If I am correct, i.e. if there is no such possibility, where can
I file a request for a csv change? Excel, for example, automatically
converts '.' to ',' and the separator from ',' to ';'.

Thanks.
Mar 12 '07 #1
9 1498
On Mar 12, 4:26 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:
Hi.

I have just seen that csv module, more exactly the Dialect class,
does not have any variable to specify the "floating point" character!
In portuguese this is ','. Not '.'. 3.1415 -3,1415.
I think this is also the case of other languages/countries.
If I am correct, i.e. if there is no such possibility, where can
I file a request for a csv change? Excel, for example, automatically
converts '.' to ',' and the separator from ',' to ';'.


Try using locale.format when writing:

import csv
import locale

locale.setlocale(locale.LC_NUMERIC, 'pt_BR.ISO8859-1')
csv_writer = csv.writer(open("foo.csv","w"), dialect='excel')
rows = (('testing', 1.23), ('testing', 2.34))
formatted_rows = ((string, locale.format('%g', number)) for
(string,number) in rows)
csv_writer.writerows(formatted_rows)
locale.atof and locale.atoi can convert a string back
to a number.

--
Hope this helps,
Steven
Mar 13 '07 #2
at*************@gmail.com escreveu:
On Mar 12, 4:26 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:
....
>
locale.setlocale(locale.LC_NUMERIC, 'pt_BR.ISO8859-1')
csv_writer = csv.writer(open("foo.csv","w"), dialect='excel')
rows = (('testing', 1.23), ('testing', 2.34))
formatted_rows = ((string, locale.format('%g', number)) for
(string,number) in rows)
csv_writer.writerows(formatted_rows)

That works but it is a pain to use.
May be I'll sublass csv or even I'll write one myself.
It would be much better to be able to specify an additional
variabel to the Dialect class and change csv.

Thanks
Paulo
Mar 13 '07 #3
Paulo da Silva schrieb:
at*************@gmail.com escreveu:
>On Mar 12, 4:26 pm, Paulo da Silva <psdasil...@esotericaX.ptXwrote:
...
>locale.setlocale(locale.LC_NUMERIC, 'pt_BR.ISO8859-1')
csv_writer = csv.writer(open("foo.csv","w"), dialect='excel')
rows = (('testing', 1.23), ('testing', 2.34))
formatted_rows = ((string, locale.format('%g', number)) for
(string,number) in rows)
csv_writer.writerows(formatted_rows)


That works but it is a pain to use.
Why? I think it's straightforward.
May be I'll sublass csv or even I'll write one myself.
It would be much better to be able to specify an additional
variabel to the Dialect class and change csv.
I don't think so. The csv module is about the subtleties that can occur
when parsing textual data with possible escape chars and the like, and
creating that data.

But IHMO its up to the user to feed it with just textual data. Because
automatically converting numbers falls flat on it's face in case of
special requirements like a limit on the number of digits to render and
the like. Better do that in a simple and straightforward preprocessing
state, as shown above.

The same is essentially true for dates as well, btw. How do you want to
deal with them?

Diez
Mar 13 '07 #4
It would be much better to be able to specify an additional
variabel to the Dialect class and change csv.
no it wouldn't
this is a locale specific problem so it should be handled there

Mar 13 '07 #5
Diez B. Roggisch escreveu:
Paulo da Silva schrieb:
....
>That works but it is a pain to use.

Why? I think it's straightforward.
That is not the point. The problem is to have things generalized.
I have some general purpose tables whose fields format I don't know
in advance.
Internally the numeric values are independent of localization.
Having a table, I don't like the idea of preformating things
before sending them to the csv for exportation. A little more
of programming and there will be no need for csv at all :-)
....
The same is essentially true for dates as well, btw. How do you want to
deal with them?
This is a pertinent question I didn't think of. May be csv should handle
data conversions using 'locale'.
It is already automatically converting numbers to text
anyway, but unconditionally using the '.'.
As for the dates the dialect could include a format like YYYY-MM-DD
hh:mm:ss. Dates are not always 'locale' only. In pt_PT we use
YYYY-MM-DD or YY-MM-DD and sometimes DD-MM-YYYY.
One must remember that csv is not part of the language. So, to have
some kind of universality, it must use the specs (in dialect) for
formatting and 'locale' for data conversions.

Mar 13 '07 #6
Paulo da Silva wrote:
Diez B. Roggisch escreveu:
>Paulo da Silva schrieb:
...
>>That works but it is a pain to use.

Why? I think it's straightforward.

That is not the point. The problem is to have things generalized.
I have some general purpose tables whose fields format I don't know
in advance.
Internally the numeric values are independent of localization.
Having a table, I don't like the idea of preformating things
before sending them to the csv for exportation. A little more
of programming and there will be no need for csv at all :-)
...
This is simply _not_ true. The details of a proper CSV format are difficult
to get right, which is precisely why the module is there.
>The same is essentially true for dates as well, btw. How do you want to
deal with them?

This is a pertinent question I didn't think of. May be csv should handle
data conversions using 'locale'.
It is already automatically converting numbers to text
anyway, but unconditionally using the '.'.
As for the dates the dialect could include a format like YYYY-MM-DD
hh:mm:ss. Dates are not always 'locale' only. In pt_PT we use
YYYY-MM-DD or YY-MM-DD and sometimes DD-MM-YYYY.
One must remember that csv is not part of the language. So, to have
some kind of universality, it must use the specs (in dialect) for
formatting and 'locale' for data conversions.
I'm not too convinced. It's so darn easy to convert the values to what one
really needs for the actual use-case. Even for your generic case - all you
need is type-based formatting dispatch.

But if you add that complexity to the writers of the csv-module, you'll end
up with a nasty configuration mechanism for special casing individual
columns, or otherwise the next user comes and says "well, I need to format
dates differently just for this case, how to do so in the csv-module". As
your own example shows.

The purpose of the module is solely to assist in creating properly formatted
csv content in the sense that it is readable and parsable and results in
strings for column values. Not more, not less (and you seem to
underestimate that task, btw.)

That it does convert non-string-values to strings when writing could be seen
as convenience, or actual bug as it might cause troubles as you perceived
them - but the solution would clearly be to get rid of this functionality,
not enhance it.

Diez
Mar 13 '07 #7

PauloThat is not the point. The problem is to have things generalized.

Well, perhaps. One of the goals of the csv module was to do things the way
Excel does things. Ideally, that would include formatting elements with
locale sensitivity. I've been working on a csv module in Python, so I
decided to try the locale.format() calls in its writerow implementation. A
number of test cases broke as a result because apparently locale.format
doesn't do its job quite the same way Excel does. I don't think pushing the
locale-sensitive formatting down into the csv module is going to be a
panacea.

Skip
Mar 13 '07 #8
Diez B. Roggisch escreveu:
Paulo da Silva wrote:
....
>
That it does convert non-string-values to strings when writing could be seen
as convenience, or actual bug as it might cause troubles as you perceived
them - but the solution would clearly be to get rid of this functionality,
not enhance it.

It's just a matter of
personal opinion, and IMO, taking in consideration what other python
modules provide, a csv module should make the appropriate data
conversions. It is simple to implement for most (all?) cases and it will
bring a lot of functionality. You just output things.
In csv output you don't need special data formats. Only the data to the
most available precision. Dates are a very special case, as are the
separators, line terminators, etc.

The python way of exporting this data should be something like:
class MyDialect(Dialect):
...
locale="pt_PT@euro"
dateformat="YYYY-MM-DD hh:mm:ss"

...
w=csv.writer(f,dialect=MyDialect)
w.writerows(foo)

Frequent tasks should be done by the modules. Not by the users.
And csv is unusable without output formatting. This is particulary true
if, as you said, the csv didn't provide any conversion at all.
So, why force the programmers to repeat the same tedious procedures
whenever they want to use csv, not to talk about the lack of performance
that implies?
Mar 13 '07 #9
sk**@pobox.com escreveu:
PauloThat is not the point. The problem is to have things generalized.

Well, perhaps. One of the goals of the csv module was to do things the way
Excel does things.
It would be nice because many csv users use it to export files to
spreadsheets and Excel is certainly a reference. But there may be a
difference! One thing is to do things as Excel does and another is
to write data in a way Excel understands. The later, IMO, is not so
complicated.

Ideally, that would include formatting elements with
locale sensitivity. I've been working on a csv module in Python, so I
decided to try the locale.format() calls in its writerow implementation. A
number of test cases broke as a result because apparently locale.format
doesn't do its job quite the same way Excel does.
locale.format does not need to write the same way Excel does. It just
should convert in a way Excel understands.
Anyway, a csv user needs to do that if it is not provided by csv.

Mar 13 '07 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

21
by: Michele Simionato | last post by:
I often feel the need to extend the string method ".endswith" to tuple arguments, in such a way to automatically check for multiple endings. For instance, here is a typical use case: if...
3
by: mrhicks | last post by:
Hello all, I have a question regarding efficeny and how to find the best approach when trying to find flag with in a structure of bit fields. I have several structures which look similar to ...
3
by: Codex Twin | last post by:
Hello apologies if this is the wrong newsgroup to be sending this to. Basically, I have an ASP.NET application that I am trying to force to use a proxy server settings. This can be done by...
7
by: Electric Co. | last post by:
Hello, note: This is for a Faculty web site that is undergoing a migration to an open source solution so my motives are legit. I need to build a relay from IIS handling URL_A to a PHP server...
34
by: samjnaa | last post by:
This is like the previous one. Please check for sanity and approve for posting at python-dev. I would like to have something like "option base" in Visual Basic. IIRC it used to allow me to...
7
by: ADN | last post by:
Hi, I am creating a custom HTTPModule to intercept the request of when the user is attempting to retrieve a session variable. For instance, if I set a session variable in my code like so: ...
5
by: hnshashi | last post by:
I have writtem kernel(2.4) module to commu. with user space appl. using netlink socket. I am getting compilation error. kernel module:-> #include <linux/skbuff.h> #include<linux/module.h> ...
3
by: Joseph Geretz | last post by:
I'm using the Request Filter documentation which can be found here: http://msdn.microsoft.com/en-us/library/system.web.httprequest.filter.aspx In this example, two filters are installed, one...
9
by: bgold12 | last post by:
Hey, I want to standardize the URL that users use to get to my site. For example, I want to change: http://example.com/ to: http://www.example.com/index.php and I want to change:
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.