Connecting Tech Pros Worldwide Forums | Help | Site Map

[unicode] inconvenient unicode conversion of non-string arguments

Holger Joukl
Guest
 
Posts: n/a
#1: Dec 13 '06

Hi there,

I consider the behaviour of unicode() inconvenient wrt to conversion of
non-string
arguments.
While you can do:
Quote:
Quote:
Quote:
>>unicode(17.3)
u'17.3'

you cannot do:
Quote:
Quote:
Quote:
>>unicode(17.3, 'ISO-8859-1', 'replace')
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: coercing to Unicode: need string or buffer, float found
Quote:
Quote:
Quote:
>>>
This is somehow annoying when you want to convert a mixed-type argument
list
to unicode strings, e.g. for a logging system (that's where it bit me) and
want to make sure that possible raw string arguments are also convertedto
unicode without errors (although by force).
Especially as this is a performance-critical part in my application so I
really
do not like to wrap unicode() into some custom tounicode() function that
handles
such cases by distinction of argument types.

Any reason why unicode() with a non-string argument should not allow the
encoding and errors arguments?
Or some good solution to work around my problem?

(Currently running on python 2.4.3)

Regards,
Holger

Der Inhalt dieser E-Mail ist vertraulich. Falls Sie nicht der angegebene
Empfänger sind oder falls diese E-Mail irrtümlich an Sie adressiertwurde,
verständigen Sie bitte den Absender sofort und löschen Sie die E-Mail
sodann. Das unerlaubte Kopieren sowie die unbefugte Übermittlung sindnicht
gestattet. Die Sicherheit von Übermittlungen per E-Mail kann nicht
garantiert werden. Falls Sie eine Bestätigung wünschen, fordern Siebitte
den Inhalt der E-Mail als Hardcopy an.

The contents of this e-mail are confidential. If you are not the named
addressee or if this transmission has been addressed to you in error,
please notify the sender immediately and then delete this e-mail. Any
unauthorized copying and transmission is forbidden. E-Mail transmission
cannot be guaranteed to be secure. If verification is required, please
request a hard copy version.



Leo Kislov
Guest
 
Posts: n/a
#2: Dec 13 '06

re: [unicode] inconvenient unicode conversion of non-string arguments



Holger Joukl wrote:
Quote:
Hi there,
>
I consider the behaviour of unicode() inconvenient wrt to conversion of
non-string
arguments.
While you can do:
>
Quote:
Quote:
>unicode(17.3)
u'17.3'
>
you cannot do:
>
Quote:
Quote:
>unicode(17.3, 'ISO-8859-1', 'replace')
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: coercing to Unicode: need string or buffer, float found
Quote:
Quote:
>>
>
This is somehow annoying when you want to convert a mixed-type argument
list
to unicode strings, e.g. for a logging system (that's where it bit me) and
want to make sure that possible raw string arguments are also converted to
unicode without errors (although by force).
Especially as this is a performance-critical part in my application so I
really
do not like to wrap unicode() into some custom tounicode() function that
handles
such cases by distinction of argument types.
>
Any reason why unicode() with a non-string argument should not allow the
encoding and errors arguments?
There is reason: encoding is a property of bytes, it is not applicable
to other objects.
Quote:
Or some good solution to work around my problem?
Do not put undecoded bytes in a mixed-type argument list. A rule of
thumb working with unicode: decode as soon as possible, encode as late
as possible.

-- Leo

Fredrik Lundh
Guest
 
Posts: n/a
#3: Dec 13 '06

re: [unicode] inconvenient unicode conversion of non-string arguments


Holger Joukl wrote:
Quote:
Ok, but I still don't see why these arguments shouldn't simply be silently
ignored
Quote:
Quote:
Quote:
>>import this
</F>

Leo Kislov
Guest
 
Posts: n/a
#4: Dec 13 '06

re: [unicode] inconvenient unicode conversion of non-string arguments



Holger Joukl wrote:
Quote:
python-list-bounces+holger.joukl=lbbw.de@python.org schrieb am 13.12.2006
11:02:30:
>
Quote:

Holger Joukl wrote:
Quote:
Hi there,
>
I consider the behaviour of unicode() inconvenient wrt to conversion of
non-string
arguments.
While you can do:
>
>unicode(17.3)
u'17.3'
>
you cannot do:
>
>unicode(17.3, 'ISO-8859-1', 'replace')
Traceback (most recent call last):
File "<stdin>", line 1, in ?
TypeError: coercing to Unicode: need string or buffer, float found
>>
[...]
Any reason why unicode() with a non-string argument should not allow
the
Quote:
Quote:
encoding and errors arguments?
There is reason: encoding is a property of bytes, it is not applicable
to other objects.
>
Ok, but I still don't see why these arguments shouldn't simply be silently
ignored
for non-string arguments.
That's rather bizzare and sloppy approach. Should

unicode(17.3, 'just-having-fun', 'I-do-not-like-errors')
unicode(17.3, 'sdlfkj', 'ewrlkj', 'eoirj', 'sdflkj')

work?

Quote:
Quote:
Quote:
Or some good solution to work around my problem?
Do not put undecoded bytes in a mixed-type argument list. A rule of
thumb working with unicode: decode as soon as possible, encode as late
as possible.
>
It's not always that easy when you deal with a tree data structure with the
tree elements containing different data types and your user may decide to
output
root.element.subelement.whateverData.
I have the problems in a logging mechanism, and it would vanish if
unicode(<non-string>, encoding, errors) would work and just ignore the
obsolete
arguments.
I don't really see from your example what stops you from putting
unicode instead of bytes into your tree, but I can believe some
libraries can cause some extra work. That's the problem with libraries,
not with builtin function unicode(). Would you be happy if floating
point value 17.3 would be stored as 8 bytes in your tree? After all,
that is how 17.3 is actually represented in computer memory. Same story
with unicode, if some library gives you raw bytes *you* have to do
extra work later.

-- Leo

Marc 'BlackJack' Rintsch
Guest
 
Posts: n/a
#5: Dec 13 '06

re: [unicode] inconvenient unicode conversion of non-string arguments


In <mailman.1508.1166002784.32031.python-list@python.org>, Holger Joukl
wrote:
Quote:
Der Inhalt dieser E-Mail ist vertraulich. Falls Sie nicht der angegebene
Empfänger sind oder falls diese E-Mail irrtümlich an Sie adressiert wurde,
verständigen Sie bitte den Absender sofort und löschen Sie die E-Mail
sodann. Das unerlaubte Kopieren sowie die unbefugte Übermittlung sind nicht
gestattet. Die Sicherheit von Übermittlungen per E-Mail kann nicht
garantiert werden. Falls Sie eine Bestätigung wünschen, fordern Sie bitte
den Inhalt der E-Mail als Hardcopy an.
>
The contents of this e-mail are confidential. If you are not the named
addressee or if this transmission has been addressed to you in error,
please notify the sender immediately and then delete this e-mail. Any
unauthorized copying and transmission is forbidden. E-Mail transmission
cannot be guaranteed to be secure. If verification is required, please
request a hard copy version.
Maybe you should rethink if it really makes sense to add this huge block
of "nonsense" to a post to a newsgroup or public mailing list. If it's
confidential, just keep it secret. ;-)

Ciao,
Marc 'BlackJack' Rintsch
Ben Finney
Guest
 
Posts: n/a
#6: Dec 14 '06

re: [unicode] inconvenient unicode conversion of non-string arguments


"Marc 'BlackJack' Rintsch" <bj_666@gmx.netwrites:
Quote:
In <mailman.1508.1166002784.32031.python-list@python.org>, Holger Joukl
wrote:
Quote:
[a meaningless disclaimer text at the bottom of every message]
>
Maybe you should rethink if it really makes sense to add this huge
block of "nonsense" to a post to a newsgroup or public mailing list.
If it's confidential, just keep it secret. ;-)
In all likelihood, the OP isn't choosing specifically to attach it;
these things are often done to *every* outgoing message at an
organisational level by people who don't think the issue through very
well.

<URL:http://goldmark.org/jeff/stupid-disclaimers/>

Please, those with such badly-configured systems, discuss the issue of
public discussion forums with the boneheads who think these disclaimer
texts are a good idea and at least try to change that behaviour.

Alternatively, post from some other mail system that doesn't slap
these obnoxious blocks onto your messages.

--
\ "I wish there was a knob on the TV to turn up the intelligence. |
`\ There's a knob called 'brightness' but it doesn't work." -- |
_o__) Eugene P. Gallagher |
Ben Finney

Closed Thread