471,353 Members | 1,579 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,353 software developers and data experts.

How do I display unicode-paths?

I've been trying to stay blissfully unaware of Unicode, however now it seems like it's my turn. From the outside it seems like a rather massive subject, so any pointers as to where I should _start_ reading would be appreciated. The usecase is a class:

class path(object):
...
def __str__(self):
return self.pathstr.encode(???)

the question is what to put at ??? to be most useful to programmers/end users?

Background: I've got a directory with a file called 'bæ', B-AE, (which everyone knows is what Norwegian sheep say :-). If I type u'bæ' at the Python command prompt, it returns:

u'b\x91'

(WinXP, 2.3.2, regular command window). With a little trial and error I found:
print u'b\x91'.encode('latin1') bæ

although
print u'b\x91'

Traceback (most recent call last):
File "<stdin>", line 1, in ?
File "e:\python23\lib\encodings\cp437.py", line 18, in encode
return codecs.charmap_encode(input,errors,encoding_map)
UnicodeEncodeError: 'charmap' codec can't encode character '\u91' in position 1:
character maps to <undefined>

not sure I'm understanding this, and when I call:

os.listdir(os.getcwdu())

I get back

u'b\xe6'

which isn't making much sense either...

help?!

-- bjorn

Jul 18 '05 #1
1 2120
Pettersen, Bjorn S wrote:
I've been trying to stay blissfully unaware of Unicode, however now it seems like it's my turn. From the outside it seems like a rather massive subject, so any pointers as to where I should _start_ reading would be appreciated. The usecase is a class:

class path(object):
...
def __str__(self):
return self.pathstr.encode(???)

the question is what to put at ??? to be most useful to programmers/end users?
It is not possible to put something "most useful" here. If you want to
convert a Unicode string to a byte string, you typically do so because
you need to put data into a byte-oriented channel (such as a file, a
socket, or a terminal). You *do* need to know the encoding of that
channel to convert the Unicode object. Unfortunately, __str__ has no way
of knowing where data will be put.

If you can assume that the bytes returned are displayed to the user on
the local system, you can try locale.getpreferredencoding(). This should
work in most cases, with the notable exception of a cmd.exe window on MS
Windows: the terminal uses an encoding *different* from the user's
preferred encoding, and it is impossible to know in advance what that
encoding will be.

So in short: Just don't convert the Unicode string into a byte string,
until you know exactly where it will be displayed.
Background: I've got a directory with a file called 'bæ', B-AE, (which everyone knows is what Norwegian sheep say :-). If I type u'bæ' at the Python command prompt, it returns:

u'b\x91'


Unfortunately, this did not work very well: Try the same in IDLE, and
you will get u'b\xe6'

Unicode character U+00E6 really is LATIN SMALL LETTER AE:
u"\N{LATIN SMALL LETTER AE}"

u'\xe6'

Unicode character U+0091 is *not* AE, it is some (useless) control
character. The problem is that the windows console was using MS CP850,
but Python was interpreting it as Latin-1 (see PEP 263).

Print u"\xe6" on the console should work fine, in Python 2.3 (since
Python finds out, at the time of putting out the bytes, what the
encoding of the console is).

Regards,
Martin

Jul 18 '05 #2

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

1 post views Thread by Posadas, Dennis | last post: by
reply views Thread by JJY | last post: by
33 posts views Thread by selowitch | last post: by
6 posts views Thread by Bill Nguyen | last post: by
6 posts views Thread by coldpizza | last post: by
1 post views Thread by jackbenimble999 | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.