Tkinter: unicode inconsistency on Windows

[Eric Brunel]

Hi all,

I'm in the process of localizing a Tkinter application and I'm struggling with a
problem for which I can't figure out a solution. The following code:

-------------------------------
from Tkinter import *

root = Tk()
root.tk.call('encoding', 'system', 'utf-8')
mb = Menu(root)
root.configure(menu=mb)
m = Menu(mb)
## s1 is UTF8 japanese for hi.ra.ga.na
s1 = '\xe3\x81\xb2\xe3\x82\x89\xe3\x81\x8c\xe3\x81\xaa'
mb.add_cascade(label=s1, menu=m)
## s2 is UTF8 japanese for KA.TA.KA.NA
s2 = '\xe3\x82\xab\xe3\x82\xbf\xe3\x82\xab\xe3\x83\x8a'
m.add_cascade(label=s2)

root.mainloop()
-------------------------------

works perfectly well on Unix (Linux Mandrake 8.0), but not on Windows 2k: the
menu item is correctly displayed as japanese katakana characters, but the menu
title is displayed incorrectly in a way that looks like a wrong encoding has
been used. I tried to explicitely pass unicode strings to Tkinter methods
(unicode(s?, 'utf-8')) and/or to force the system encoding at tcl/tk's level via
root.tk.call('encoding', 'system', 'utf-8'), but nothing seems to help.

AFAICS, the menu bar accepts only titles encoded in latin1; any other encoding
ends up messed up.

So I'm stuck: I can't find any way to display menu titles in anything else than
latin1 under Windows. Here are the versions I use:
- Python 2.1.1
- tcl/tk 8.3.2
- Windows 2k, french version if it changes anything

TIA
--
- Eric Brunel <eric dot brunel at pragmadev dot com> -
PragmaDev : Real Time Software Development Tools - http://www.pragmadev.com

[Eric Brunel]

Eric Brunel wrote:

Hi all,

I'm in the process of localizing a Tkinter application and I'm
struggling with a problem for which I can't figure out a solution. The
following code:

-------------------------------
from Tkinter import *

root = Tk()
root.tk.call('encoding', 'system', 'utf-8')
mb = Menu(root)
root.configure(menu=mb)
m = Menu(mb)
## s1 is UTF8 japanese for hi.ra.ga.na
s1 = '\xe3\x81\xb2\xe3\x82\x89\xe3\x81\x8c\xe3\x81\xaa'
mb.add_cascade(label=s1, menu=m)
## s2 is UTF8 japanese for KA.TA.KA.NA
s2 = '\xe3\x82\xab\xe3\x82\xbf\xe3\x82\xab\xe3\x83\x8a'
m.add_cascade(label=s2)

root.mainloop()
-------------------------------

works perfectly well on Unix (Linux Mandrake 8.0), but not on Windows
2k: the menu item is correctly displayed as japanese katakana
characters, but the menu title is displayed incorrectly in a way that
looks like a wrong encoding has been used. I tried to explicitely pass
unicode strings to Tkinter methods (unicode(s?, 'utf-8')) and/or to
force the system encoding at tcl/tk's level via root.tk.call('encoding',
'system', 'utf-8'), but nothing seems to help.

AFAICS, the menu bar accepts only titles encoded in latin1; any other
encoding ends up messed up.

So I'm stuck: I can't find any way to display menu titles in anything
else than latin1 under Windows. Here are the versions I use:
- Python 2.1.1
- tcl/tk 8.3.2
- Windows 2k, french version if it changes anything

Replying to myself: I gathered a few information from several sources and I
finally concluded that the problem lied in the file FixTk.py, as described in
http://minilien.com/?UThwyhrIGF

The reason for this problem is that the tcl/tk libraries get included before the
correct setup allowing them to find their available encodings was made.

So I replaced the file FixTk.py in <python install dir>/Lib/lib-tk by:

---FixTk.py--------------------
import sys, os
ver = '8.3' # You may have to replace this with your actual tcl/tk version
for t in "tcl", "tk":
v = os.path.join(sys.prefix, "tcl", t+ver)
if os.path.exists(os.path.join(v, "tclIndex")):
os.environ[t.upper() + "_LIBRARY"] = v
-------------------------------

This does not really solve the problem, since japanese characters still do not
show up in menu titles, even if a japanese font is installed. But it works with
the current Windows language: menu titles in french with a french version of
Windows are correctly displayed.
--
- Eric Brunel <eric dot brunel at pragmadev dot com> -
PragmaDev : Real Time Software Development Tools - http://www.pragmadev.com