How to emit Cyrillic and Chinese via unicode from console mode?

[Siegfried Heintze]

Can someone point me to an example of a little program that emits non-ascii
Unicode characters (Russian or Chinese perhaps)? The unicode
Russian/Cyrillic alphabet starts at 0x410. Is this possible to do in a
console mode program? If not, I guess I would want to use pywin32 to create
a window and a message pump and display it there. I anticipate using pywin32
for some other function calls.

Thanks!
Siegfried

[rs387]

On Sep 14, 2:03Â*am, "Siegfried Heintze" <[e-mail]:

Can someone point me to an example of a little program that emits non-ascii
Unicode characters (Russian or Chinese perhaps)?

The following doesn't quite work, but I'll post it anyway since it
actually ends up printing the characters. Perhaps someone can point
out what causes the exception at the end?

The important thing is to set the console codepage to 65001, which is
UTF-8. This lets you output utf8-encoded text and see the Unicode
chars displayed.

import sys
import encodings.utf_8
import win32console

sys.stdout = encodings.utf_8.StreamWriter(sys.stdout)

win32console.SetConsoleCP(65001)
win32console.SetConsoleOutputCP(65001)

s = "English: ok\n"
s += u'Russian: \u0420\u0443\u0441\u0441\u043a\u0438\u0439\n'
s += u'Greek: \u03bc\u03b5\u03b3\u03b1\u03bb\u03cd
\u03c4\u03b5\u03c1\u03b7\n'

print s

If redirected to file, all is well, this prints everything properly in
UTF-8. If ran on the console, this also prints everything correctly,
but then throws a mysterious exception:

English: ok
Russian: Ð*уÑ�Ñ�кий
Greek: μεγαλ�τε�η
Traceback (most recent call last):
File "I:\Temp\utf8console.py", line 18, in <module>
print s
File "C:\Progs\Python25\lib\codecs.py", line 304, in write
self.stream.write(data)
IOError: [Errno 0] Error

Any ideas?

Roman

P.S. This really ought to Just Work in this day and age, and do so
without all those 65001/utf8 incantations. Pity that it doesn't. Sigh.

[Gertjan Klein]

rs387 wrote:

>sys.stdout = encodings.utf_8.StreamWriter(sys.stdout)

win32console.SetConsoleCP(65001)
win32console.SetConsoleOutputCP(65001)

[...]

>If redirected to file, all is well, this prints everything properly in
UTF-8. If ran on the console, this also prints everything correctly,
but then throws a mysterious exception:

Interesting. On my system (Windows XP) the console codepage does not
change, and hence the characters don't print properly (I get some of the
CP437 line drawing characters, for example). I have never been able to
convince windows to assume/support UTF-8 encoding in the console,
programatically or otherwise. :(

Gertjan.

--
Gertjan Klein <gk****@xs4all.nl>

[rs387]

On Sep 14, 11:51*am, Gertjan Klein <[e-mail]:

Interesting. On my system (Windows XP) the console codepage does not
change, and hence the characters don't print properly (I get some of the
CP437 line drawing characters, for example). I have never been able to
convince windows to assume/support UTF-8 encoding in the console,
programatically or otherwise. :(

I found that a useful test is to create a directory whose name
contains chars from various languages, then run "cmd" and see if "dir"
displays them correctly. This worked fine for me in WinXP, even though
my system locale is Russian. Would be interesting to know if you can
get the console to display international chars this way.

[pataphor]

On Sun, 14 Sep 2008 01:02:39 -0700 (PDT)
rs387 <rs******@gmail.comwrote:

On Sep 14, 2:03Â*am, "Siegfried Heintze" <[e-mail]:

Can someone point me to an example of a little program that emits
non-ascii Unicode characters (Russian or Chinese perhaps)?

The following doesn't quite work, but I'll post it anyway since it
actually ends up printing the characters.

That's more like it! Just answer with whatever one has. Here's another
gem:

from Tkinter import *
from collections import deque

def byn(x,n =5 ):
L = deque(x)
R = []
while L:
R.append(L.popleft())
if len(R) == n:
yield ''.join(R)
R = []
if R:
yield ''.join(R)

root = Tk()
start = int('16A6',16)
end = int('16F0',16)
g = (unichr(i) for i in xrange(start, end+1))
L = byn(g,16)
s = '\n'.join(L)
w = Label(root, text=s,font = ("freemono","80"))
w.pack()

root.mainloop()

P.

[=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=]

Can someone point me to an example of a little program that emits non-ascii

Unicode characters (Russian or Chinese perhaps)? The unicode
Russian/Cyrillic alphabet starts at 0x410. Is this possible to do in a
console mode program? If not, I guess I would want to use pywin32 to create
a window and a message pump and display it there. I anticipate using pywin32
for some other function calls.

It also depends on your console. On Linux, with an UTF-8
capable-and-enabled console,

pyprint u"\u0413"
Г

work just fine.

Regards,
Martin