471,310 Members | 1,359 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,310 software developers and data experts.

a question about unicode in python

i have a python source code test.py

# -*- coding: UTF-8 -*-

# s is a unicode string, include chinese
s = u''

then i run

$ python test.py
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 0-1:
invalid data

by in python interactive, it is right
>>s = u''
why?

Jun 12 '07 #1
5 1298
In <11*********************@i13g2000prf.googlegroups. com>, hzqij wrote:
i have a python source code test.py

# -*- coding: UTF-8 -*-

# s is a unicode string, include chinese
s = u'*三'

then i run

$ python test.py
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 0-1:
invalid data

by in python interactive, it is right
>>>s = u'*三'

why?
Does the "coding comment" match the actual encoding of the source file?

Ciao,
Marc 'BlackJack' Rintsch
Jun 12 '07 #2
hzqij schrieb:
i have a python source code test.py

# -*- coding: UTF-8 -*-

# s is a unicode string, include chinese
s = u''

then i run

$ python test.py
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 0-1:
invalid data

by in python interactive, it is right
>>>s = u''

why?

just an idea: is your text editor really supporting utf-8? In the mail
it is only displayed as '??' which looks for me as the mail editor did
not send the mail as utf. Try to attach a correct text file.

Jun 12 '07 #3
On 6/12/07, hzqij <hz*******@gmail.comwrote:
i have a python source code test.py

# -*- coding: UTF-8 -*-
As Marc pointed out, you should test the actual file encoding of the
program to check that it is, in fact, UTF-8 encoded. If you're on a
Unix/Linux system you should be able to test for a UTF-8 encoded file
using the "file" command, e.g.

evan@dhcp-10-10-7-101 ~ $ file ~/uni.py
/home/evan/uni.py: UTF-8 Unicode text

--
Evan Klitzke <ev**@yelp.com>
Jun 12 '07 #4
On Jun 12, 12:29 pm, "Evan Klitzke" <e...@yelp.comwrote:
On 6/12/07, hzqij <hzqij1...@gmail.comwrote:
i have a python source code test.py
# -*- coding: UTF-8 -*-

As Marc pointed out, you should test the actual file encoding of the
program to check that it is, in fact, UTF-8 encoded. If you're on a
Unix/Linux system you should be able to test for a UTF-8 encoded file
using the "file" command, e.g.

evan@dhcp-10-10-7-101 ~ $ file ~/uni.py
/home/evan/uni.py: UTF-8 Unicode text

--
Evan Klitzke <e...@yelp.com>
If you're using IDLE to edit the source with, you can set IDLE to
encode in utf8 by going to Options, Configure IDLE, General Tab, and
change the Default Source Encoding to utf-8.

Mike

Jun 12 '07 #5
2007/6/12, WolfgangZ <wo****@gmx.net>:
hzqij schrieb:
i have a python source code test.py

# -*- coding: UTF-8 -*-

# s is a unicode string, include chinese
s = u'*三'

then i run

$ python test.py
UnicodeDecodeError: 'utf8' codec can't decode bytes in position 0-1:
invalid data

by in python interactive, it is right
>>s = u'*三'
why?

just an idea: is your text editor really supporting utf-8? In the mail
it is only displayed as '??' which looks for me as the mail editor did
not send the mail as utf. Try to attach a correct text file.
That must be your mail client, not his text editor or mail client. I
do see two Chinese characters in the message.

--
Andre Engels, an*********@gmail.com
ICQ: 6260644 -- Skype: a_engels
Jun 13 '07 #6

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

3 posts views Thread by yzzzzz | last post: by
4 posts views Thread by Guilherme Salgado | last post: by
2 posts views Thread by Neil Schemenauer | last post: by
1 post views Thread by Kenneth McDonald | last post: by
19 posts views Thread by Thomas W | last post: by
7 posts views Thread by JTree | last post: by
7 posts views Thread by 7stud | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.