compile(unicode) & source encoding

[Denis S. Otkidach]

I have 2 questions about compile

1. What is the reason to encode source code to utf-8? I guess
"python -U" behavior is the best choice in this case. The
following snippet seems to fix this, but I'm not sure it's
correct in all cases:

for const in code.co_consts:
if type(const) is str:
consts.append(const.decode('utf-8'))
else:
consts.append(const)
import new
code = new.code(code.co_argcount, code.co_nlocals,
code.co_stacksize, code.co_flags, code.co_code,
tuple(consts), code.co_names, code.co_varnames,
code.co_filename, code.co_name,
code.co_firstlineno, code.co_lnotab)
return code
2. Why there is now option to define encoding of source passed to
compile function? Yes, I can define encoding by prefixing my
code with '# -*- coding: %s -*-\n' % encoding, but it's not
handy.

--
Denis S. Otkidach
http://www.python.ru/ [ru]

[Just]

In article <ma*************************************@python.or g>,
"Denis S. Otkidach" <od*@strana.ru> wrote:

2. Why there is now option to define encoding of source passed to
compile function? Yes, I can define encoding by prefixing my
code with '# -*- coding: %s -*-\n' % encoding, but it's not
handy.

You can just pass a unicode string (in 2.3 at least), so you can do the
decoding yourself.

Just

[Martin v. Löwis]

"Denis S. Otkidach" <od*@strana.ru> writes:

[In the context of compile()]

1. What is the reason to encode source code to utf-8?
I think it is because Just found that the most straight-forward thing
to do: UTF-8 supports all Unicode characters, and you have to choose
*some* encoding, as the resulting byte strings need to be encoded
somehow.

In addition, anybody desiring a different encoding can explicitly
invoke .encode()
2. Why there is now option to define encoding of source passed to
compile function?

No, although this should be implemented. It is not implemented at this
point because it is very hard to do, and I have not found a good way
to do it, yet.

Regards,
Martin