python3 - the hardest hello world ever ?

[Helmut Jarausch]

Hi,

do I miss something (I do hope so) or is switching to Python3
really hard for Latin1-users?

My simplest hello world script - which uses a few German
umlaut characters - doesn't look very intuitive.
I have to set an internal property (with leading underscore)
for each output file I'm using - right?

#!/usr/local/bin/python3.0
# _*_ coding: latin1 _*_

import sys

# the following call doesn't do the job
# sys.setfilesystemencoding('latin1')

# but this ugly one (to be done for each output file)
sys.stdout._encoding='latin1'

print("Hallo, Süßes Python")
Thanks for any enlightening on that subject,
Helmut.

--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

[pjacobi.de]

Hi Helmut, All,

do I miss something (I do hope so) or is switching to Python3
really hard for Latin1-users?

It's as complicated as ever -- if you have used unicode strings
in the past (as the 3.0 strings now are always unicode strings).

# sys.setfilesystemencoding('latin1')

This cares about the character encoding in filenames, not
in file content.

sys.setdefaultencoding('iso-8859-1') # or 'latin1'
would do the job, but only in sitecustomize.py. After
initializing, the function is no longer available.

And using it in sitecustomize.py is sort of discouraged.

IMHO the assumptions the typical Python installation makes
about the character encoding used in the system are much too
conservative. E.g. under Windows it should it use
GetLocaleInfo (LOCALE_USER_DEFAULT, LOCALE_IDEFAULTANSICODEPAGE, ...).

Then a lot of things would work out of the box. Of course
including some methods to shoot yourself in the foot, which
you are prevented from by the current behaviour.
Regards,
Peter

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

do I miss something (I do hope so) or is switching to Python3

really hard for Latin1-users?

Why do you want to switch? sys.stdout.encoding should already be
iso-8859-1, if you are a Latin1-user.

Regards,
Martin

[Brian Quinlan]

Hey Helmut,

Did you try just:

print("Hallo, Süßes Python")

Cheers,
Brian

Helmut Jarausch wrote:

Hi,

do I miss something (I do hope so) or is switching to Python3
really hard for Latin1-users?

My simplest hello world script - which uses a few German
umlaut characters - doesn't look very intuitive.
I have to set an internal property (with leading underscore)
for each output file I'm using - right?

#!/usr/local/bin/python3.0
# _*_ coding: latin1 _*_

import sys

# the following call doesn't do the job
# sys.setfilesystemencoding('latin1')

# but this ugly one (to be done for each output file)
sys.stdout._encoding='latin1'

print("Hallo, Süßes Python")
Thanks for any enlightening on that subject,
Helmut.

[Ben Finney]

Helmut Jarausch <ja******@skynet.bewrites:

I have to set an internal property (with leading underscore)
for each output file I'm using - right?

If you're referring to the source encoding declaration: No,
underscores have no effect. The specification is at
<URL:http://www.python.org/doc/2.5.2/ref/encodings.html>.

#!/usr/local/bin/python3.0
# _*_ coding: latin1 _*_

I'm not sure why you use underscores in this line. The usual form is
to use a mode line as recognised by Emacs::

# -*- coding: latin1 -*-

or Vim::

# vim: fileencoding=latin1 :

--
\ “Pinky, are you pondering what I'm pondering?� “I think so, |
`\ Brain, but don't you need a swimming pool to play Marco Polo?� |
_o__) —_Pinky and The Brain_ |
Ben Finney

[Helmut Jarausch]

Martin v. Löwis wrote:

>do I miss something (I do hope so) or is switching to Python3
really hard for Latin1-users?

Why do you want to switch? sys.stdout.encoding should already be
iso-8859-1, if you are a Latin1-user.

What defines me as latin1-user?

commenting
# sys.stdout._encoding='latin1'
I get
Traceback (most recent call last):
File "latin1.py", line 8, in <module>

File "/usr/local/lib/python3.0/io.py", line 1485, in write
b = encoder.encode(s)
File "/usr/local/lib/python3.0/encodings/ascii.py", line 22, in encode
return codecs.ascii_encode(input, self.errors)[0]
UnicodeEncodeError: 'ascii' codec can't encode characters in position 1-2:
ordinal not in range(128)

So my system seems to be an ASCII system?
Thanks,
Helmut

--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

[Helmut Jarausch]

Ben Finney wrote:

Helmut Jarausch <ja******@skynet.bewrites:

>I have to set an internal property (with leading underscore)
for each output file I'm using - right?

If you're referring to the source encoding declaration: No,
underscores have no effect. The specification is at
<URL:http://www.python.org/doc/2.5.2/ref/encodings.html>.

>#!/usr/local/bin/python3.0
# _*_ coding: latin1 _*_

I'm not sure why you use underscores in this line. The usual form is
to use a mode line as recognised by Emacs::

# -*- coding: latin1 -*-

or Vim::

# vim: fileencoding=latin1 :

No, I meant the underscore in sys.stdout._encoding='latin1'
^

As for the source encoding, I have used the underscore version
which seems to work, as well.

Thanks,
Helmut.
--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

[Helmut Jarausch]

Brian Quinlan wrote:

Hey Helmut,

Did you try just:

print("Hallo, Süßes Python")

Yes, but that doesn't work here.
Please see my reply to Martin's reply.

Thanks,
Helmut.

--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

[Helmut Jarausch]

Brian Quinlan wrote:

Hey Helmut,

Did you try just:

print("Hallo, Süßes Python")

Yes, but that doesn't work here.
Please see my reply to Martin's reply.

Thanks,
Helmut.

--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

[Orestis Markou]

I would just use UTF-8 and be done with it.

Set your editor to write UTF-8 files, set the correct #coding at your
python script, make sure your terminal supports outputting UTF-8
characters (and your font has the correct glyphs) and everything
should be fine. No trickery required.

Even for Python 2.x, the only extra thing needed was the u"" kind of
strings. No other trickery in sys.stdout required. What platform do
you use?

Orestis
--
or*****@orestis.gr
http://orestis.gr/


On 15 Oct 2008, at 11:12, Helmut Jarausch wrote:

Brian Quinlan wrote:

>Hey Helmut,
Did you try just:
print("Hallo, Süßes Python")

Yes, but that doesn't work here.
Please see my reply to Martin's reply.

Thanks,
Helmut.

--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany
--
http://mail.python.org/mailman/listinfo/python-list

[Paul Boddie]

On 15 Okt, 12:08, Helmut Jarausch <[e-mail]>
wrote:

>
What defines me as latin1-user?

What does sys.stdout.encoding say? In Python 2.x, at least, that
attribute should reflect the capabilities of your environment
(specifically, the character encoding) and help determine whether it
makes sense for Python to try and encode Unicode objects (plain
strings in Python 3.x) using a particular output encoding when
printing those objects to the display.

Paul

[Helmut Jarausch]

Paul Boddie wrote:

On 15 Okt, 12:08, Helmut Jarausch <[e-mail]>
wrote:

>What defines me as latin1-user?

What does sys.stdout.encoding say? In Python 2.x, at least, that

It says ansi_x3.4-1968

Where can I change this?

attribute should reflect the capabilities of your environment
(specifically, the character encoding) and help determine whether it
makes sense for Python to try and encode Unicode objects (plain
strings in Python 3.x) using a particular output encoding when
printing those objects to the display.

Thanks,
Helmut.
--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

[Diez B. Roggisch]

Helmut Jarausch wrote:

Paul Boddie wrote:

>On 15 Okt, 12:08, Helmut Jarausch <[e-mail]>
wrote:

>>What defines me as latin1-user?

What does sys.stdout.encoding say? In Python 2.x, at least, that

It says ansi_x3.4-1968

Where can I change this?

By changing your console's terminal settings. See what

locale -a

outputs.

See this:
(devtools)dir@client8049:~$ locale -a
C
en_AU.utf8
en_BW.utf8
en_CA.utf8
en_DK.utf8
en_GB.utf8
en_HK.utf8
en_IE.utf8
en_IN
en_NZ.utf8
en_PH.utf8
en_SG.utf8
en_US.utf8
en_ZA.utf8
en_ZW.utf8
POSIX
(devtools)dir@client8049:~$ python
Python 2.5.2 (r252:60911, Jul 31 2008, 17:28:52)
[GCC 4.2.3 (Ubuntu 4.2.3-2ubuntu7)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
Welcome to rlcompleter2 0.96
for nice experiences hit <tabmultiple times

>>import sys
sys.stdout.encoding

'UTF-8'

>>>

Diez

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

What defines me as latin1-user?

That your locale is based on Latin-1, e.g. because it is a German
locale. How precisely that works depends on the operating system.

So my system seems to be an ASCII system?

At least that's what Python determined. If Python couldn't have found
out that you usually use Latin-1, your system is misconfigured. If
Python could have found out, but failed to do so, it's a bug in Python.

Regards,
Martin

[Helmut Jarausch]

Martin v. Löwis wrote:

>What defines me as latin1-user?

That your locale is based on Latin-1, e.g. because it is a German
locale. How precisely that works depends on the operating system.

>So my system seems to be an ASCII system?

At least that's what Python determined. If Python couldn't have found
out that you usually use Latin-1, your system is misconfigured. If
Python could have found out, but failed to do so, it's a bug in Python.

Many thanks, it works when setting the LANG environment variable.

Still, I wished it were possible call sys.setdefaultencoding
at the very beginning of a script.

Why isn't that possible?

Helmut.
--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

Still, I wished it were possible call sys.setdefaultencoding

at the very beginning of a script.

Why isn't that possible?

The default encoding was used when combining byte-oriented
text and unicode-oriented text. Such combination is no longer
supported, hence the notion of a default encoding
has disappeared. You have to perform conversion between bytes
and strings now explicitly.

Regards,
Martin

[Helmut Jarausch]

Martin v. Löwis wrote:

>Still, I wished it were possible call sys.setdefaultencoding
at the very beginning of a script.

Why isn't that possible?

The default encoding was used when combining byte-oriented
text and unicode-oriented text. Such combination is no longer
supported, hence the notion of a default encoding
has disappeared. You have to perform conversion between bytes
and strings now explicitly.

I meant setting the default encoding which is used by print (e.g.) when
outputting the internal unicode string to a file.
As far as I understood, currently I am fixed to setting either
the 'locale' or to switch settings for each output file (by settting
the _encoding property.
I wished I could override the locale settings within a Python script.

Thanks,
Helmut.
--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

[Paul Boddie]

On 16 Okt, 11:28, Helmut Jarausch <[e-mail]>
wrote:

>
I meant setting the default encoding which is used by print (e.g.) when
outputting the internal unicode string to a file.
As far as I understood, currently I am fixed to setting either
the 'locale' or to switch settings for each output file (by settting
the _encoding property.
I wished I could override the locale settings within a Python script.

You could use the locale module. ;-)

But seriously, I'd like to know whether the program I posted works
with Python 2.x because there could be differences between 2.x and
3.x, and we'd obviously like to solve your problems regardless of
which Python version you're using.

Paul

[Helmut Jarausch]

Paul Boddie wrote:

On 16 Okt, 11:28, Helmut Jarausch <[e-mail]>
wrote:

>I meant setting the default encoding which is used by print (e.g.) when
outputting the internal unicode string to a file.
As far as I understood, currently I am fixed to setting either
the 'locale' or to switch settings for each output file (by settting
the _encoding property.
I wished I could override the locale settings within a Python script.

You could use the locale module. ;-)

But seriously, I'd like to know whether the program I posted works
with Python 2.x because there could be differences between 2.x and
3.x, and we'd obviously like to solve your problems regardless of
which Python version you're using.

Yes, of course.
I have always worked with latin-1 strings with an US locale under
python-2.x with x < 6 (I haven't tried 2.6, though). I hope to switch to 3.0
as soon as possible.
--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

[=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=]

I meant setting the default encoding which is used by print (e.g.) when

outputting the internal unicode string to a file.

Having such a thing would be conceptually wrong. What encoding should
be used depends on the file - different files may have different
encodings. When opening a file, you need to specify the encoding.

As far as I understood, currently I am fixed to setting either
the 'locale' or to switch settings for each output file (by settting
the _encoding property.

That's not true. You can also specify the encoding when opening the file

I wished I could override the locale settings within a Python script.

You can monkey-patch locale.getpreferredencoding, which is used when
determining what encoding to use when opening new files. I don't
recommend doing so, though.

Regards,
Martin

[Ross Ridge]

Helmut Jarausch <ja******@skynet.bewrote:

># but this ugly one (to be done for each output file)
sys.stdout._encoding='latin1'

Is this writable "_encoding" attribute, with a leading underscore (_),
documented anywhere? Does it actually work? Would it happen to be
supported in 2.5 or 2.6? The fact that I can't change the encoding
attribute of sys.stdout/stderr/stdin has caused problems for me in
the past.

Ross Ridge

--
l/ // Ross Ridge -- The Great HTMU
[oo][oo] rr****@csclub.uwaterloo.ca
-()-/()/ http://www.csclub.uwaterloo.ca/~rridge/
db //

[Helmut Jarausch]

Ross Ridge wrote:

Helmut Jarausch <ja******@skynet.bewrote:

># but this ugly one (to be done for each output file)
sys.stdout._encoding='latin1'

Is this writable "_encoding" attribute, with a leading underscore (_),
documented anywhere? Does it actually work? Would it happen to be
supported in 2.5 or 2.6? The fact that I can't change the encoding
attribute of sys.stdout/stderr/stdin has caused problems for me in
the past.

Yes, it does work in python-3.

--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

[henning.vonbargen]

Many thanks, it works when setting the LANG environment variable.

BTW:
For Windows users, when running Python command-line programs,
you can also modify the properties of the "cmd.exe" window and
tell windows to use the TT Lucida Console font instead of the raster
font.

Then, before starting the Python program, do a
CHCP 1252
This way the sys.stdout.encoding will be cp1252
(tested with Python 2.4.3 and 2.5.1).