string.encode on HP-UX

[Richard Townsend]

Using Python-2.3.4 on HP-UX11i, the following code:

import locale
loc = locale.setlocal e(locale.LC_ALL )
print 'locale =', loc
loc = locale.nl_langi nfo(locale.CODE SET)
print 'locale =', loc
print 'hello'.encode( loc, 'replace')

produces:

locale = C C C C C C
locale = roman8
Traceback (most recent call last):
File "test_locale.py ", line 13, in ?
print 'hello'.encode( loc, 'replace')
LookupError: unknown encoding: roman8
[The same code on SUSE 9.1 doesn't raise an exception].

Should I be able to pass the value returned by nl_langinfo() to the
string.encode call?

Similar code is used by wxGlade and this exception prevents it from
running.

Does anybody know how to fix this on HP-UX?

--
Richard Townsend

[Michael Hudson]

Richard Townsend <ri******@edshk .demon.co.uk> writes:

Using Python-2.3.4 on HP-UX11i, the following code:

import locale
loc = locale.setlocal e(locale.LC_ALL )
print 'locale =', loc
loc = locale.nl_langi nfo(locale.CODE SET)
print 'locale =', loc
print 'hello'.encode( loc, 'replace')

produces:

locale = C C C C C C
locale = roman8
Traceback (most recent call last):
File "test_locale.py ", line 13, in ?
print 'hello'.encode( loc, 'replace')
LookupError: unknown encoding: roman8
[The same code on SUSE 9.1 doesn't raise an exception].

Should I be able to pass the value returned by nl_langinfo() to the
string.encode call?

What is roman8? If it's some hp-ux specific thingy, I guess the
solution is to teach Python what to do with it. If it's just HP's
name for iso-8859-1 or something then this is easy (mucking with
encodings.alias es).

If it's some custom encoding, finding out what unicode codepoint each
octet maps to and writing a codec a la macroman can't be impossibly
hard.

I guess a patch would be welcome either way.

Cheers,
mwh

--
Windows 2000: Smaller cow. Just as much crap.
-- Jim's pedigree of operating systems, asr

[Christopher T King]

On Wed, 21 Jul 2004, Richard Townsend wrote:

Using Python-2.3.4 on HP-UX11i, the following code:

import locale
loc = locale.setlocal e(locale.LC_ALL )
print 'locale =', loc
loc = locale.nl_langi nfo(locale.CODE SET)
print 'locale =', loc
print 'hello'.encode( loc, 'replace')

produces:

locale = C C C C C C
locale = roman8
Traceback (most recent call last):
File "test_locale.py ", line 13, in ?
print 'hello'.encode( loc, 'replace')
LookupError: unknown encoding: roman8

[The same code on SUSE 9.1 doesn't raise an exception].

My guess is roman8 is HP-UX's version of latin_1. Setting an alias fixes
this:

import encodings
encodings.alias es.aliases['roman8']='latin_1'
'hello'.encode( 'roman8')

'hello'

You can add those first two lines to a sitecustomize.p y file, located
somewhere in your Python path (generally ~/site-packages/ or
/usr/local/lib/python2.X/ should work).

[Michael Piotrowski]

Christopher T King <sq******@WPI.E DU> writes:

My guess is roman8 is HP-UX's version of latin_1. Setting an alias fixes
this:

No, HP Roman-8 is *not* the same as Latin 1 (ISO 8859-1). You can
find a table at, e.g., <http://www.kostis.net/charsets/hproman8.htm>.

--
Michael Piotrowski, M.A. <mx*@dynalabs.d e>
Public key at <http://www.dynalabs.de/mxp/pubkey.txt>

[Christopher T King]

On Wed, 21 Jul 2004, Michael Piotrowski wrote:

Christopher T King <sq******@WPI.E DU> writes:

My guess is roman8 is HP-UX's version of latin_1. Setting an alias fixes
this:

No, HP Roman-8 is *not* the same as Latin 1 (ISO 8859-1). You can
find a table at, e.g., <http://www.kostis.net/charsets/hproman8.htm>.

Well, the first 128 characters are the same. I'd say that's close enough,
right?
;)

[Michael Piotrowski]

Christopher T King <sq******@WPI.E DU> writes:

On Wed, 21 Jul 2004, Michael Piotrowski wrote:

Christopher T King <sq******@WPI.E DU> writes:

My guess is roman8 is HP-UX's version of latin_1. Setting an alias fixes
this:

No, HP Roman-8 is *not* the same as Latin 1 (ISO 8859-1). You can
find a table at, e.g., <http://www.kostis.net/charsets/hproman8.htm>.

Well, the first 128 characters are the same. I'd say that's close enough,
right?
;)

For Americans, perhaps ;-)

--
Michael Piotrowski, M.A. <mx*@dynalabs.d e>
Public key at <http://www.dynalabs.de/mxp/pubkey.txt>

[Richard Townsend]

Hi Christopher,

Thanks for your suggestion, however it produces two problems for me.

1. If I execute the code in the interpreter, it still fails like this:

import encodings
encodings.alias es.aliases['roman8']='latin_1'
'hello'.encode( 'roman8')

Traceback (most recent call last):
File "<pyshell#3 >", line 1, in -toplevel-
'hello'.encode( 'roman8')
LookupError: unknown encoding: roman8
2. If I put the code in site-packages/sitecustomize.p y, it fails like
this:

capulet:home/richardt > python -v
# installing zipimport hook
import zipimport # builtin
# installed zipimport hook
# /opt/python/lib/python2.3/site.pyc matches /opt/python/lib/python2.3/s
ite.py
import site # precompiled from /opt/python/lib/python2.3/site.pyc
# /opt/python/lib/python2.3/os.pyc matches /opt/python/lib/python2.3/os.
py
import os # precompiled from /opt/python/lib/python2.3/os.pyc
import posix # builtin
# /opt/python/lib/python2.3/posixpath.pyc matches /opt/python/lib/python
2.3/posixpath.py
import posixpath # precompiled from /opt/python/lib/python2.3/posixpath.
pyc
# /opt/python/lib/python2.3/stat.pyc matches /opt/python/lib/python2.3/s
tat.py
import stat # precompiled from /opt/python/lib/python2.3/stat.pyc
# /opt/python/lib/python2.3/UserDict.pyc matches /opt/python/lib/python2
..3/UserDict.py
import UserDict # precompiled from /opt/python/lib/python2.3/UserDict.py
c
# /opt/python/lib/python2.3/copy_reg.pyc matches /opt/python/lib/python2
..3/copy_reg.py
import copy_reg # precompiled from /opt/python/lib/python2.3/copy_reg.py
c
# /opt/python/lib/python2.3/types.pyc matches /opt/python/lib/python2.3/
types.py
import types # precompiled from /opt/python/lib/python2.3/types.pyc
# /opt/python/lib/python2.3/site-packages/sitecustomize.p yc matches
/opt/python/lib/python2.3/site-packages/sitecustomize.p y
import sitecustomize # precompiled from /opt/python/lib/python2.3/site-
packages/sitecustomize.p yc
import encodings # directory /opt/python/lib/python2.3/encodings
# /opt/python/lib/python2.3/encodings/__init__.pyc matches /opt/python/l
ib/python2.3/encodings/__init__.py
import encodings # precompiled from /opt/python/lib/python2.3/encodings/
__init__.pyc
# /opt/python/lib/python2.3/codecs.pyc matches /opt/python/lib/python2.3
/codecs.py
import codecs # precompiled from /opt/python/lib/python2.3/codecs.pyc
import _codecs # builtin
'import site' failed; traceback:
Traceback (most recent call last):
File "/opt/python/lib/python2.3/site.py", line 355, in ?
import sitecustomize
File "/opt/python/lib/python2.3/site-packages/sitecustomize.p y", line
7, in ?
encodings.alias es.aliases['roman8']='latin_1'
AttributeError: 'module' object has no attribute 'aliases'
# /opt/python/lib/python2.3/warnings.pyc matches /opt/python/lib/python2
..3/warnings.py
import warnings # precompiled from /opt/python/lib/python2.3/warnings.py
c
# /opt/python/lib/python2.3/linecache.pyc matches /opt/python/lib/python
2.3/linecache.py
import linecache # precompiled from /opt/python/lib/python2.3/linecache.
pyc
# /opt/python/lib/python2.3/encodings/aliases.pyc matches /opt/python/li
b/python2.3/encodings/aliases.py
import encodings.alias es # precompiled from /opt/python/lib/python2.3/en
codings/aliases.pyc
Python 2.3.4 (#3, May 28 2004, 13:24:19) [C] on hp-ux11
Type "help", "copyright" , "credits" or "license" for more information.

--
Richard Townsend

[Richard Townsend]

Further, if I put the following in sitecustomize.p y:

import encodings
print dir(encodings)

I get:

['CodecRegistryE rror', '__builtins__', '__doc__', '__file__',
'__name__', '__path__', '_cache', '_import_tail',
'_norm_encoding _map', '_unknown', 'codecs', 'exceptions',
'normalize_enco ding', 'search_functio n', 'types']

Notice there is no 'aliases' attribute.

But if I then run this interactively:

import encodings
dir(encodings)

['CodecRegistryE rror', '__builtins__', '__doc__', '__file__',
'__name__', '__path__', '_cache', '_import_tail',
'_norm_encoding _map', '_unknown', 'aliases', 'codecs', 'exceptions',
'normalize_enco ding', 'search_functio n', 'types']

then the 'aliases' attribute is there.
--
Richard Townsend

[Christopher T King]

On Thu, 22 Jul 2004, Richard Townsend wrote:

Further, if I put the following in sitecustomize.p y:

import encodings
print dir(encodings)

I get:

['CodecRegistryE rror', '__builtins__', '__doc__', '__file__',
'__name__', '__path__', '_cache', '_import_tail',
'_norm_encoding _map', '_unknown', 'codecs', 'exceptions',
'normalize_enco ding', 'search_functio n', 'types']

Notice there is no 'aliases' attribute.

Oops, I had only tested it at the prompt :P I had assumed sitecustomize.p y
was run after everything was set up.

This code uses a more defined interface for altering the codecs registry,
uses 'ascii' instead of 'latin_1' (to prevent some confusion), and
I've actually tested it in sitecustomize.p y:

import codecs

def roman8(n):
if n=='roman8':
return codecs.lookup(' ascii')

codecs.register (roman8)

This achieves the same effect as the aliases trick (which I'm surprised
didn't work for you at the prompt), but is less tricksy and should
therefore work better.

[Richard Brodie]

"Christophe r T King" <sq******@WPI.E DU> wrote in message
news:Pi******** *************** *************** @ccc2.wpi.edu.. .

This achieves the same effect as the aliases trick (which I'm surprised
didn't work for you at the prompt), but is less tricksy and should
therefore work better.

I've put what should be a proper codec as a patch on SF. Really needs
testing on HP-UX though....