2.3: test test_normalization failed

[Hans-Joachim Widmaier]

When I compiled Python 2.3 at home, everything went well. make test
reported no problems. At work, though, I got:

test test_normalization failed -- 03F9;03F9;03F9;03A3;03A3;

At first I thought it might be a problem with a somewhat inconsistently
upgraded system, but a few looks showed me the real difference.
At home I had updated Python quite frequently and downloaded
NormalizationTest.txt some months ago. At work, I downloaded it last week.
With revision 3.2.0 of said file the test passes, with revision 4.0.0 it
fails.

I have almost zip knowledge of the whole unicode business 'til now, so
I'm not overly concerned. Was 3.2.0 buggy? Is 4.0.0 buggy? Are there
different, incompatible specs? (Now _that_ I can't think of!)

Whatever, maybe someone can give a little insight.

Thanks for a great 2.3!

Hans-Joachim

[Michael Hudson]

Hans-Joachim Widmaier <hj********@web.de> writes:

When I compiled Python 2.3 at home, everything went well. make test
reported no problems. At work, though, I got:

test test_normalization failed -- 03F9;03F9;03F9;03A3;03A3;

At first I thought it might be a problem with a somewhat inconsistently
upgraded system, but a few looks showed me the real difference.
At home I had updated Python quite frequently and downloaded
NormalizationTest.txt some months ago. At work, I downloaded it last week.
With revision 3.2.0 of said file the test passes, with revision 4.0.0 it
fails.
This is known: Python from CVS explicitly requests the 3.2 file.
I have almost zip knowledge of the whole unicode business 'til now, so
I'm not overly concerned. Was 3.2.0 buggy? Is 4.0.0 buggy? Are there
different, incompatible specs? (Now _that_ I can't think of!)

I would imagine it being something like unicode 4 adding more
characters, but Python 2.3 -- being built on 3.2 normalization data --
doesn't know how these new character compose or decompose. Just a
guess.

Cheers,
mwh

--
Enlightenment is probably antithetical to impatience.
-- Erik Naggum, comp.lang.lisp

[Hans-Joachim Widmaier]

Michael Hudson <mw*@python.net> wrote in message news:<7h*************@pc150.maths.bris.ac.uk>...

[test failure with 4.0.0]

This is known: Python from CVS explicitly requests the 3.2 file.
Maybe test_normalization.py should look at the file version itself and
tell the unsuspicious that it really wants the 3.2 version.
I would imagine it being something like unicode 4 adding more
characters, but Python 2.3 -- being built on 3.2 normalization data --
doesn't know how these new character compose or decompose. Just a
guess.

Sounds reasonable. So the 2.3 normalization is just incomplete
regarding the newest standard.

Dismissed.

Thanks,
Hans-Joachim

[Michael Hudson]

hj********@web.de (Hans-Joachim Widmaier) writes:

Michael Hudson <mw*@python.net> wrote in message news:<7h*************@pc150.maths.bris.ac.uk>...

[test failure with 4.0.0]

This is known: Python from CVS explicitly requests the 3.2 file.

Maybe test_normalization.py should look at the file version itself and
tell the unsuspicious that it really wants the 3.2 version.

It's kind of hard to do that for the already released 2.3 :-)

I would imagine it being something like unicode 4 adding more
characters, but Python 2.3 -- being built on 3.2 normalization data --
doesn't know how these new character compose or decompose. Just a
guess.

Sounds reasonable. So the 2.3 normalization is just incomplete
regarding the newest standard.

Yep. I would guess it likely that 2.4 will be up to date again, but
it's not really my field.

Cheers,
mwh

--
Java is a WORA language! (Write Once, Run Away)
-- James Vandenberg (on pr*******@egroups.com)
& quoted by David Rush on comp.lang.scheme

[Martin v. Löwis]

Michael Hudson <mw*@python.net> writes:

Sounds reasonable. So the 2.3 normalization is just incomplete
regarding the newest standard.

Yep. I would guess it likely that 2.4 will be up to date again, but
it's not really my field.

It will be tricky to update. Python now also supports IDNA
(encodings/idna.py), which implements RFC 3490/91/92, which requires
the Unicode 3.2 database; updating to 4.0 would be a violation of that
RFC. So if we want to update the database, we need to find a way still
to keep the old one.

Regards,
Martin