AJAX in prototype.js vs ISO-8859-1

[Jean-Philippe Encausse]

Hi,

I got 2 bugs using AJAX with ISO-8859-1:

1. While serializing form's value using prototype.js I lost accent
éàè ... because it use encodeURIComponent() function. I saw on
google, for ISO-8859-1 encoding I have to use escape(). It works fine
!

2. While receving content from AJAX Request with header <?xml
version='1.0' encoding='ISO-8859-1'?> I lost accent éàè ... only
under IE 6 it 's work fine under firefox.

What should I do ;-( ?

Best Regards,
Jean-Philippe Encausse

[Martin Honnen]

Jean-Philippe Encausse wrote:

I got 2 bugs using AJAX with ISO-8859-1:

1. While serializing form's value using prototype.js I lost accent
éàè ... because it use encodeURIComponent() function. I saw on
google, for ISO-8859-1 encoding I have to use escape(). It works fine
!

2. While receving content from AJAX Request with header <?xml
version='1.0' encoding='ISO-8859-1'?> I lost accent éàè ... only
under IE 6 it 's work fine under firefox.

What should I do ;-( ?

Well learn about encodings, if you rely on code that used
encodeURIComponent then you should be aware which encoding it uses and
adapt your server side code.
And I am pretty sure that MSXML in IE can handle XML with encoding
ISO-8859-1 so if you use responseXML and that response is indeed
properly encoded then the responseXML will work fine.
If you want to use responseText however then you or anyone else on the
AJAX trip needs to be aware that MSXML as documented assumes UTF-8 by
default when building responseText, it might also detect byte order
marks for other Unicode encodings, but if will never look at the content
(e.g. in your example the XML declaration) to find out about any
encoding when building responseText.
See
<http://msdn.microsoft.com/library/default.asp?url=/library/en-us/xmlsdk/html/974ceaf1-5c94-4e19-9498-a3b03df6f537.asp>

If you exchange your data as XML then you should be aware that XML
parsers are only required to support UTF-8 and UTF-16 so choosing a
different encoding in theory decreases your chances of having other apps
consume your data. In reality on the web XML parsers will probably not
have any problem with ISO-8859-1. But with MSXML you then need to ensure
that the XML parser parses the response and you use responseXML to
access the data.

If you want to use responseText then you would better send your data
UTF-8 encoded.

And if you think there are bugs in that prototype.js then contact the
author of that code.
--

Martin Honnen
http://JavaScript.FAQTs.com/

[Jean-Philippe Encausse]

Well, I'm not sure to understand everything

1. I have read lots of Article about encodeURIComponent() explaining
that it doesn't correctly encode ISO-8859-1 éèà ... After some test
it doesn't work on IE or Firefox but escape() did. May be I have to
specify a specific encoding ? My page header declare ISO-8859-1 so
JavaScript inside should be aware of it ?

2. prototype.js seems to use responseText instead of responseXML and
doesn't accept change BUT I change my JSP sending XML and it's working
great !
2.1 Header is now <?xml version='1.0' encoding='UTF-8'?>
2.2 Response is now <%=
response.setContentType("text/xml;charset=UTF-8"); %> (Here is the
trick !)
Thanks,
Jean-Philippe Encausse

[Martin Honnen]

Jean-Philippe Encausse wrote:

1. I have read lots of Article about encodeURIComponent() explaining
that it doesn't correctly encode ISO-8859-1 éèà ... After some test
it doesn't work on IE or Firefox but escape() did. May be I have to
specify a specific encoding ? My page header declare ISO-8859-1 so
JavaScript inside should be aware of it ?

No, the function encodeURIComponent is defined in the ECMAScript
specification and that specification defines strings as a sequence of
Unicode characters and defines how encodeURIComponent works on them.
Each character that needs to be encoded is replaced by an escape
sequence of its UTF-8 representation so
encodeURIComponent('é')
always yields
%C3%A9
as the UTF-8 representation of the character é is the two-byte sequence
C3A9 (in hex code) and each byte is escaped as %DD in the URI then.

So what the function does is clearly defined and does not in any way
depend on the function being used in a browser or in script in a HTML
document or on the encoding of the script or HTML document.
--

Martin Honnen
http://JavaScript.FAQTs.com/