Prototype, Safari and Japanese problems?

[Doug Lerner]

I'm working on a client/server app that seems to work fine in OS Firefox and
Windows IE and Firefox.

However, in OS X Safari, although the UI/communications themselves work
fine, if the characters getting sent back and forth are in Japanese they
come back from the server "moji bake" (corrupted).

Anybody have any ideas why this might work differently in Safari than in
Firefox or IE?

Thanks!

doug

[Martin Honnen]

Doug Lerner wrote:

However, in OS X Safari, although the UI/communications themselves work
fine, if the characters getting sent back and forth are in Japanese they
come back from the server "moji bake" (corrupted).

Consider to use UTF-8 encoded Unicode characters and make sure the
server sends a HTTP header declaring e.g.
Content-Type: application/xml; charset=UTF-8
or if you send plain text then e.g.
Content-Type: text/plain; charset=UTF-8
I assume you use XMLHttpRequest to send and receive data, with the above
both responseXML or responseText should hopefully work correctly
(although I am not familiar with details of the Safari implementation) .

--

Martin Honnen
http://JavaScript.FAQTs.com/

[Doug Lerner]

On 1/18/06 10:05 PM, in article
43************* **********@news read2.arcor-online.net, "Martin Honnen"
<ma*******@yaho o.de> wrote:

Doug Lerner wrote:

However, in OS X Safari, although the UI/communications themselves work
fine, if the characters getting sent back and forth are in Japanese they
come back from the server "moji bake" (corrupted).

Consider to use UTF-8 encoded Unicode characters and make sure the
server sends a HTTP header declaring e.g.
Content-Type: application/xml; charset=UTF-8
or if you send plain text then e.g.
Content-Type: text/plain; charset=UTF-8
I assume you use XMLHttpRequest to send and receive data, with the above
both responseXML or responseText should hopefully work correctly
(although I am not familiar with details of the Safari implementation) .

Thanks for your response.

In my case, the return data is not being used to replace an entire HTML
page, it is being used to set the innerHTML for a <div> tag. So sending the
HTTP header wouldn't be appropriate in this case, would it?

The charset for the entire page itself is correct already.

Any thoughts about this?

Thanks!

doug

[VK]

Doug Lerner wrote:

On 1/18/06 10:05 PM, in article
43************* **********@news read2.arcor-online.net, "Martin Honnen"
<ma*******@yaho o.de> wrote:

Doug Lerner wrote:

However, in OS X Safari, although the UI/communications themselves work
fine, if the characters getting sent back and forth are in Japanese they
come back from the server "moji bake" (corrupted).

Consider to use UTF-8 encoded Unicode characters and make sure the
server sends a HTTP header declaring e.g.
Content-Type: application/xml; charset=UTF-8
or if you send plain text then e.g.
Content-Type: text/plain; charset=UTF-8
I assume you use XMLHttpRequest to send and receive data, with the above
both responseXML or responseText should hopefully work correctly
(although I am not familiar with details of the Safari implementation) .

Thanks for your response.

In my case, the return data is not being used to replace an entire HTML
page, it is being used to set the innerHTML for a <div> tag. So sending the
HTTP header wouldn't be appropriate in this case, would it?

The charset for the entire page itself is correct already.

Any thoughts about this?

AFAIK browser page is a "single encoding" unit therefore you cannot
display one paragraph in iso-8859-1 and another in say Shift_JIS. This
is why Unicode became originally needed. Did you try to have the entire
page served in UTF-8 including any further server interchange?

[Martin Honnen]

Doug Lerner wrote:

In my case, the return data is not being used to replace an entire HTML
page, it is being used to set the innerHTML for a <div> tag. So sending the
HTTP header wouldn't be appropriate in this case, would it?

The charset for the entire page itself is correct already.

If you use XMLHttpRequest and the responseText property then the client
receiving the response needs to know the charset/encoding of the
response to build responseText correctly. At least in theory, MSXML used
by IE does not care about HTTP headers when building responseText and
assumes UTF-8 so that is why I suggested to use UTF-8 and to make that
known with the HTTP header as that way MSXML and Mozilla and Opera and
hopefully Safari should then give you responseText with any characters
Unicode can encode, be that Japanese or whatever else.
--

Martin Honnen
http://JavaScript.FAQTs.com/

[Doug Lerner]

On 1/19/06 6:15 PM, in article
11************* *********@g47g2 00...legr oups.com, "VK"
<sc**********@y ahoo.com> wrote:

Thanks for your response.

In my case, the return data is not being used to replace an entire HTML
page, it is being used to set the innerHTML for a <div> tag. So sending the
HTTP header wouldn't be appropriate in this case, would it?

The charset for the entire page itself is correct already.

Any thoughts about this?

AFAIK browser page is a "single encoding" unit therefore you cannot
display one paragraph in iso-8859-1 and another in say Shift_JIS. This
is why Unicode became originally needed. Did you try to have the entire
page served in UTF-8 including any further server interchange?

Yes, I tried that. Something just seems "different" about the way it is
working in Safari. With Firefox or IE, it seems to work fine, even with the
Shift_JIS charset.

But with Safari the Japanese seems to get corrupted. I think it *is* sending
it to the server in UTF-8. That is, if I log the received data on the server
side and examine it it appears to be UTF-8 data. But when I send it back to
the browser, even if the browser's charset is UTF-8 it shows up looking
corrupted.

doug

[Doug Lerner]

On 1/19/06 9:10 PM, in article
43************* **********@news read4.arcor-online.net, "Martin Honnen"
<ma*******@yaho o.de> wrote:

Doug Lerner wrote:

In my case, the return data is not being used to replace an entire HTML
page, it is being used to set the innerHTML for a <div> tag. So sending the
HTTP header wouldn't be appropriate in this case, would it?

The charset for the entire page itself is correct already.

If you use XMLHttpRequest and the responseText property then the client
receiving the response needs to know the charset/encoding of the
response to build responseText correctly. At least in theory, MSXML used
by IE does not care about HTTP headers when building responseText and
assumes UTF-8 so that is why I suggested to use UTF-8 and to make that
known with the HTTP header as that way MSXML and Mozilla and Opera and
hopefully Safari should then give you responseText with any characters
Unicode can encode, be that Japanese or whatever else.

Thanks for your note. As mentioned in my note, the charset for the entire
page being served was correct. And I tried UTF-8 instead, but that didn't
seem to help.

It displays fine with all browsers except for Safari. I'm sort of stumped.

doug

[VK]

Doug Lerner wrote:

On 1/19/06 6:15 PM, in article
11************* *********@g47g2 00...legr oups.com, "VK"
<sc**********@y ahoo.com> wrote:

Thanks for your response.

In my case, the return data is not being used to replace an entire HTML
page, it is being used to set the innerHTML for a <div> tag. So sending the
HTTP header wouldn't be appropriate in this case, would it?

The charset for the entire page itself is correct already.

Any thoughts about this?

AFAIK browser page is a "single encoding" unit therefore you cannot
display one paragraph in iso-8859-1 and another in say Shift_JIS. This
is why Unicode became originally needed. Did you try to have the entire
page served in UTF-8 including any further server interchange?

Yes, I tried that. Something just seems "different" about the way it is
working in Safari. With Firefox or IE, it seems to work fine, even with the
Shift_JIS charset.

But with Safari the Japanese seems to get corrupted.

Corrupted in what way?
1) Latin ASCII chars ?
2) Unicode "missing gliph" chars ? (empty squares)
3) Japanese gliphs but not the needed one ?

[Martin Honnen]

Doug Lerner wrote:

Thanks for your note. As mentioned in my note, the charset for the entire
page being served was correct.

I am not sure I understand what an entire page is or why that matters or
what you are doing exactly, no wonder as you have not posted any code so
far.
If you serve one HTML document with script and that script makes further
requests to the server using XMLHttpRequest then the server needs to
make sure it sends proper HTTP response headers (with Content-Type and
charset parameter) in its response to any request the script makes if
the browser's XMLHttpRequest implementation should have a chance to
build responseText properly. It does not matter if you do not consider
or indeed such a response is not a complete HTML document but if a HTTP
response is processed then to build a responseText string you need to
know the charset to properly decode the bytes in the response body into
the responseText string.

--

Martin Honnen
http://JavaScript.FAQTs.com/

[Thomas 'PointedEars' Lahn]

VK wrote:

AFAIK browser page is a "single encoding" unit
True.
therefore you cannot display one paragraph in iso-8859-1 and another in
say Shift_JIS.
False. Character references and character entity references are there ever
since to workaround this issue. For example, it is perfectly reasonable,
possible and Valid to use the hexadecimal byte sequence 26 23 38 32 31 31
3B 0A, this represents the decimal character reference `–', for
displaying the Unicode character named "EN DASH" (U+2013) in a HTML
resource encoded with ISO-8859-1 and served as such. In fact, the same
byte sequence could _not_ result in that character reference if the
resource was encoded with a UTF and served as such.
This is why Unicode became originally needed.
Not quite. The reason for creating the Unicode standard and subsequently
the Unicode character set was that _one_ standard and _one_ character set
for all characters was needed, so that one encoding would suffice for all
characters and all textual resources, not only SGML-conforming ones such
as HTML documents, and that the latter then could contain the characters
as they are, without any character reference or character entity reference
which would save bandwidth and disk space.
Did you try to have the entire page served in UTF-8 including any further
server interchange?

The encoding of an SGML-conforming markup resource, that is, how the
character data of a resource is encoded, does not have any impact on
the characters that can be displayed with it, that is, what character
sets can be used to display that data.

Despite its name, the "charset" label of the HTTP Content-Type header
specifies the _encoding_ of the served resource, not necessarily the
character set(s) that is/are to be used to display it; the confusing
name of the label is because of the history of MIME to which HTTP had
to adhere.
HTH

PointedEars