Internationalization and msxml

[Developer]

Hello,

I have an xml retrieved by a client running Japanese windows from a server
running Japanese windows. I can view this document in IE.6. (The Opera
browser, though, reports the error.)

However, when I try to parse the xml with a sax reader I'm implementing, I
get the message "A name contained an invalid character" in the fatalError()
method of my ISAXErrorHandler.

I built another sample app using DOM. When I try to load the file, the
Load() fails with
Exception: System.Xml.XmlException: The '?' character, hexadecimal value
0xFF7A,
cannot be included in a name. Line 16, position 13.
at System.Xml.XmlScanner.ScanName()
at System.Xml.XmlScanner.ScanMarkup()
at System.Xml.XmlScanner.ScanToken(Int32 expected)
at System.Xml.XmlTextReader.SetElementValues()
at System.Xml.XmlTextReader.ParseElement()
at System.Xml.XmlTextReader.Read()
at System.Xml.XmlLoader.LoadChildren(XmlNode parent)
at System.Xml.XmlLoader.LoadElementNode()
at System.Xml.XmlLoader.LoadCurrentNode()
at System.Xml.XmlLoader.LoadChildren(XmlNode parent)
at System.Xml.XmlLoader.LoadElementNode()
at System.Xml.XmlLoader.LoadCurrentNode()
at System.Xml.XmlLoader.LoadChildren(XmlNode parent)
at System.Xml.XmlLoader.LoadElementNode()
at System.Xml.XmlLoader.LoadCurrentNode()
at System.Xml.XmlLoader.LoadDocSequence(XmlDocument parentDoc)
at System.Xml.XmlLoader.Load(XmlDocument doc, XmlReader reader, Boolean
prese
rveWhitespace)
at System.Xml.XmlDocument.Load(XmlReader reader)

What am I overlooking in trying to read this document?

Thanks for any suggestions.

CB

[Kevin Yu [MSFT]]

Hi Developer,

First of all, I would like to confirm my understanding of your issue. From
your description, I understand that you are having trouble parsing an Xml
document both with SAX and XmlDocument. If there is any misunderstanding,
please feel free to let me know.

Based on my research, this might be an encoding issue. Could you please let
me know, how the Xml was gained and how it was parsed? Because we need to
know the format that Xml is saved in. Have you tried to open it in IE? If
yes, can you see the encoding mode of that file? If it can be opened by IE
successfully, could you show me the code that you're using to parse the
file in .NET?

If anything is unclear, please feel free to reply to the post.

Kevin Yu
=======
"This posting is provided "AS IS" with no warranties, and confers no
rights."

[Developer]

Hello, Kevin,

The client is running on Japanese Windows, connecting to a Japanese server.
Via HTTP, we prefer to use POST, but some servers only work with GET. In
either case, we write the returned stream to a file.
The header on the returned xml says it is UTF-8.
We create a SaxReader, give it a content handler and an error handler, and
call ISAXXMLReader->parseURL(filename). I can watch the startElement() and
endElement() methods as the file is read until the
ISAXErrorHandler::fatalError() method is called with the error message "A
name contained an invalid character".

Internet Explorer displays the file correctly as a mix of English and
single- and multi-byte Japanese characters. However, Mozilla FireFox and
Opera 7.x browsers both indicate a problem with the xml -- "not well
formed".

The XmlDocument in my test app gives the error:
Exception: System.Xml.XmlException: The '?' character, hexadecimal value
0xFF7A,
cannot be included in a name. Line 15, position 15.

As for the code that parses the file, it is a simple implementation of the
ISAXXMLReader interface. In the startElement(), characters(), and
endElement() methods the code simply writes the localName or characters to
the console.

Let me know what other information you need. I would be glad to send you
the sample project(s), as well as the troublesome XML.

Thanks for your help.

"Kevin Yu [MSFT]" <v-****@online.microsoft.com> wrote in message
news:Nq**************@cpmsftngxa10.phx.gbl...

Hi Developer,

First of all, I would like to confirm my understanding of your issue. From
your description, I understand that you are having trouble parsing an Xml
document both with SAX and XmlDocument. If there is any misunderstanding,
please feel free to let me know.

Based on my research, this might be an encoding issue. Could you please let me know, how the Xml was gained and how it was parsed? Because we need to
know the format that Xml is saved in. Have you tried to open it in IE? If
yes, can you see the encoding mode of that file? If it can be opened by IE
successfully, could you show me the code that you're using to parse the
file in .NET?

If anything is unclear, please feel free to reply to the post.

Kevin Yu
=======
"This posting is provided "AS IS" with no warranties, and confers no
rights."

[Developer]

I was just told by one of our developers that the Java XML parsers don't
read the file, either.

"Developer" <wa******@mapinfo.nope.com> wrote in message
news:%2****************@TK2MSFTNGP09.phx.gbl...

Hello, Kevin,

The client is running on Japanese Windows, connecting to a Japanese server. Via HTTP, we prefer to use POST, but some servers only work with GET. In
either case, we write the returned stream to a file.
The header on the returned xml says it is UTF-8.
We create a SaxReader, give it a content handler and an error handler, and
call ISAXXMLReader->parseURL(filename). I can watch the startElement() and endElement() methods as the file is read until the
ISAXErrorHandler::fatalError() method is called with the error message "A
name contained an invalid character".

Internet Explorer displays the file correctly as a mix of English and
single- and multi-byte Japanese characters. However, Mozilla FireFox and
Opera 7.x browsers both indicate a problem with the xml -- "not well
formed".

The XmlDocument in my test app gives the error:
Exception: System.Xml.XmlException: The '?' character, hexadecimal value
0xFF7A,
cannot be included in a name. Line 15, position 15.

As for the code that parses the file, it is a simple implementation of the
ISAXXMLReader interface. In the startElement(), characters(), and
endElement() methods the code simply writes the localName or characters to
the console.

Let me know what other information you need. I would be glad to send you
the sample project(s), as well as the troublesome XML.

Thanks for your help.

"Kevin Yu [MSFT]" <v-****@online.microsoft.com> wrote in message
news:Nq**************@cpmsftngxa10.phx.gbl...

Hi Developer,

First of all, I would like to confirm my understanding of your issue. From your description, I understand that you are having trouble parsing an Xml document both with SAX and XmlDocument. If there is any misunderstanding, please feel free to let me know.

Based on my research, this might be an encoding issue. Could you please

let

me know, how the Xml was gained and how it was parsed? Because we need to know the format that Xml is saved in. Have you tried to open it in IE? If yes, can you see the encoding mode of that file? If it can be opened by IE successfully, could you show me the code that you're using to parse the
file in .NET?

If anything is unclear, please feel free to reply to the post.

Kevin Yu
=======
"This posting is provided "AS IS" with no warranties, and confers no
rights."

[Kevin Yu [MSFT]]

Hi Developer,

Thanks for your reply. It would be helpful if you can send me the xml file
and the code you're trying to parse it using an XmlDocument. Remove
'online' from the no spam email alias is my email address.

Kevin Yu
=======
"This posting is provided "AS IS" with no warranties, and confers no
rights."

[Kevin Yu [MSFT]]

Hi Developer,

Thanks for your Xml document. Based on my research, the Xml document
contains invalid characters. Although IE can still parse it, because IE
allows errors in the document. However, the SAX reader and XmlDocument
requires a well formed xml document. If you open the xml file in IE, you
will notice an exclamation mark with a yellow triangle in the lower-left
corner on the status bar. Double click it will show the error message.

Kevin Yu
=======
"This posting is provided "AS IS" with no warranties, and confers no
rights."

[Developer]

I am using IE 6, and don't see a yellow triangle in the status bar. The
status bar panels contain "Done", [blank], [blank], [blank], "My Computer".

I understand that the document contains a mix of English and Japanese
characters. The Japanese characters are a mix of different size characters,
as well.
Can you tell me what characters are invalid, and why? According to our
internationalization developers, all the characters in the file are valid
Japanese, in one character set or another.

Thanks for your help.

"Kevin Yu [MSFT]" <v-****@online.microsoft.com> wrote in message
news:yy****************@cpmsftngxa06.phx.gbl...

Hi Developer,

Thanks for your Xml document. Based on my research, the Xml document
contains invalid characters. Although IE can still parse it, because IE
allows errors in the document. However, the SAX reader and XmlDocument
requires a well formed xml document. If you open the xml file in IE, you
will notice an exclamation mark with a yellow triangle in the lower-left
corner on the status bar. Double click it will show the error message.

Kevin Yu
=======
"This posting is provided "AS IS" with no warranties, and confers no
rights."

[Developer]

Any information on what characters are invalid, and why?
If msxml can't handle this document (which our engineers assure me is
well-formed and valid xml), can you suggest another parser? How does IE
display this document? Does it ignore the "invalid character" errors?
Thanks for your help.

"Kevin Yu [MSFT]" <v-****@online.microsoft.com> wrote in message
news:yy****************@cpmsftngxa06.phx.gbl...

Hi Developer,

Thanks for your Xml document. Based on my research, the Xml document
contains invalid characters. Although IE can still parse it, because IE
allows errors in the document. However, the SAX reader and XmlDocument
requires a well formed xml document. If you open the xml file in IE, you
will notice an exclamation mark with a yellow triangle in the lower-left
corner on the status bar. Double click it will show the error message.

Kevin Yu
=======
"This posting is provided "AS IS" with no warranties, and confers no
rights."

[Kevin Yu [MSFT]]

Hi Developer,

My Internet Explorer opens the .xml file and it shows that "Error on Page"
on the left side of the status bar. IE ignores the invalid character in the
document. Based on my research, there are more than one invalid characters
in the document, and it only returns the first invalid one which is located
at line 16, Column 18.

Kevin Yu
=======
"This posting is provided "AS IS" with no warranties, and confers no
rights."

[Developer]

The problematic document contained unicode characters in the "compatibility
area" (i.e. with character code greater than #xF900 and less than #xFFFE),
which are not allowed in XML names.