Encoding troubles

[Jeroen De Brabander]

Hi,

I have problems with XML Encoding concerning special characters like é and
è.
I have set the processing instruction to UTF-16 and the file is saved as
UTF-16.
The XML is valid and well-formed, this isn't the problem.

I have 2 machines running exactly the same application (code is identical,
running under the same account) for parsing an XML. I gave both applications
the same XML-file.

Result:
The first application reads the é as é : correct
The second application reads the é as ?? : what is the matter here?

The unicode-regional settings are identical on both machines.

What did I miss?

Thx!

Jeroen db

[Martin Honnen]

Jeroen De Brabander wrote:

I have problems with XML Encoding concerning special characters like é and
è.
I have set the processing instruction to UTF-16 and the file is saved as
UTF-16.
The XML is valid and well-formed, this isn't the problem.

I have 2 machines running exactly the same application (code is identical,
running under the same account) for parsing an XML. I gave both applications
the same XML-file.

Result:
The first application reads the é as é : correct
The second application reads the é as ?? : what is the matter here?

If you don't get an XML parse error about invalid characters then let us
assume the file is indeed properly UTF-16 encoded. What kind of
application is that, where/how do you try to output the characters read
from the XML? Is that an ASP.NET application that sends stuff to the
browser, or is it a Windows console application sending stuff to the
console window, or is it a WinForms GUI application displaying the
output in some control?

--

Martin Honnen --- MVP XML
http://JavaScript.FAQTs.com/

[Jeroen De Brabander]

Hi Martin,

Thx for your quick reply.

There are indeed no parse errors.

It is a webservice on WS2003 that parses the XML and creates SharePoint
websites from it.
The content of the XML is used to define title and description for the
websites.

On my testing machine the special characters are translated correctly, in
the live environment I get question marks.

Kind regards,

Jeroen De Brabander
"Martin Honnen" <ma*******@yahoo.de> wrote in message
news:ed**************@TK2MSFTNGP11.phx.gbl...

Jeroen De Brabander wrote:

I have problems with XML Encoding concerning special characters like é
and è.
I have set the processing instruction to UTF-16 and the file is saved as
UTF-16.
The XML is valid and well-formed, this isn't the problem.

I have 2 machines running exactly the same application (code is
identical, running under the same account) for parsing an XML. I gave
both applications the same XML-file.

Result:
The first application reads the é as é : correct
The second application reads the é as ?? : what is the matter here?

If you don't get an XML parse error about invalid characters then let us
assume the file is indeed properly UTF-16 encoded. What kind of
application is that, where/how do you try to output the characters read
from the XML? Is that an ASP.NET application that sends stuff to the
browser, or is it a Windows console application sending stuff to the
console window, or is it a WinForms GUI application displaying the output
in some control?

--

Martin Honnen --- MVP XML
http://JavaScript.FAQTs.com/

[Martin Honnen]

Jeroen De Brabander wrote:

There are indeed no parse errors.

It is a webservice on WS2003 that parses the XML and creates SharePoint
websites from it.
The content of the XML is used to define title and description for the
websites.

On my testing machine the special characters are translated correctly, in
the live environment I get question marks.

I have no knowledge about Sharepoint but if you say web sites then the
ouput problem with the '??' is probably in a browser rendering those web
sites. Assuming the content is served over HTTP then you need to make
sure the output encoding SharePoint creates is transmitted to the
browser/client in a HTTP response header e.g. in ASP you would do
Response.ContentType = "text/html; charset=UTF-8"
for instance where of course the charset needs to be what the server
generates.
I don't think it is an XML problem really, if you can parse the XML
input fine then in your .NET code you should have strings with unicode
characters and the problem occurs later when your application creates
the output and probably then when the client/browser tries to render the
output.
--

Martin Honnen --- MVP XML
http://JavaScript.FAQTs.com/

[Jeroen De Brabander]

"Martin Honnen" <ma*******@yahoo.de> wrote in message
news:e2**************@tk2msftngp13.phx.gbl...

Jeroen De Brabander wrote:

There are indeed no parse errors.

It is a webservice on WS2003 that parses the XML and creates SharePoint
websites from it.
The content of the XML is used to define title and description for the
websites.

On my testing machine the special characters are translated correctly, in
the live environment I get question marks.

I have no knowledge about Sharepoint but if you say web sites then the
ouput problem with the '??' is probably in a browser rendering those web
sites. Assuming the content is served over HTTP then you need to make sure
the output encoding SharePoint creates is transmitted to the
browser/client in a HTTP response header e.g. in ASP you would do
Response.ContentType = "text/html; charset=UTF-8"
for instance where of course the charset needs to be what the server
generates.
I don't think it is an XML problem really, if you can parse the XML input
fine then in your .NET code you should have strings with unicode
characters and the problem occurs later when your application creates the
output and probably then when the client/browser tries to render the
output.
--

Martin Honnen --- MVP XML
http://JavaScript.FAQTs.com/

Hi Martin,

If what you say is the case, than I can't explain why the rendering in my
test-environment is fine.
The test sharepoint environment is identical to the one in production.

If I watch the XML (the file), it is fine. I select the nodes en use the
string as input for the description/title fields.
In test it is ok, in production I get "??".

I agree with you that the problem probably is not the XML.
The only difference is that they're on a different machine ... Can there be
an influence of regional machine settings?

Thx for your support !

Jeroen db

[Martin Honnen]

Jeroen De Brabander wrote:

If I watch the XML (the file), it is fine. I select the nodes en use the
string as input for the description/title fields.
In test it is ok, in production I get "??".

Is the production site online so that you could post a URL where one can
see the problem?
Do you visit those Sharepoint sites with a web browser like MS IE or
Firefox?

If you agree it is not an XML problem then you could try to look for
help in a Sharepoint group or forum. In my understanding you can ignore
the XML, you have a string with certain characters and have problem to
get them rendered properly in a client as far as I understand.
--

Martin Honnen --- MVP XML
http://JavaScript.FAQTs.com/

[Jeroen De Brabander]

Hi Martin,

Unfortunately I can't give you a URL as it is not a public site. As client
we only use IE.
Strange thing is that when we access both test and production sites from one
single client machine, the testsite shows the correct é-values, the
productionsite shows ??.

So the problem is not:
- the XML-format
- the client

I start thinking that there is a language/regional settings-difference
between the 2 webservices (test/production). The webservice takes the values
from the XML (the XML is fine) and accepts the values in UTF-8 instead of
UTF-16 in case of the production environment. Maybe there is a difference in
configuration of IIS. I'll try to figure it out.

Thanks for the support, Martin !

Kind regards,

Jeroen db

"Martin Honnen" <ma*******@yahoo.de> wrote in message
news:ub**************@TK2MSFTNGP09.phx.gbl...

Jeroen De Brabander wrote:

If I watch the XML (the file), it is fine. I select the nodes en use the
string as input for the description/title fields.
In test it is ok, in production I get "??".

Is the production site online so that you could post a URL where one can
see the problem?
Do you visit those Sharepoint sites with a web browser like MS IE or
Firefox?

If you agree it is not an XML problem then you could try to look for help
in a Sharepoint group or forum. In my understanding you can ignore the
XML, you have a string with certain characters and have problem to get
them rendered properly in a client as far as I understand.
--

Martin Honnen --- MVP XML
http://JavaScript.FAQTs.com/