Foreign Characters in XML

[Hugh Janus]

Hi all,

I posted a couple of weeks ago with what I thought was a problem with
the file system reading accented characters however, after debugging
line by line I have now found the true problem.

I am storing a list of files in an XML file as a sort of database.
Some of these filenames have accented characters (i.e. á é í ó ú
or ñ). However, upon writing the filename to the XML file, the
accented character is dropped. This causes a problem upon re-reading
the filenames because the program can not find the files because their
'saved' filename is now different. For example, the word "más" is
saved in the XML file as "ms".

Any ideas how I can work around this? I could strip out the accents
and replace them with their "normal" equivalent i.e. á becomes a. But
this is a sort of bodge fix as I will lose the link to the original
file. Also, I can see a scenario where a file may get overwritten
because the modified filename is the same as an existing file perhaps.

So, to put it blunty, I'm stuck! Help!
Thanks

[Herfried K. Wagner [MVP]]

"Hugh Janus" <my************ *@hotmail.com> schrieb:

I am storing a list of files in an XML file as a sort of database.
Some of these filenames have accented characters (i.e. á é í ó ú
or ñ). However, upon writing the filename to the XML file, the
accented character is dropped. This causes a problem upon re-reading
the filenames because the program can not find the files because their
'saved' filename is now different. For example, the word "más" is
saved in the XML file as "ms".

How are you currently writing data to the XML file? Which classes are you
using? It's likely that the problem is caused by a wrong encoding used to
persist the data.

--
M S Herfried K. Wagner
M V P <URL:http://dotnet.mvps.org/>
V B <URL:http://classicvb.org/petition/>

[Matthew.Gertz]

Hi, Hugh,
I'm not sure what you're code looks like, but you may need to "tokenize" (encode) these characters. They should be stored or read in as either UTF-8 or Unicode (XML processors are supposed to recognize these). This should "just work" if you are using the .NET framework's System.Xml code to generate or read an XML document -- you shouldn't have to do anything. Are you generating your own XML instead, & parsing it on your own? If so, you will need to do the encoding yourself, and will need to make sure the file you create has the appropriate header detailing the text type -- it sounds like you're translating them to bare ASCII when you're writing them out. You can use the System.Test.UTF 8Encoding class to translate strings between "normal" strings and UTF8, for example.

Let us know how it goes. (I'll be away for a few days, but will check back when I get back Friday.)

--Matt Gertz--*
VB Compiler Dev Lead

-----Original Message-----
From: Hugh Janus
Posted At: Monday, January 09, 2006 11:41 AM
Posted To: microsoft.publi c.dotnet.langua ges.vb
Conversation: Foreign Characters in XML
Subject: Foreign Characters in XML
Hi all,

I posted a couple of weeks ago with what I thought was a problem with
the file system reading accented characters however, after debugging
line by line I have now found the true problem.

I am storing a list of files in an XML file as a sort of database.
Some of these filenames have accented characters (i.e. =E1 =E9 =ED =F3 =FA
or =F1). However, upon writing the filename to the XML file, the
accented character is dropped. This causes a problem upon re-reading
the filenames because the program can not find the files because their
'saved' filename is now different. For example, the word "m=E1s" is
saved in the XML file as "ms".

Any ideas how I can work around this? I could strip out the accents
and replace them with their "normal" equivalent i.e. =E1 becomes a. But
this is a sort of bodge fix as I will lose the link to the original
file. Also, I can see a scenario where a file may get overwritten
because the modified filename is the same as an existing file perhaps.

So, to put it blunty, I'm stuck! Help!
Thanks

[Hugh Janus]

Herfried K. Wagner [MVP] wrote:

How are you currently writing data to the XML file? Which classes are you
using? It's likely that the problem is caused by a wrong encoding used to
persist the data.

I am using the class StreamReader and StreamWriter. Can I specify the
enconding with these in order to have the accented characters?

[Carlos J. Quintero [VB MVP]]

The StreamReader and StreamWriter classes have overloaded constructors to
specify the encoding:

public StreamReader ( System.String path , System.Text.Enc oding encoding )

public StreamWriter ( System.String path , System.Boolean append ,
System.Text.Enc oding encoding )
Member of System.IO.Strea mWriter

You will have to use the System.Text.Enc oding.Default encoding.
--

Best regards,

Carlos J. Quintero

MZ-Tools: Productivity add-ins for Visual Studio 2005, Visual Studio .NET,
VB6, VB5 and VBA
You can code, design and document much faster in VB.NET, C#, C++ or VJ#
Free resources for add-in developers:
http://www.mztools.com
"Hugh Janus" <my************ *@hotmail.com> escribió en el mensaje
news:11******** **************@ g14g2000cwa.goo glegroups.com.. .

Herfried K. Wagner [MVP] wrote:

How are you currently writing data to the XML file? Which classes are
you
using? It's likely that the problem is caused by a wrong encoding used
to
persist the data.

I am using the class StreamReader and StreamWriter. Can I specify the
enconding with these in order to have the accented characters?

[Hugh Janus]

Carlos J. Quintero [VB MVP] wrote:

The StreamReader and StreamWriter classes have overloaded constructors to
specify the encoding:

public StreamReader ( System.String path , System.Text.Enc oding encoding )

public StreamWriter ( System.String path , System.Boolean append ,
System.Text.Enc oding encoding )
Member of System.IO.Strea mWriter

You will have to use the System.Text.Enc oding.Default encoding.
--

Thanks Carlos for this. I'll give it a try and post back if it fails.

I assume that the System.Text.Enc oding.Default will cater for all
accents and the Ñ ?

[Carlos J. Quintero [VB MVP]]

> I assume that the System.Text.Enc oding.Default will cater for all accents

and the Ñ ?

The Default encoding uses your Windows code page instead of Unicode. As long
as your Windows code page (Control Panel, Regional Settings) matches the
code page of the computer used to generate the files, it will work.

--

Best regards,

Carlos J. Quintero

MZ-Tools: Productivity add-ins for Visual Studio 2005, Visual Studio .NET,
VB6, VB5 and VBA
You can code, design and document much faster in VB.NET, C#, C++ or VJ#
Free resources for add-in developers:
http://www.mztools.com

[Hugh Janus]

Carlos J. Quintero [VB MVP] wrote:

I assume that the System.Text.Enc oding.Default will cater for all accents
and the Ñ ?

The Default encoding uses your Windows code page instead of Unicode. As long
as your Windows code page (Control Panel, Regional Settings) matches the
code page of the computer used to generate the files, it will work.

--

Ah, well there is a problem. I am developing on a computer that is set
to Spanish regional settings but the app very possibly could be
installed on a computer with different regional settings. Is there a
universal one I could use?

[Herfried K. Wagner [MVP]]

"Hugh Janus" <my************ *@hotmail.com> schrieb:

Ah, well there is a problem. I am developing on a computer that is set
to Spanish regional settings but the app very possibly could be
installed on a computer with different regional settings. Is there a
universal one I could use?

I'd go with 'Encoding.UTF8' or 'Encoding.Unico de' (which is UTF-16).

--
M S Herfried K. Wagner
M V P <URL:http://dotnet.mvps.org/>
V B <URL:http://classicvb.org/petition/>

[Cor Ligthert [MVP]]

Hugh,

Have a look at those code tables for Unicode.

OS systems
http://www.microsoft.com/globaldev/r...ocversion.mspx

As you can see in the last page, are countries where is spoken Western
European languages, standard using code page 1252.

I hope this helps a little bit?

Cor