ToUpper()

[Ornette]

Hello,

I'm trying to convert strings to upper without the accents. For the moment,
ToUpper converts é to E with an accent...
I tried to set up english culture (en) but it's the same...

Any ideas ?

Ornette.

[Ornette]

Ok, finally I did it like this :

private string ReplaceAccents(string chaine)
{

string strAccents=
"ÀÁÂÃÄÅàáâãäåÒÓÔÕÖØòóôõöøÈÉÊËèéêëÌÍÎÏìíîïÙÚÛÜùúûüÿ ÑñÇç";
string strNoAccents =
"AAAAAAaaaaaaOOOOOOooooooEEEEeeeeIIIIiiiiUUUUuuuuy NnCc";

char[] tAccent = strAccents.ToCharArray();
char[] tNoAccent = strNoAccents .ToCharArray();

for(int i=0; i<strAccents.Length; i++)
{
chaine = chaine.Replace(tAccent [i].ToString(), tNoAccent
[i].ToString());
}
return chaine;
}

J'ai pas trouvé mieux, même si ça boucle un peu pour rien...

Ornette.
"Ornette" <[e-mail] écrit dans le message de
news:DF**********************************@microsof t.com...

Hello,

I'm trying to convert strings to upper without the accents. For the
moment, ToUpper converts é to E with an accent...
I tried to set up english culture (en) but it's the same...

Any ideas ?

Ornette.

[Ornette]

Hello,

This is better :

byte[] bString =
System.Text.Encoding.GetEncoding(1251).GetBytes(St ringAvecAccents);
string stringSansAccent = System.Text.Encoding.ASCII.GetString(bString );

Reference CodePage :
http://www.microsoft.com/globaldev/r...sbcs/1251.mspx

Ornette.

"Ornette" <[e-mail] écrit dans le message de
news:56**********************************@microsof t.com...

Ok, finally I did it like this :

private string ReplaceAccents(string chaine)
{

string strAccents=
"ÀÁÂÃÄÅàáâãäåÒÓÔÕÖØòóôõöøÈÉÊËèéêëÌÍÎÏìíîïÙÚÛÜùúûüÿ ÑñÇç";
string strNoAccents =
"AAAAAAaaaaaaOOOOOOooooooEEEEeeeeIIIIiiiiUUUUuuuuy NnCc";

char[] tAccent = strAccents.ToCharArray();
char[] tNoAccent = strNoAccents .ToCharArray();

for(int i=0; i<strAccents.Length; i++)
{
chaine = chaine.Replace(tAccent [i].ToString(), tNoAccent
[i].ToString());
}
return chaine;
}

J'ai pas trouvé mieux, même si ça boucle un peu pour rien...

Ornette.
"Ornette" <[e-mail] écrit dans le message de
news:DF**********************************@microsof t.com...

>Hello,

I'm trying to convert strings to upper without the accents. For the
moment, ToUpper converts é to E with an accent...
I tried to set up english culture (en) but it's the same...

Any ideas ?

Ornette.

[Jon Skeet [C# MVP]]

Ornette <[e-mail]:

This is better :

byte[] bString =
System.Text.Encoding.GetEncoding(1251).GetBytes(St ringAvecAccents);
string stringSansAccent = System.Text.Encoding.ASCII.GetString(bString );

Well, that's assuming that the encoding will find the closest match
letter. It may work now, but there's no guarantee that it will in the
future.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Ornette]

Hello,

So how would you do ?

Ornette.

"Jon Skeet [C# MVP]" <sk***@pobox.coma écrit dans le message de
news:MP************************@msnews.microsoft.c om...

Ornette <[e-mail]:

>This is better :

byte[] bString =
System.Text.Encoding.GetEncoding(1251).GetBytes(S tringAvecAccents);
string stringSansAccent = System.Text.Encoding.ASCII.GetString(bString );

Well, that's assuming that the encoding will find the closest match
letter. It may work now, but there's no guarantee that it will in the
future.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Jon Skeet [C# MVP]]

Ornette <[e-mail]:

So how would you do ?

The mapping table idea you had before looked best to me, although I
wouldn't quite implement it the same way. I'd have a look up table for
every possible character, where it defaults to the Unicode character,
but for all the accented characters you care about, you specify the
non-accented version.

You'd then call ToCharArray() on the string in question, go through
each character replacing the original with the mapped character, and
then create a new string with the char array.

It does require you to manually map all the accented characters you
care about though.

My guess is that there are libraries around to do this somewhere, but I
don't know of any myself.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Ornette]

Ok, thank you for your point of view.
I really agree.

For the the librairies, I also didn't find any one.

Have a nice day and thanks again.

Ornette.

"Jon Skeet [C# MVP]" <sk***@pobox.coma écrit dans le message de
news:MP************************@msnews.microsoft.c om...

Ornette <[e-mail]:

>So how would you do ?

The mapping table idea you had before looked best to me, although I
wouldn't quite implement it the same way. I'd have a look up table for
every possible character, where it defaults to the Unicode character,
but for all the accented characters you care about, you specify the
non-accented version.

You'd then call ToCharArray() on the string in question, go through
each character replacing the original with the mapped character, and
then create a new string with the char array.

It does require you to manually map all the accented characters you
care about though.

My guess is that there are libraries around to do this somewhere, but I
don't know of any myself.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Chris Mullins [MVP]]

The closest thing that comes to mind is an RFC called stringprep. There are
a wide variety of stringprep profiles, and while they don't quite do what
you're looking for, they're close. Included in stringprep is a set of
mapping tables for Uppder->Lower case conversions. These are (in that
context) called case-foldings, are are found in table B.2. Unfortunatly,
they're Upper->Lower, not the other way around.

Stringprep:
http://www.faqs.org/rfcs/rfc3454.html

There are a number of profiles:
[Profile for Internaional Domain Names]
http://www.rfc-editor.org/rfc/rfc3491.txt

[Profile for iSCSI names]
http://tools.ietf.org/html/draft-iet...string-prep-01

[Profile for SASL UserNames & Passwords]
http://www.ietf.org/rfc/rfc4013.txt

[Profile for XMPP Resources]
http://www.xmpp.org/internet-drafts/...ceprep-02.html

There's a C# implementation of this RFC that's part of the libidn library.
There's also a C++ & Java version.
http://www.gnu.org/software/libidn/

We've actually got a full implemention of stringprep as well - it's much
more .Net 2.0 ish than the libidn one, which is just a native C++ app that
was then ported to Java & .Net. It's found in our open-source SoapBox
Framework.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"Jon Skeet [C# MVP]" <sk***@pobox.comwrote in message
news:MP************************@msnews.microsoft.c om...

Ornette <[e-mail]:

>So how would you do ?

The mapping table idea you had before looked best to me, although I
wouldn't quite implement it the same way. I'd have a look up table for
every possible character, where it defaults to the Unicode character,
but for all the accented characters you care about, you specify the
non-accented version.

You'd then call ToCharArray() on the string in question, go through
each character replacing the original with the mapped character, and
then create a new string with the char array.

It does require you to manually map all the accented characters you
care about though.

My guess is that there are libraries around to do this somewhere, but I
don't know of any myself.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[JR]

In a more general way:

There is a Unicode database at

http://www.unicode.org/Public/UNIDATA/

You could do what you want in two steps: decompose the string to base
characters followed by accent (NFKD normalization), then remove the accents.

JR
"Chris Mullins [MVP]" <cm******@yahoo.comëúá
áäåãòä:us**************@TK2MSFTNGP05.phx.gbl...

The closest thing that comes to mind is an RFC called stringprep. There
are a wide variety of stringprep profiles, and while they don't quite do
what you're looking for, they're close. Included in stringprep is a set of
mapping tables for Uppder->Lower case conversions. These are (in that
context) called case-foldings, are are found in table B.2. Unfortunatly,
they're Upper->Lower, not the other way around.

Stringprep:
http://www.faqs.org/rfcs/rfc3454.html

There are a number of profiles:
[Profile for Internaional Domain Names]
http://www.rfc-editor.org/rfc/rfc3491.txt

[Profile for iSCSI names]
http://tools.ietf.org/html/draft-iet...string-prep-01

[Profile for SASL UserNames & Passwords]
http://www.ietf.org/rfc/rfc4013.txt

[Profile for XMPP Resources]
http://www.xmpp.org/internet-drafts/...ceprep-02.html

There's a C# implementation of this RFC that's part of the libidn library.
There's also a C++ & Java version.
http://www.gnu.org/software/libidn/

We've actually got a full implemention of stringprep as well - it's much
more .Net 2.0 ish than the libidn one, which is just a native C++ app that
was then ported to Java & .Net. It's found in our open-source SoapBox
Framework.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"Jon Skeet [C# MVP]" <sk***@pobox.comwrote in message
news:MP************************@msnews.microsoft.c om...

>Ornette <[e-mail]:

>>So how would you do ?

The mapping table idea you had before looked best to me, although I
wouldn't quite implement it the same way. I'd have a look up table for
every possible character, where it defaults to the Unicode character,
but for all the accented characters you care about, you specify the
non-accented version.

You'd then call ToCharArray() on the string in question, go through
each character replacing the original with the mapped character, and
then create a new string with the char array.

It does require you to manually map all the accented characters you
care about though.

My guess is that there are libraries around to do this somewhere, but I
don't know of any myself.

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Chris Mullins]

I hadn't thought of that, but it's certainly an option.

Doing the normalization in .Net 2.0 is easy enough:

string s = "test";
string normalized = s.Normalize(NormalizationForm.FormKC);

Then you can iterate over the normalized string looking for (and
removing) the accents.

--
Chris Mullins
On Mar 1, 11:15 pm, "JR" <[e-mail]:

In a more general way:

There is a Unicode database at

http://www.unicode.org/Public/UNIDATA/

You could do what you want in two steps: decompose the string to base
characters followed by accent (NFKD normalization), then remove the accents.

JR

"Chris Mullins [MVP]" <[e-mail]ëúá
áäåãòä:[e-mail]...

The closest thing that comes to mind is an RFC called stringprep. There
are a wide variety of stringprep profiles, and while they don't quite do
what you're looking for, they're close. Included in stringprep is a setof
mapping tables for Uppder->Lower case conversions. These are (in that
context) called case-foldings, are are found in table B.2. Unfortunatly,
they're Upper->Lower, not the other way around.

Stringprep:
http://www.faqs.org/rfcs/rfc3454.html

There are a number of profiles:
[Profile for Internaional Domain Names]
http://www.rfc-editor.org/rfc/rfc3491.txt

[Profile for iSCSI names]
http://tools.ietf.org/html/draft-iet...string-prep-01

[Profile for SASL UserNames & Passwords]
http://www.ietf.org/rfc/rfc4013.txt

[Profile for XMPP Resources]
http://www.xmpp.org/internet-drafts/...pp-resourcepre...

There's a C# implementation of this RFC that's part of the libidn library.
There's also a C++ & Java version.
http://www.gnu.org/software/libidn/

We've actually got a full implemention of stringprep as well - it's much
more .Net 2.0 ish than the libidn one, which is just a native C++ app that
was then ported to Java & .Net. It's found in our open-source SoapBox
Framework.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"Jon Skeet [C# MVP]" <[e-mail] in message
news:MP************************@msnews.microsoft.c om...

Ornette <[e-mail]:
So how would you do ?

The mapping table idea you had before looked best to me, although I
wouldn't quite implement it the same way. I'd have a look up table for
every possible character, where it defaults to the Unicode character,
but for all the accented characters you care about, you specify the
non-accented version.

You'd then call ToCharArray() on the string in question, go through
each character replacing the original with the mapped character, and
then create a new string with the char array.

It does require you to manually map all the accented characters you
care about though.

My guess is that there are libraries around to do this somewhere, but I
don't know of any myself.

--
Jon Skeet - <[e-mail]>
http://www.pobox.com/~skeet Blog:http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Jon Skeet [C# MVP]]

Chris Mullins <cm******@yahoo.comwrote:

I hadn't thought of that, but it's certainly an option.

Doing the normalization in .Net 2.0 is easy enough:

string s = "test";
string normalized = s.Normalize(NormalizationForm.FormKC);

Then you can iterate over the normalized string looking for (and
removing) the accents.

Cool - I hadn't seen that 2.0 had normalization stuff. Fantastic!

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Chris Mullins [MVP]]

Yea, .Net 2.0 also added the IDN stuff (which includes the PunyCode
algorithm), which came as a complete surprise to me:

System.Globalization.IdnMapping mapping = new
System.Globalization.IdnMapping();
normalized = mapping.GetAscii(normalized);

(just don't run an empty string through the IdnMapping class, or you'll get
an exception)

Now, if only they exposed the BiDirectional stuff...
--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins
"Jon Skeet [C# MVP]" <sk***@pobox.comwrote in message
news:MP************************@msnews.microsoft.c om...

Chris Mullins <cm******@yahoo.comwrote:

>I hadn't thought of that, but it's certainly an option.

Doing the normalization in .Net 2.0 is easy enough:

string s = "test";
string normalized = s.Normalize(NormalizationForm.FormKC);

Then you can iterate over the normalized string looking for (and
removing) the accents.

Cool - I hadn't seen that 2.0 had normalization stuff. Fantastic!

--
Jon Skeet - <sk***@pobox.com>
http://www.pobox.com/~skeet Blog: http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[JR]

You meant NormalizationForm.FormKD.

Looking into it, I see a simpler method: After normalization, use
ASCIIEncoding with DecoderReplacementFallback replacing invalid ASCII
characters (which will be the accents) with the empty string.

JR

"Chris Mullins" <cm******@yahoo.com???
??????:11*********************@31g2000cwt.googlegr oups.com...
I hadn't thought of that, but it's certainly an option.

Doing the normalization in .Net 2.0 is easy enough:

string s = "test";
string normalized = s.Normalize(NormalizationForm.FormKC);

Then you can iterate over the normalized string looking for (and
removing) the accents.

--
Chris Mullins
On Mar 1, 11:15 pm, "JR" <[e-mail]:

In a more general way:

There is a Unicode database at

http://www.unicode.org/Public/UNIDATA/

You could do what you want in two steps: decompose the string to base
characters followed by accent (NFKD normalization), then remove the
accents.

JR

"Chris Mullins [MVP]" <[e-mail]ëúá
áäåãòä:[e-mail]...

The closest thing that comes to mind is an RFC called stringprep. There
are a wide variety of stringprep profiles, and while they don't quite do
what you're looking for, they're close. Included in stringprep is a set
of
mapping tables for Uppder->Lower case conversions. These are (in that
context) called case-foldings, are are found in table B.2. Unfortunatly,
they're Upper->Lower, not the other way around.

Stringprep:
http://www.faqs.org/rfcs/rfc3454.html

There are a number of profiles:
[Profile for Internaional Domain Names]
http://www.rfc-editor.org/rfc/rfc3491.txt

[Profile for iSCSI names]
http://tools.ietf.org/html/draft-iet...string-prep-01

[Profile for SASL UserNames & Passwords]
http://www.ietf.org/rfc/rfc4013.txt

[Profile for XMPP Resources]
http://www.xmpp.org/internet-drafts/...pp-resourcepre...

There's a C# implementation of this RFC that's part of the libidn
library.
There's also a C++ & Java version.
http://www.gnu.org/software/libidn/

We've actually got a full implemention of stringprep as well - it's much
more .Net 2.0 ish than the libidn one, which is just a native C++ app
that
was then ported to Java & .Net. It's found in our open-source SoapBox
Framework.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"Jon Skeet [C# MVP]" <[e-mail] in message
news:MP************************@msnews.microsoft.c om...

Ornette <[e-mail]:
So how would you do ?

The mapping table idea you had before looked best to me, although I
wouldn't quite implement it the same way. I'd have a look up table for
every possible character, where it defaults to the Unicode character,
but for all the accented characters you care about, you specify the
non-accented version.

You'd then call ToCharArray() on the string in question, go through
each character replacing the original with the mapped character, and
then create a new string with the char array.

It does require you to manually map all the accented characters you
care about though.

My guess is that there are libraries around to do this somewhere, but I
don't know of any myself.

--
Jon Skeet - <[e-mail]>
http://www.pobox.com/~skeet Blog:http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Chris Mullins [MVP]]

Opps. Definatly KD!

We want to do the decomposition & make our changes. For KC would decompose &
then perform a canonical recompose - which would defeat the purpose!

I've never used (or even seen) the DecoderReplacementFallback - that's
another good idea. By now the original poster has probably given up and will
never try any of these solutions, but I think they would very cleanly do the
trick.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"JR" <No****@qsm.co.ilwrote in message
news:%2******************@TK2MSFTNGP06.phx.gbl...

You meant NormalizationForm.FormKD.

Looking into it, I see a simpler method: After normalization, use
ASCIIEncoding with DecoderReplacementFallback replacing invalid ASCII
characters (which will be the accents) with the empty string.

JR

"Chris Mullins" <cm******@yahoo.com???
??????:11*********************@31g2000cwt.googlegr oups.com...
I hadn't thought of that, but it's certainly an option.

Doing the normalization in .Net 2.0 is easy enough:

string s = "test";
string normalized = s.Normalize(NormalizationForm.FormKC);

Then you can iterate over the normalized string looking for (and
removing) the accents.

--
Chris Mullins
On Mar 1, 11:15 pm, "JR" <[e-mail]:

>In a more general way:

There is a Unicode database at

http://www.unicode.org/Public/UNIDATA/

You could do what you want in two steps: decompose the string to base
characters followed by accent (NFKD normalization), then remove the
accents.

JR

"Chris Mullins [MVP]" <[e-mail]ëúá
áäåãòä:[e-mail]...

The closest thing that comes to mind is an RFC called stringprep. There
are a wide variety of stringprep profiles, and while they don't quite
do
what you're looking for, they're close. Included in stringprep is a set
of
mapping tables for Uppder->Lower case conversions. These are (in that
context) called case-foldings, are are found in table B.2.
Unfortunatly,
they're Upper->Lower, not the other way around.

Stringprep:
http://www.faqs.org/rfcs/rfc3454.html

There are a number of profiles:
[Profile for Internaional Domain Names]
http://www.rfc-editor.org/rfc/rfc3491.txt

[Profile for iSCSI names]
http://tools.ietf.org/html/draft-iet...string-prep-01

[Profile for SASL UserNames & Passwords]
http://www.ietf.org/rfc/rfc4013.txt

[Profile for XMPP Resources]
http://www.xmpp.org/internet-drafts/...pp-resourcepre...

There's a C# implementation of this RFC that's part of the libidn
library.
There's also a C++ & Java version.
http://www.gnu.org/software/libidn/

We've actually got a full implemention of stringprep as well - it's
much
more .Net 2.0 ish than the libidn one, which is just a native C++ app
that
was then ported to Java & .Net. It's found in our open-source SoapBox
Framework.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"Jon Skeet [C# MVP]" <[e-mail] in message
news:MP************************@msnews.microsoft. com...
Ornette <[e-mail]:
So how would you do ?

>The mapping table idea you had before looked best to me, although I
wouldn't quite implement it the same way. I'd have a look up table for
every possible character, where it defaults to the Unicode character,
but for all the accented characters you care about, you specify the
non-accented version.

>You'd then call ToCharArray() on the string in question, go through
each character replacing the original with the mapped character, and
then create a new string with the char array.

>It does require you to manually map all the accented characters you
care about though.

>My guess is that there are libraries around to do this somewhere, but
I
don't know of any myself.

>--
Jon Skeet - <[e-mail]>
http://www.pobox.com/~skeet Blog:http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Chris Mullins [MVP]]

For anyone still paying attention, the complete (and working + tested) code
is:

string s = "áäåãòä:usdBDlGXHHA";
string normalized = s.Normalize(NormalizationForm.FormKD);

Encoding ascii = Encoding.GetEncoding(
"us-ascii",
new EncoderReplacementFallback(string.Empty),
new DecoderReplacementFallback(string.Empty));

byte[] encodedBytes = new byte[ascii.GetByteCount(normalized)];
int numberOfEncodedBytes = ascii.GetBytes(normalized, 0, normalized.Length,
encodedBytes, 0);

string newString = ascii.GetString(encodedBytes).ToUpper();
MessageBox.Show(newString);

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"JR" <No****@qsm.co.ilwrote in message
news:%2******************@TK2MSFTNGP06.phx.gbl...

You meant NormalizationForm.FormKD.

Looking into it, I see a simpler method: After normalization, use
ASCIIEncoding with DecoderReplacementFallback replacing invalid ASCII
characters (which will be the accents) with the empty string.

JR

"Chris Mullins" <cm******@yahoo.com???
??????:11*********************@31g2000cwt.googlegr oups.com...
I hadn't thought of that, but it's certainly an option.

Doing the normalization in .Net 2.0 is easy enough:

string s = "test";
string normalized = s.Normalize(NormalizationForm.FormKC);

Then you can iterate over the normalized string looking for (and
removing) the accents.

--
Chris Mullins
On Mar 1, 11:15 pm, "JR" <[e-mail]:

>In a more general way:

There is a Unicode database at

http://www.unicode.org/Public/UNIDATA/

You could do what you want in two steps: decompose the string to base
characters followed by accent (NFKD normalization), then remove the
accents.

JR

"Chris Mullins [MVP]" <[e-mail]ëúá
áäåãòä:[e-mail]...

The closest thing that comes to mind is an RFC called stringprep. There
are a wide variety of stringprep profiles, and while they don't quite
do
what you're looking for, they're close. Included in stringprep is a set
of
mapping tables for Uppder->Lower case conversions. These are (in that
context) called case-foldings, are are found in table B.2.
Unfortunatly,
they're Upper->Lower, not the other way around.

Stringprep:
http://www.faqs.org/rfcs/rfc3454.html

There are a number of profiles:
[Profile for Internaional Domain Names]
http://www.rfc-editor.org/rfc/rfc3491.txt

[Profile for iSCSI names]
http://tools.ietf.org/html/draft-iet...string-prep-01

[Profile for SASL UserNames & Passwords]
http://www.ietf.org/rfc/rfc4013.txt

[Profile for XMPP Resources]
http://www.xmpp.org/internet-drafts/...pp-resourcepre...

There's a C# implementation of this RFC that's part of the libidn
library.
There's also a C++ & Java version.
http://www.gnu.org/software/libidn/

We've actually got a full implemention of stringprep as well - it's
much
more .Net 2.0 ish than the libidn one, which is just a native C++ app
that
was then ported to Java & .Net. It's found in our open-source SoapBox
Framework.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"Jon Skeet [C# MVP]" <[e-mail] in message
news:MP************************@msnews.microsoft. com...
Ornette <[e-mail]:
So how would you do ?

>The mapping table idea you had before looked best to me, although I
wouldn't quite implement it the same way. I'd have a look up table for
every possible character, where it defaults to the Unicode character,
but for all the accented characters you care about, you specify the
non-accented version.

>You'd then call ToCharArray() on the string in question, go through
each character replacing the original with the mapped character, and
then create a new string with the char array.

>It does require you to manually map all the accented characters you
care about though.

>My guess is that there are libraries around to do this somewhere, but
I
don't know of any myself.

>--
Jon Skeet - <[e-mail]>
http://www.pobox.com/~skeet Blog:http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Ornette]

Hello,

I following the thread which is very interesting.
I's thinking about wrinting a "StringHelper" class to do the jos and re-use
:-)

Thanks a lot for your participation to this subject !!!

Ornette.
"Chris Mullins [MVP]" <cm******@yahoo.coma écrit dans le message de
news:e$**************@TK2MSFTNGP06.phx.gbl...

Opps. Definatly KD!

We want to do the decomposition & make our changes. For KC would decompose
& then perform a canonical recompose - which would defeat the purpose!

I've never used (or even seen) the DecoderReplacementFallback - that's
another good idea. By now the original poster has probably given up and
will never try any of these solutions, but I think they would very cleanly
do the trick.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"JR" <No****@qsm.co.ilwrote in message
news:%2******************@TK2MSFTNGP06.phx.gbl...

>You meant NormalizationForm.FormKD.

Looking into it, I see a simpler method: After normalization, use
ASCIIEncoding with DecoderReplacementFallback replacing invalid ASCII
characters (which will be the accents) with the empty string.

JR

"Chris Mullins" <cm******@yahoo.com???
??????:11*********************@31g2000cwt.googleg roups.com...
I hadn't thought of that, but it's certainly an option.

Doing the normalization in .Net 2.0 is easy enough:

string s = "test";
string normalized = s.Normalize(NormalizationForm.FormKC);

Then you can iterate over the normalized string looking for (and
removing) the accents.

--
Chris Mullins
On Mar 1, 11:15 pm, "JR" <[e-mail]:

>>In a more general way:

There is a Unicode database at

http://www.unicode.org/Public/UNIDATA/

You could do what you want in two steps: decompose the string to base
characters followed by accent (NFKD normalization), then remove the
accents.

JR

"Chris Mullins [MVP]" <[e-mail]ëúá
áäåãòä:[e-mail].. .

The closest thing that comes to mind is an RFC called stringprep.
There
are a wide variety of stringprep profiles, and while they don't quite
do
what you're looking for, they're close. Included in stringprep is a
set of
mapping tables for Uppder->Lower case conversions. These are (in that
context) called case-foldings, are are found in table B.2.
Unfortunatly,
they're Upper->Lower, not the other way around.

Stringprep:
http://www.faqs.org/rfcs/rfc3454.html

There are a number of profiles:
[Profile for Internaional Domain Names]
http://www.rfc-editor.org/rfc/rfc3491.txt

[Profile for iSCSI names]
http://tools.ietf.org/html/draft-iet...string-prep-01

[Profile for SASL UserNames & Passwords]
http://www.ietf.org/rfc/rfc4013.txt

[Profile for XMPP Resources]
http://www.xmpp.org/internet-drafts/...pp-resourcepre...

There's a C# implementation of this RFC that's part of the libidn
library.
There's also a C++ & Java version.
http://www.gnu.org/software/libidn/

We've actually got a full implemention of stringprep as well - it's
much
more .Net 2.0 ish than the libidn one, which is just a native C++ app
that
was then ported to Java & .Net. It's found in our open-source SoapBox
Framework.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"Jon Skeet [C# MVP]" <[e-mail] in message
news:MP************************@msnews.microsoft .com...
Ornette <[e-mail]:
So how would you do ?

The mapping table idea you had before looked best to me, although I
wouldn't quite implement it the same way. I'd have a look up table
for
every possible character, where it defaults to the Unicode character,
but for all the accented characters you care about, you specify the
non-accented version.

You'd then call ToCharArray() on the string in question, go through
each character replacing the original with the mapped character, and
then create a new string with the char array.

It does require you to manually map all the accented characters you
care about though.

My guess is that there are libraries around to do this somewhere, but
I
don't know of any myself.

--
Jon Skeet - <[e-mail]>
http://www.pobox.com/~skeet Blog:http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Chris Mullins [MVP]]

I'm sitting in front of my computer, and not feeling much like working (too
busy sneezing and being sick!), so I broke down and turned the solution to
this problem into a little blog entry.

http://www.coversant.com/Default.asp...=88&EntryID=30

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"Ornette" <[e-mail] in message
news:AB**********************************@microsof t.com...

Hello,

I following the thread which is very interesting.
I's thinking about wrinting a "StringHelper" class to do the jos and
re-use :-)

Thanks a lot for your participation to this subject !!!

Ornette.
"Chris Mullins [MVP]" <cm******@yahoo.coma écrit dans le message de
news:e$**************@TK2MSFTNGP06.phx.gbl...

>Opps. Definatly KD!

We want to do the decomposition & make our changes. For KC would
decompose & then perform a canonical recompose - which would defeat the
purpose!

I've never used (or even seen) the DecoderReplacementFallback - that's
another good idea. By now the original poster has probably given up and
will never try any of these solutions, but I think they would very
cleanly do the trick.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"JR" <No****@qsm.co.ilwrote in message
news:%2******************@TK2MSFTNGP06.phx.gbl. ..

>>You meant NormalizationForm.FormKD.

Looking into it, I see a simpler method: After normalization, use
ASCIIEncoding with DecoderReplacementFallback replacing invalid ASCII
characters (which will be the accents) with the empty string.

JR

"Chris Mullins" <cm******@yahoo.com???
??????:11*********************@31g2000cwt.google groups.com...
I hadn't thought of that, but it's certainly an option.

Doing the normalization in .Net 2.0 is easy enough:

string s = "test";
string normalized = s.Normalize(NormalizationForm.FormKC);

Then you can iterate over the normalized string looking for (and
removing) the accents.

--
Chris Mullins
On Mar 1, 11:15 pm, "JR" <[e-mail]:
In a more general way:

There is a Unicode database at

http://www.unicode.org/Public/UNIDATA/

You could do what you want in two steps: decompose the string to base
characters followed by accent (NFKD normalization), then remove the
accents.

JR

"Chris Mullins [MVP]" <[e-mail]ëúá
áäåãòä:[e-mail]. ..

The closest thing that comes to mind is an RFC called stringprep.
There
are a wide variety of stringprep profiles, and while they don't quite
do
what you're looking for, they're close. Included in stringprep is a
set of
mapping tables for Uppder->Lower case conversions. These are (in that
context) called case-foldings, are are found in table B.2.
Unfortunatly,
they're Upper->Lower, not the other way around.

Stringprep:
http://www.faqs.org/rfcs/rfc3454.html

There are a number of profiles:
[Profile for Internaional Domain Names]
http://www.rfc-editor.org/rfc/rfc3491.txt

[Profile for iSCSI names]
http://tools.ietf.org/html/draft-iet...string-prep-01

[Profile for SASL UserNames & Passwords]
http://www.ietf.org/rfc/rfc4013.txt

[Profile for XMPP Resources]
http://www.xmpp.org/internet-drafts/...pp-resourcepre...

There's a C# implementation of this RFC that's part of the libidn
library.
There's also a C++ & Java version.
http://www.gnu.org/software/libidn/

We've actually got a full implemention of stringprep as well - it's
much
more .Net 2.0 ish than the libidn one, which is just a native C++ app
that
was then ported to Java & .Net. It's found in our open-source SoapBox
Framework.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"Jon Skeet [C# MVP]" <[e-mail] in message
news:MP************************@msnews.microsof t.com...
Ornette <[e-mail]:
So how would you do ?

The mapping table idea you had before looked best to me, although I
wouldn't quite implement it the same way. I'd have a look up table
for
every possible character, where it defaults to the Unicode
character,
but for all the accented characters you care about, you specify the
non-accented version.

You'd then call ToCharArray() on the string in question, go through
each character replacing the original with the mapped character, and
then create a new string with the char array.

It does require you to manually map all the accented characters you
care about though.

My guess is that there are libraries around to do this somewhere,
but I
don't know of any myself.

--
Jon Skeet - <[e-mail]>
http://www.pobox.com/~skeet Blog:http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Ornette]

Hello,

Thank you for this really understandable article (even for french people :-)
Nice & clear !!

Ornette

"Chris Mullins [MVP]" <cm******@yahoo.coma écrit dans le message de
news:ec**************@TK2MSFTNGP03.phx.gbl...

I'm sitting in front of my computer, and not feeling much like working
(too busy sneezing and being sick!), so I broke down and turned the
solution to this problem into a little blog entry.

http://www.coversant.com/Default.asp...=88&EntryID=30

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"Ornette" <[e-mail] in message
news:AB**********************************@microsof t.com...

>Hello,

I following the thread which is very interesting.
I's thinking about wrinting a "StringHelper" class to do the jos and
re-use :-)

Thanks a lot for your participation to this subject !!!

Ornette.
"Chris Mullins [MVP]" <cm******@yahoo.coma écrit dans le message de
news:e$**************@TK2MSFTNGP06.phx.gbl...

>>Opps. Definatly KD!

We want to do the decomposition & make our changes. For KC would
decompose & then perform a canonical recompose - which would defeat the
purpose!

I've never used (or even seen) the DecoderReplacementFallback - that's
another good idea. By now the original poster has probably given up and
will never try any of these solutions, but I think they would very
cleanly do the trick.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"JR" <No****@qsm.co.ilwrote in message
news:%2******************@TK2MSFTNGP06.phx.gbl.. .
You meant NormalizationForm.FormKD.

Looking into it, I see a simpler method: After normalization, use
ASCIIEncoding with DecoderReplacementFallback replacing invalid ASCII
characters (which will be the accents) with the empty string.

JR

"Chris Mullins" <cm******@yahoo.com???
??????:11*********************@31g2000cwt.googl egroups.com...
I hadn't thought of that, but it's certainly an option.

Doing the normalization in .Net 2.0 is easy enough:

string s = "test";
string normalized = s.Normalize(NormalizationForm.FormKC);

Then you can iterate over the normalized string looking for (and
removing) the accents.

--
Chris Mullins
On Mar 1, 11:15 pm, "JR" <[e-mail]:
In a more general way:
>
There is a Unicode database at
>
http://www.unicode.org/Public/UNIDATA/
>
You could do what you want in two steps: decompose the string to base
characters followed by accent (NFKD normalization), then remove the
accents.
>
JR
>
"Chris Mullins [MVP]" <[e-mail]ëúá
áäåãòä:[e-mail].. .
>
The closest thing that comes to mind is an RFC called stringprep.
There
are a wide variety of stringprep profiles, and while they don't
quite do
what you're looking for, they're close. Included in stringprep is a
set of
mapping tables for Uppder->Lower case conversions. These are (in
that
context) called case-foldings, are are found in table B.2.
Unfortunatly,
they're Upper->Lower, not the other way around.
>
Stringprep:
http://www.faqs.org/rfcs/rfc3454.html
>
There are a number of profiles:
[Profile for Internaional Domain Names]
http://www.rfc-editor.org/rfc/rfc3491.txt
>
[Profile for iSCSI names]
http://tools.ietf.org/html/draft-iet...string-prep-01
>
[Profile for SASL UserNames & Passwords]
http://www.ietf.org/rfc/rfc4013.txt
>
[Profile for XMPP Resources]
http://www.xmpp.org/internet-drafts/...pp-resourcepre...
>
There's a C# implementation of this RFC that's part of the libidn
library.
There's also a C++ & Java version.
http://www.gnu.org/software/libidn/
>
We've actually got a full implemention of stringprep as well - it's
much
more .Net 2.0 ish than the libidn one, which is just a native C++
app that
was then ported to Java & .Net. It's found in our open-source
SoapBox
Framework.
>
--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins
>
"Jon Skeet [C# MVP]" <[e-mail] in message
news:MP************************@msnews.microso ft.com...
Ornette <[e-mail]:
>So how would you do ?
>
The mapping table idea you had before looked best to me, although I
wouldn't quite implement it the same way. I'd have a look up table
for
every possible character, where it defaults to the Unicode
character,
but for all the accented characters you care about, you specify the
non-accented version.
>
You'd then call ToCharArray() on the string in question, go through
each character replacing the original with the mapped character,
and
then create a new string with the char array.
>
It does require you to manually map all the accented characters you
care about though.
>
My guess is that there are libraries around to do this somewhere,
but I
don't know of any myself.
>
--
Jon Skeet - <[e-mail]>
>http://www.pobox.com/~skeet Blog:http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Cor Ligthert [MVP]]

Chris,

Maybe for your blog, be aware that there is a Microsoft VisualBasic method
StrConv(mystring, VbStrConv.Narrow)

However I saw that it only is converting 16bit Asian languages to 8 bit not
European languages (Latin characters).

Cor

"Chris Mullins [MVP]" <cm******@yahoo.comschreef in bericht
news:ec**************@TK2MSFTNGP03.phx.gbl...

I'm sitting in front of my computer, and not feeling much like working
(too busy sneezing and being sick!), so I broke down and turned the
solution to this problem into a little blog entry.

http://www.coversant.com/Default.asp...=88&EntryID=30

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"Ornette" <[e-mail] in message
news:AB**********************************@microsof t.com...

>Hello,

I following the thread which is very interesting.
I's thinking about wrinting a "StringHelper" class to do the jos and
re-use :-)

Thanks a lot for your participation to this subject !!!

Ornette.
"Chris Mullins [MVP]" <cm******@yahoo.coma écrit dans le message de
news:e$**************@TK2MSFTNGP06.phx.gbl...

>>Opps. Definatly KD!

We want to do the decomposition & make our changes. For KC would
decompose & then perform a canonical recompose - which would defeat the
purpose!

I've never used (or even seen) the DecoderReplacementFallback - that's
another good idea. By now the original poster has probably given up and
will never try any of these solutions, but I think they would very
cleanly do the trick.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins

"JR" <No****@qsm.co.ilwrote in message
news:%2******************@TK2MSFTNGP06.phx.gbl.. .
You meant NormalizationForm.FormKD.

Looking into it, I see a simpler method: After normalization, use
ASCIIEncoding with DecoderReplacementFallback replacing invalid ASCII
characters (which will be the accents) with the empty string.

JR

"Chris Mullins" <cm******@yahoo.com???
??????:11*********************@31g2000cwt.googl egroups.com...
I hadn't thought of that, but it's certainly an option.

Doing the normalization in .Net 2.0 is easy enough:

string s = "test";
string normalized = s.Normalize(NormalizationForm.FormKC);

Then you can iterate over the normalized string looking for (and
removing) the accents.

--
Chris Mullins
On Mar 1, 11:15 pm, "JR" <[e-mail]:
In a more general way:
>
There is a Unicode database at
>
http://www.unicode.org/Public/UNIDATA/
>
You could do what you want in two steps: decompose the string to base
characters followed by accent (NFKD normalization), then remove the
accents.
>
JR
>
"Chris Mullins [MVP]" <[e-mail]ëúá
áäåãòä:[e-mail].. .
>
The closest thing that comes to mind is an RFC called stringprep.
There
are a wide variety of stringprep profiles, and while they don't
quite do
what you're looking for, they're close. Included in stringprep is a
set of
mapping tables for Uppder->Lower case conversions. These are (in
that
context) called case-foldings, are are found in table B.2.
Unfortunatly,
they're Upper->Lower, not the other way around.
>
Stringprep:
http://www.faqs.org/rfcs/rfc3454.html
>
There are a number of profiles:
[Profile for Internaional Domain Names]
http://www.rfc-editor.org/rfc/rfc3491.txt
>
[Profile for iSCSI names]
http://tools.ietf.org/html/draft-iet...string-prep-01
>
[Profile for SASL UserNames & Passwords]
http://www.ietf.org/rfc/rfc4013.txt
>
[Profile for XMPP Resources]
http://www.xmpp.org/internet-drafts/...pp-resourcepre...
>
There's a C# implementation of this RFC that's part of the libidn
library.
There's also a C++ & Java version.
http://www.gnu.org/software/libidn/
>
We've actually got a full implemention of stringprep as well - it's
much
more .Net 2.0 ish than the libidn one, which is just a native C++
app that
was then ported to Java & .Net. It's found in our open-source
SoapBox
Framework.
>
--
Chris Mullins, MCSD.NET, MCPD:Enterprise, Microsoft C# MVP
http://www.coversant.com/blogs/cmullins
>
"Jon Skeet [C# MVP]" <[e-mail] in message
news:MP************************@msnews.microso ft.com...
Ornette <[e-mail]:
>So how would you do ?
>
The mapping table idea you had before looked best to me, although I
wouldn't quite implement it the same way. I'd have a look up table
for
every possible character, where it defaults to the Unicode
character,
but for all the accented characters you care about, you specify the
non-accented version.
>
You'd then call ToCharArray() on the string in question, go through
each character replacing the original with the mapped character,
and
then create a new string with the char array.
>
It does require you to manually map all the accented characters you
care about though.
>
My guess is that there are libraries around to do this somewhere,
but I
don't know of any myself.
>
--
Jon Skeet - <[e-mail]>
>http://www.pobox.com/~skeet Blog:http://www.msmvps.com/jon.skeet
If replying to the group, please do not mail me too

[Chris Mullins]

As was pointed out to me in a blog comment, my original solution to
this problem - using the ASCII encoder to avoid the string iteration -
was pretty silly.

The code in my blog has been updated to use the correct method - the
Unicode Character Info class. More details at:
http://www.coversant.com/Default.asp...=88&EntryID=30

--
Chris Mullins