473,219 Members | 1,453 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,219 software developers and data experts.

how to convert characters to upper case in utf8 env.

Hi, i have a situaion where i need to convert the characters entered in

an text field to upper case using C. The configuration id utf8
environment in which user can enter any character (single , double,
triple byte etc). I need to convert to upper case only those characters

which has got upper case. ie if an user enter bot english and japanese
characters in the text field, then I should convert only english
characters, not japanese.

I have seen that the C functions toupper() and tolower() handles multi
byte characters from Solaris 8. I am not sure with other platforms.
can any one suggest the best approch for the above scenario.

Mar 16 '06 #1
8 20192
cs******@gmail.com writes:
Hi, i have a situaion where i need to convert the characters entered in
Hi. Do you know you triple-posted this?
an text field to upper case using C. The configuration id utf8
environment in which user can enter any character (single , double,
triple byte etc). I need to convert to upper case only those characters

which has got upper case. ie if an user enter bot english and japanese
characters in the text field, then I should convert only english
characters, not japanese.

I have seen that the C functions toupper() and tolower() handles multi
byte characters from Solaris 8. I am not sure with other platforms.

can any one suggest the best approch for the above scenario.


The encodings supported by your C implementation for operations in
toupper() and tolower() are implementation-defined: you'll need to
look it up in your documentation.

You might need to use setlocale() with something like "en_US.UTF8",
and restore the locale afterwards. This may or may not work. And you
still won't be able to work with multibyte strings directly: you'll
have to convert to and from wchar_t's.

Your best bet is to use a (off-topic) specialized library devoted to
manipulating UTF8 strings. IBM has one:
http://www-306.ibm.com/software/glob.../icu/index.jsp

This provides a u_strToLower() function in ustring.h. Please don't
post here for questions regarding this library, however, as it is
off-topic for this NG.

HTH,
-Micah
Mar 16 '06 #2
Micah Cowan a crit :
Your best bet is to use a (off-topic) specialized library devoted to
manipulating UTF8 strings. IBM has one:


And glib can do it too.
Mar 17 '06 #3
On Thursday 16 March 2006 20:26, Micah Cowan opined (in
<87************@mcowan.barracudanetworks.com>):
cs******@gmail.com writes:

Hi, i have a situaion where i need to convert the characters entered


Hi. Do you know you triple-posted this?


It's the blinkin' Google. It does it sometimes.

--
BR, Vladimir

"There is hopeful symbolism in the fact that flags do not wave in a
vacuum."
-- Arthur C. Clarke

Mar 17 '06 #4
cs******@gmail.com wrote:
Hi, i have a situaion where i need to convert the characters entered in

an text field to upper case using C. The configuration id utf8
environment in which user can enter any character (single , double,
triple byte etc). I need to convert to upper case only those characters


Latin based uppercasing is easy, just convert to those characters
exactly in the lower-case or upper-case ASCII range. This is one of
the properties of UTF-8. However, to perform correct case change over
the whole Unicode range, you need to simply know which characters have
either a upper case or capitalization case alternative character (as
well as the reverse.) This information is available from the standard
Unicode data table.

Oh yeah, and this off topic here in comp.lang.c. ANSI C does not have
a notion of portable internationalization, let alone Unicode (though
some compilers implement wchar_t as Unicode, this cannot be relied
upon.)

--
Paul Hsieh
http://www.pobox.com/~qed/
http://bstring.sf.net/

Mar 17 '06 #5
On Thu, 16 Mar 2006 10:17:13 -0800, csanjith wrote:
Hi, i have a situaion where i need to convert the characters entered in

an text field to upper case using C. The configuration id utf8
environment in which user can enter any character (single , double,
triple byte etc). I need to convert to upper case only those characters

which has got upper case. ie if an user enter bot english and japanese
characters in the text field, then I should convert only english
characters, not japanese.

I have seen that the C functions toupper() and tolower() handles multi
byte characters from Solaris 8. I am not sure with other platforms.


It would seem improbable that toupper(), on Solaris or elsewhere, could
give the correct output for all valid input when using UTF-8, UTF-16,
UTF-32 or any other Unicode encoding variant.

For all Unicode encoding variants there are some "characters" (or
"graphemes" in Unicode terminology) which can be encoded
equivalently across multiple sets of integer values. Assume UTF-32, and
our Unicode string is an array of uint32_t integers. The Latin-1 character
a+umlaut could be described equivalently within the range of one 32-bit
integer with a value of 0x000000C1 or by combining two integers,
0x00000041-0x00000301. The latter representation could not be passed to
toupper() or tolower(), considering that neither can take an array.

[http://www.unicode.org/faq/char_combmark.html#8]

Hmmmm. Do any Unicode gurus know if 0x0061-0301 would accomplish a capital
A + umlaut? Regardless, I strongly suspect that there are many graphemes
in many scripts where such a trick could never work, but where notions
like uppercase or lowercase are still meaningful.
Mar 17 '06 #6
On Thu, 16 Mar 2006 17:29:38 -0800, websnarf wrote:
cs******@gmail.com wrote:
Hi, i have a situaion where i need to convert the characters entered in

an text field to upper case using C. The configuration id utf8
environment in which user can enter any character (single , double,
triple byte etc). I need to convert to upper case only those characters
Latin based uppercasing is easy, just convert to those characters
exactly in the lower-case or upper-case ASCII range. This is one of
the properties of UTF-8. However, to perform correct case change over
the whole Unicode range, you need to simply know which characters have
either a upper case or capitalization case alternative character (as
well as the reverse.) This information is available from the standard
Unicode data table.


Latin != ASCII. ASCII is 7-bit, ISO Latin encodings are 8-bit. Non-ASCII
code points, in UTF-8, are multibyte. How do you pass an array to toupper()?
Oh yeah, and this off topic here in comp.lang.c. ANSI C does not have
a notion of portable internationalization, let alone Unicode (though
some compilers implement wchar_t as Unicode, this cannot be relied
upon.)


The wide-character API is not sufficient to support Unicode.
But you're right, this is off-topic. Anybody know where this would be
on-topic, though?
Mar 17 '06 #7
William Ahern <wi*****@25thandClement.com> wrote:
On Thu, 16 Mar 2006 10:17:13 -0800, csanjith wrote:
Hi, i have a situaion where i need to convert the characters entered in
an text field to upper case using C. The configuration id utf8
environment in which user can enter any character (single , double,
triple byte etc). I need to convert to upper case only those characters
which has got upper case. ie if an user enter bot english and japanese
characters in the text field, then I should convert only english
characters, not japanese.
For all Unicode encoding variants there are some "characters" (or
"graphemes" in Unicode terminology) which can be encoded
equivalently across multiple sets of integer values. Assume UTF-32, and
our Unicode string is an array of uint32_t integers. The Latin-1 character
a+umlaut could be described equivalently within the range of one 32-bit
integer with a value of 0x000000C1 or by combining two integers,
0x00000041-0x00000301. The latter representation could not be passed to
toupper() or tolower(), considering that neither can take an array.

[http://www.unicode.org/faq/char_combmark.html#8]

Hmmmm. Do any Unicode gurus know if 0x0061-0301 would accomplish a capital
A + umlaut?


If I read the Unicode Standard correctly, yes, it would. However, the
right question is: do you really _want_ to capitalise an accented lowed
case letter to an accented upper case letter? In Dutch you wouldn't.

There are (at least) two reasonable C solutions:
- trust that your implementation handles this correctly, for example by
letting the sysadmin of the system the program runs on install
language-specific libraries for the <ctype.h> functions, and just use
tolower() and toupper(), as you would otherwise;
- assume that you know better than J. Random Sysadmin which characters
you want to capitalise, and write your own case-changing functions
with knowledge about the Unicode tables.
Something is to be said for either solution; the first is simpler and
more flexible, in the second case the results are more strictly known.

Richard
Mar 17 '06 #8


we******@gmail.com wrote On 03/16/06 20:29,:
cs******@gmail.com wrote:
Hi, i have a situaion where i need to convert the characters entered in

an text field to upper case using C. The configuration id utf8
environment in which user can enter any character (single , double,
triple byte etc). I need to convert to upper case only those characters



Latin based uppercasing is easy, just convert to those characters
exactly in the lower-case or upper-case ASCII range. [...]


Note that all of "" appear
in the alphabets of Latinic (Latinous?) languages, are
outside the ASCII lower-case range, and yet have upper-
case equivalents.

--
Er*********@sun.com

Mar 17 '06 #9

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

23
by: Hallvard B Furuseth | last post by:
Has someone got a Python routine or module which converts Unicode strings to lowercase (or uppercase)? What I actually need to do is to compare a number of strings in a case-insensitive manner,...
43
by: Vladimir | last post by:
Method UnicodeEncoding.GetMaxByteCount(charCount) returns charCount * 2. Method UTF8Encoding.GetMaxByteCount(charCount) returns charCount * 4. But why that? Look: /* Each Unicode character...
3
by: hunterb | last post by:
I have a file which has no BOM and contains mostly single byte chars. There are numerous double byte chars (Japanese) which appear throughout. I need to take the resulting Unicode and store it in a...
13
by: Dan V. | last post by:
How do I create a one line text file with these control codes? e.g.: 144 = 0x90 and 147 = 0x93? I am trying to create a one line text file with these characters all one one row with no spaces. ...
6
by: Friso Wiskerke | last post by:
Hi all, I'm creating a fixed length textfile with data which is sent out to a third-party which in turn reads the file and processes it. Some of the characters are not part of the lower ASCII...
2
by: Dino Buljubasic | last post by:
Hi, I would like to convert characters as typed in my text box to upper case. How can I do this? Thank you
3
by: csanjith | last post by:
Hi, i have a situaion where i need to convert the characters entered in an text field to upper case using C. The configuration id utf8 environment in which user can enter any character (single ,...
4
by: uday.sen | last post by:
Hi, I need to convert a string from UTF8 to wide character (wchar_t *). I perform the same in windows using: MultiByteToWideChar(CP_UTF8, 0, pInput, -1, pOutput, nLen); However, in linux...
4
by: Tux SC | last post by:
Hello, I have a JavaScript function which receives as input a string which in some cases contain non-UTF8 characters. The output of this function is used by another utility which fails if it...
0
by: VivesProcSPL | last post by:
Obviously, one of the original purposes of SQL is to make data query processing easy. The language uses many English-like terms and syntax in an effort to make it easy to learn, particularly for...
3
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 3 Jan 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). For other local times, please check World Time Buddy In...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: MeoLessi9 | last post by:
I have VirtualBox installed on Windows 11 and now I would like to install Kali on a virtual machine. However, on the official website, I see two options: "Installer images" and "Virtual machines"....

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.