strstr for Unicode characters

[Kelvin Moss]

Hi all,

How could one write an strstr function to work with unicode characters?

Are there existing implementations/solutions/api for doing so?

Any pointers would be appreciated.

Thanks ..

[Papastefanos Serafeim]

? "Kelvin Moss" <km**********@yahoo.com?????? ??? ??????
news:11**********************@p79g2000cwp.googlegr oups.com...

Hi all,

How could one write an strstr function to work with unicode characters?

Are there existing implementations/solutions/api for doing so?

Any pointers would be appreciated.

Thanks ..

For windows, there is wcsstr.
Check
http://msdn.microsoft.com/library/de...c_._mbsstr.asp

--
Papastefanos Serafeim

[Ioannis Papadopoulos]

Kelvin Moss wrote:

Hi all,

How could one write an strstr function to work with unicode characters?

Are there existing implementations/solutions/api for doing so?

Any pointers would be appreciated.

Thanks ..

You could always use memcpy() or memmove().

[SM Ryan]

# Hi all,
#
# How could one write an strstr function to work with unicode characters?
#
# Are there existing implementations/solutions/api for doing so?

String functions should work just fine on UTF-8 encoded unicode
characters - minding that nonASCII characters will have codes greater
than 127 (or less than zero) and might be represented by multiple bytes.
For something like strstr which should only be looking for byte
sequences without embedded zeros, it should be fine, while strchr
can be problematically. There is also wide character (wc...) type
and functions becoming available which will probably be 16 bit or
wider unicode characters.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
Don't say anything. Especially you.

[Une bévue]

SM Ryan <wy*****@tango-sierra-oscar-foxtrot-tango.fake.orgwrote:

There is also wide character (wc...) type
and functions becoming available which will probably be 16 bit or
wider unicode characters.

for example as UTF16 used on Mac OS X File System ???
--
une bévue

[Kelvin Moss]

SM Ryan wrote:

# Hi all,
#
# How could one write an strstr function to work with unicode characters?
#
# Are there existing implementations/solutions/api for doing so?

String functions should work just fine on UTF-8 encoded unicode
characters - minding that nonASCII characters will have codes greater
than 127 (or less than zero) and might be represented by multiple bytes.
For something like strstr which should only be looking for byte
sequences without embedded zeros, it should be fine, while strchr
can be problematically.

Yes, I am dealing with UTF8 encoded Unicode characters. So you mean to
say that as long as I don't have embedded zeroes in the strings strstr
should be fine. Right? I think this assumption may not work quite well
in real applications. Your thoughts?

Thanks ..

[Stephen Sprunk]

"Kelvin Moss" <km**********@yahoo.comwrote in message
news:11**********************@p79g2000cwp.googlegr oups.com...

SM Ryan wrote:

>String functions should work just fine on UTF-8 encoded unicode
characters - minding that nonASCII characters will have codes greater
than 127 (or less than zero) and might be represented by multiple
bytes.
For something like strstr which should only be looking for byte
sequences without embedded zeros, it should be fine, while strchr
can be problematically.

Yes, I am dealing with UTF8 encoded Unicode characters. So you mean to
say that as long as I don't have embedded zeroes in the strings strstr
should be fine. Right? I think this assumption may not work quite
well
in real applications. Your thoughts?

UTF-8 won't have any embedded zeroes by definition; the encoding was
specifically designed to work transparently with C code that assumed
ASCII or some 8-bit ASCII-based encoding.

S

--
Stephen Sprunk "God does not play dice." --Albert Einstein
CCIE #3723 "God is an inveterate gambler, and He throws the
K5SSS dice at every possible opportunity." --Stephen Hawking

--
Posted via a free Usenet account from http://www.teranews.com

[Ioannis Papadopoulos]

Ioannis Papadopoulos wrote:

Kelvin Moss wrote:

>Hi all,

How could one write an strstr function to work with unicode characters?

Are there existing implementations/solutions/api for doing so?

Any pointers would be appreciated.

Thanks ..

You could always use memcpy() or memmove().

I do not know what I was thinking at the time. I thought you wanted a
function for strcpy. Please ignore my previous reply.

I tried strstr() for unicode chars and seems to work.

[micans]

Kelvin Moss wrote:

Hi all,

How could one write an strstr function to work with unicode characters?

Are there existing implementations/solutions/api for doing so?

Do you want to deal with issues such as normalization? E.g. combining
characters can be represented in (many) different ways. In that case,
I've previously worked in a project that used the (IBM) ICU libraries
(licensed under the X license, GPL compatible).

Stijn

[SM Ryan]

"Kelvin Moss" <km**********@yahoo.comwrote:
#
# SM Ryan wrote:
# # Hi all,
# #
# # How could one write an strstr function to work with unicode characters?
# #
# # Are there existing implementations/solutions/api for doing so?
# >
# String functions should work just fine on UTF-8 encoded unicode
# characters - minding that nonASCII characters will have codes greater
# than 127 (or less than zero) and might be represented by multiple bytes.
# For something like strstr which should only be looking for byte
# sequences without embedded zeros, it should be fine, while strchr
# can be problematically.
#
# Yes, I am dealing with UTF8 encoded Unicode characters. So you mean to
# say that as long as I don't have embedded zeroes in the strings strstr
# should be fine. Right? I think this assumption may not work quite well
# in real applications. Your thoughts?
#
# Thanks ..
#
#
#

--
SM Ryan http://www.rawbw.com/~wyrmwif/
I love the smell of commerce in the morning.

[SM Ryan]

pe*******@laponie.com.invalid (=?ISO-8859-1?Q?Une_b=E9vue?=) wrote:
# SM Ryan <wy*****@tango-sierra-oscar-foxtrot-tango.fake.orgwrote:
#
# There is also wide character (wc...) type
# and functions becoming available which will probably be 16 bit or
# wider unicode characters.
#
# for example as UTF16 used on Mac OS X File System ???

MacOSX file paths are UTF-8 encoding of Unicode (16 bit I think).
The file name length limit is the number of UTF-8 bytes.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
Leave it to the Catholics to destroy existence.

[SM Ryan]

"Kelvin Moss" <km**********@yahoo.comwrote:
#
# SM Ryan wrote:
# # Hi all,
# #
# # How could one write an strstr function to work with unicode characters?
# #
# # Are there existing implementations/solutions/api for doing so?
# >
# String functions should work just fine on UTF-8 encoded unicode
# characters - minding that nonASCII characters will have codes greater
# than 127 (or less than zero) and might be represented by multiple bytes.
# For something like strstr which should only be looking for byte
# sequences without embedded zeros, it should be fine, while strchr
# can be problematically.
#
# Yes, I am dealing with UTF8 encoded Unicode characters. So you mean to
# say that as long as I don't have embedded zeroes in the strings strstr
# should be fine. Right? I think this assumption may not work quite well
# in real applications. Your thoughts?

UTF-8 bytes are ASCII characters plus nonzero bytes; UTF-8 encoding
does not insert zero bytes where none existed before. As long as all
you're doing is shuffling bytes around, you can use most str* functions.
Functions like strchr which expect one char to be one character
will only work on the ASCII subset.

In FILEs, you have to negotiate with other programs how they will
interpret byte sequences. If all the applications assume UTF-8
encodings in FILEs, and they handle UTF-8 internally, then everything
will be fine.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
I love the smell of commerce in the morning.

[SM Ryan]

"Kelvin Moss" <km**********@yahoo.comwrote:
#
# SM Ryan wrote:
# # Hi all,
# #
# # How could one write an strstr function to work with unicode characters?
# #
# # Are there existing implementations/solutions/api for doing so?
# >
# String functions should work just fine on UTF-8 encoded unicode
# characters - minding that nonASCII characters will have codes greater
# than 127 (or less than zero) and might be represented by multiple bytes.
# For something like strstr which should only be looking for byte
# sequences without embedded zeros, it should be fine, while strchr
# can be problematically.
#
# Yes, I am dealing with UTF8 encoded Unicode characters. So you mean to
# say that as long as I don't have embedded zeroes in the strings strstr
# should be fine. Right? I think this assumption may not work quite well
# in real applications. Your thoughts?

UTF-8 bytes are ASCII characters plus nonzero bytes; UTF-8 encoding
does not insert zero bytes where none existed before. As long as all
you're doing is shuffling bytes around, you can use most str* functions.
Functions like strchr which expect one char to be one character
will only work on the ASCII subset.

In FILEs, you have to negotiate with other programs how they will
interpret byte sequences. If all the applications assume UTF-8
encodings in FILEs, and they handle UTF-8 internally, then everything
will be fine.

--
SM Ryan http://www.rawbw.com/~wyrmwif/
I love the smell of commerce in the morning.

[Une bévue]

SM Ryan <wy*****@tango-sierra-oscar-foxtrot-tango.fake.orgwrote:

>
MacOSX file paths are UTF-8 encoding of Unicode (16 bit I think).
The file name length limit is the number of UTF-8 bytes.

you're right is a extract on TN 2078 "migrating from FSSpecs to FSRefs"
:

struct FSRef {
UInt8 hidden[80]; /* private to File Manager*/
};

however at paragraph "FSRefs and long Unicode file names" they wrote :

OSErr FSRefGetName( const FSRef *fsRef, HFSUniStr255 *name )
{
return( FSGetCatalogInfo(fsRef, kFSCatInfoNone, NULL, name, NULL,
NULL) );
}

An HFSUniStr255 is defined as:

struct HFSUniStr255 {
UInt16 length; /* number of unicode characters */
UniChar unicode[255]; /* unicode characters */
};

How file names are encoded

HFS+ disks store file names as UTF-16 in an Apple-modified form of
-------------------------------^^^^^^^
Normalization Form D (decomposed). This form excludes certain
compatibility decompositions and parts of the symbol blocks, in order to
assure round-trip of file names to Mac OS encodings (applications using
the HFS APIs assume they get the same bytes out that they put in).

did I miss somethong ?
--
une bévue