C++ string class tokenizer method?

[Generic Usenet Account]

Is it that I am blurry eyed, or is it indeed that the C++ string class
has no tokenizer method defined? I have defined my own functions [that
I have posted to comp.sources.d], but I would prefer to use the
standard functions, if available.

Thanks
Bhat

[Larry I Smith]

Generic Usenet Account wrote:

Is it that I am blurry eyed, or is it indeed that the C++ string class
has no tokenizer method defined? I have defined my own functions [that
I have posted to comp.sources.d], but I would prefer to use the
standard functions, if available.

Thanks
Bhat

Not in the same sense as the "C" strtok().
You can use c_str() to create a "C" nul-terminated
string from a std::string, then use strtok() on
that nul-terminated string.

char * cstr = strdup(cppstr.c_str());
// use strtok() on 'cstr'
// then free() 'cstr'

The C++ string class has many methods for parsing:
find(), find_first_of(), find_first_not_of(),
find_last_of(), find_last_not_of(), rfind(),
replace(), swap(), substr(), etc, etc, etc...

Given all of the methods of std::string, I've
never had to use strtok() with them.

Regards,
Larry

[Jack Klein]

On Thu, 16 Jun 2005 00:08:20 GMT, Larry I Smith
<la***********@verizon.net> wrote in comp.lang.c++:

Generic Usenet Account wrote:

Is it that I am blurry eyed, or is it indeed that the C++ string class
has no tokenizer method defined? I have defined my own functions [that
I have posted to comp.sources.d], but I would prefer to use the
standard functions, if available.

Thanks
Bhat

Not in the same sense as the "C" strtok().
You can use c_str() to create a "C" nul-terminated
string from a std::string, then use strtok() on
that nul-terminated string.

char * cstr = strdup(cppstr.c_str());

Just wanted to point out that:

1. There is no C or C++ standard library function strdup().

2. It is rather inadvisable to create a function of your own with
that name, seeing as how names beginning with "str" followed by a
lower case letter are reserved in many contexts in C++. And in all
contexts as a function name in C.
// use strtok() on 'cstr'
// then free() 'cstr'

The C++ string class has many methods for parsing:
find(), find_first_of(), find_first_not_of(),
find_last_of(), find_last_not_of(), rfind(),
replace(), swap(), substr(), etc, etc, etc...

Given all of the methods of std::string, I've
never had to use strtok() with them.

Regards,
Larry

--
Jack Klein
Home: http://JK-Technology.Com
FAQs for
comp.lang.c http://www.eskimo.com/~scs/C-faq/top.html
comp.lang.c++ http://www.parashift.com/c++-faq-lite/
alt.comp.lang.learn.c-c++
http://www.contrib.andrew.cmu.edu/~a...FAQ-acllc.html

[Larry I Smith]

Jack Klein wrote:

On Thu, 16 Jun 2005 00:08:20 GMT, Larry I Smith
<la***********@verizon.net> wrote in comp.lang.c++:

Generic Usenet Account wrote:

Is it that I am blurry eyed, or is it indeed that the C++ string class
has no tokenizer method defined? I have defined my own functions [that
I have posted to comp.sources.d], but I would prefer to use the
standard functions, if available.

Thanks
Bhat
Not in the same sense as the "C" strtok().
You can use c_str() to create a "C" nul-terminated
string from a std::string, then use strtok() on
that nul-terminated string.

char * cstr = strdup(cppstr.c_str());

Just wanted to point out that:

1. There is no C or C++ standard library function strdup().

Maybe, but I've never seen a "C" <string.h> (aka <cstring>) that
didn't have a strdup() - at least on MS Windows, Linux, & Solaris.
strtok(), strdup(), strlen(), strcpy(), etc are all valid "C"
functions that may be used in C++ when <string.h> or <cstring>
is included.


2. It is rather inadvisable to create a function of your own with
that name, seeing as how names beginning with "str" followed by a
lower case letter are reserved in many contexts in C++. And in all
contexts as a function name in C.

// use strtok() on 'cstr'
// then free() 'cstr'

The C++ string class has many methods for parsing:
find(), find_first_of(), find_first_not_of(),
find_last_of(), find_last_not_of(), rfind(),
replace(), swap(), substr(), etc, etc, etc...

Given all of the methods of std::string, I've
never had to use strtok() with them.

Regards,
Larry

Regards,
Larry

[Rolf Magnus]

Larry I Smith wrote:

The C++ string class has many methods for parsing:
find(), find_first_of(), find_first_not_of(),
find_last_of(), find_last_not_of(), rfind(),
replace(), swap(), substr(), etc, etc, etc...

And there are stringstreams.

[Default User]

Larry I Smith wrote:

Jack Klein wrote:

1. There is no C or C++ standard library function strdup().

Maybe, but I've never seen a "C" <string.h> (aka <cstring>) that
didn't have a strdup() - at least on MS Windows, Linux, & Solaris.
strtok(), strdup(), strlen(), strcpy(), etc are all valid "C"
functions that may be used in C++ when <string.h> or <cstring>
is included.

Three out the four you mention are standard C library functions that
are guaranteed to be available. However, strdup() is not. Regardless of
whether it's a common extension or not, we try very hard to stick to
the standard language in this newsgroup.

Brian

[Larry I Smith]

Default User wrote:

Larry I Smith wrote:

Jack Klein wrote:

1. There is no C or C++ standard library function strdup().

Maybe, but I've never seen a "C" <string.h> (aka <cstring>) that
didn't have a strdup() - at least on MS Windows, Linux, & Solaris.
strtok(), strdup(), strlen(), strcpy(), etc are all valid "C"
functions that may be used in C++ when <string.h> or <cstring>
is included.

Three out the four you mention are standard C library functions that
are guaranteed to be available. However, strdup() is not. Regardless of
whether it's a common extension or not, we try very hard to stick to
the standard language in this newsgroup.

Brian

Yes, I know, but on occasion real-world programs have to
use functions that are not in the Standard (read, write, socket,
fork, poll, select, etc). There's nothing wrong with that.

Regards,
Larry

[Default User]

Larry I Smith wrote:

Default User wrote:

Three out the four you mention are standard C library functions that
are guaranteed to be available. However, strdup() is not. Regardless of
whether it's a common extension or not, we try very hard to stick to
the standard language in this newsgroup.

Yes, I know, but on occasion real-world programs have to
use functions that are not in the Standard (read, write, socket,
fork, poll, select, etc). There's nothing wrong with that.

There is in this newgroup. Most of those are necessary for certain
platform-specific purposes. Platform-specific programs are best left to
the appropriate newsgroup.

It also doesn't apply to your use of strdup(). One can write that in
perfectly standard fashion:

char *s = new char[strlen(str.c_str() + 1];

strcpy (s, str.c_str());

Brian

[Larry I Smith]

Default User wrote:

Larry I Smith wrote:

Default User wrote:

Three out the four you mention are standard C library functions that
are guaranteed to be available. However, strdup() is not. Regardless of
whether it's a common extension or not, we try very hard to stick to
the standard language in this newsgroup.

Yes, I know, but on occasion real-world programs have to
use functions that are not in the Standard (read, write, socket,
fork, poll, select, etc). There's nothing wrong with that.

There is in this newgroup. Most of those are necessary for certain
platform-specific purposes. Platform-specific programs are best left to
the appropriate newsgroup.

It also doesn't apply to your use of strdup(). One can write that in
perfectly standard fashion:

char *s = new char[strlen(str.c_str() + 1];

strcpy (s, str.c_str());

Brian

I think you're making WAY too much out of this strdup() issue.
Just because strdup() is not in the Standard, doesn't mean it can't
be mentioned. Boost is mentioned all of the time - and NO,
there's no difference between the two, non-Standard is
non-Standard, whether it be Boost or strdup(). So what.

It's available (almost) everywhere; so I mentioned it as a
possible solution to the OP's question.

It's no big deal. Without C/C++ methods NOT in the Standards,
there wouldn't be any networking, GUI, etc. I'm surprised that
merely mentioning the ubiquitous strdup() has upset you so - sorry.

If you look at my orifinal post, you'll see that I'm well
aware of the alternatives. After 30 years in Information
Technology, I have some idea about how everything works.

Have a nice day.

Regards,
Larry

[Rolf Magnus]

When I write "here" somewhere down in my posting, I mean comp.lang.c++.

Larry I Smith wrote:

I think you're making WAY too much out of this strdup() issue.
Just because strdup() is not in the Standard, doesn't mean it can't
be mentioned.
It can be mentioned, but in comp.lang.c++, it's off-topic. So you should
avoid mentioning it here, even if only to avoid discussions about it being
off-topic. ;-)
If you write an example of how to do things and you use non-standard
functions for it, you can be sure that someone will tell you so.
However, if you feel you really _have_ to mention a non-standard function
here, you should at least explicitly write that it's not a standard C++
function. Most regulars (including myself) are quite pedantic about that.
Boost is mentioned all of the time - and NO,
there's no difference between the two, non-Standard is
non-Standard, whether it be Boost or strdup(). So what.
Well, boost is closely related to the standard in that the standard commitee
is discussing about it.
strdup isn't even allowed as it is now, since function names starting with
'str' followed by a lowercase letter are reserved. Also, the standard
headers are not allowed to contain additional non-standard functions. A
conforming implementaion is therefore actually not allowed to offer this
function.
So strdup is not only not part of the standard or in any way related to it,
but even violating it.
As a consequence, some compilers that do have that funciton will, when
switching on ISO-C++ mode, not offer it anymore and the program will fail
to compile, which shows that this is actually not just an academic issue.
It's available (almost) everywhere; so I mentioned it as a
possible solution to the OP's question.

It's no big deal. Without C/C++ methods NOT in the Standards,
there wouldn't be any networking, GUI, etc.

Probably, but "networking, GUI, etc." are off-topic here, too.

[Larry I Smith]

Rolf Magnus wrote:

When I write "here" somewhere down in my posting, I mean comp.lang.c++.

Larry I Smith wrote:

I think you're making WAY too much out of this strdup() issue.
Just because strdup() is not in the Standard, doesn't mean it can't
be mentioned.

It can be mentioned, but in comp.lang.c++, it's off-topic. So you should
avoid mentioning it here, even if only to avoid discussions about it being
off-topic. ;-)
If you write an example of how to do things and you use non-standard
functions for it, you can be sure that someone will tell you so.
However, if you feel you really _have_ to mention a non-standard function
here, you should at least explicitly write that it's not a standard C++
function. Most regulars (including myself) are quite pedantic about that.

Boost is mentioned all of the time - and NO,
there's no difference between the two, non-Standard is
non-Standard, whether it be Boost or strdup(). So what.

Well, boost is closely related to the standard in that the standard commitee
is discussing about it.
strdup isn't even allowed as it is now, since function names starting with
'str' followed by a lowercase letter are reserved. Also, the standard
headers are not allowed to contain additional non-standard functions. A
conforming implementaion is therefore actually not allowed to offer this
function.
So strdup is not only not part of the standard or in any way related to it,
but even violating it.
As a consequence, some compilers that do have that funciton will, when
switching on ISO-C++ mode, not offer it anymore and the program will fail
to compile, which shows that this is actually not just an academic issue.

It's available (almost) everywhere; so I mentioned it as a
possible solution to the OP's question.

It's no big deal. Without C/C++ methods NOT in the Standards,
there wouldn't be any networking, GUI, etc.

Probably, but "networking, GUI, etc." are off-topic here, too.

Yeah, I let the thread get out of control because 'Default User'
insulted my intelligence...

Sorry,
Larry

[Default User]

Larry I Smith wrote:

[bunch of stuff about strdup()]

I think you're making WAY too much out of this strdup() issue.
Topicality is important. All you had to do was say, "oh yeah, you're
right." It was the justification for off-topic posting that has led
this on.
Just because strdup() is not in the Standard, doesn't mean it can't
be mentioned. Boost is mentioned all of the time - and NO,
there's no difference between the two, non-Standard is
non-Standard, whether it be Boost or strdup(). So what.
I agree about Boost, and have said so many times.
It's available (almost) everywhere; so I mentioned it as a
possible solution to the OP's question.
Again, it wasn't anything necessary. It can be done in a standard
manner.
It's no big deal. Without C/C++ methods NOT in the Standards,
there wouldn't be any networking, GUI, etc.
That's true, but that doesn't those things topical. Normally we mention
them to a person as we are showing them the door, "what you need is
socket(), you'll need to go over to comp.unix.programmer for that."
I'm surprised that
merely mentioning the ubiquitous strdup() has upset you so - sorry.
You are confusing a desire to maintain topicality with being upset. I'm
sorry that my pointing out your failure to respect the newsgroup has
upset you so.
If you look at my orifinal post, you'll see that I'm well
aware of the alternatives. After 30 years in Information
Technology, I have some idea about how everything works.

That's terrific.
Brian

[cookie__raver]

Check out STLSoft's string_tokeniser (note the UK English spelling!).
(http://www.stlsoft.org/downloads.html) The docs for it suck (like they
do for all their stuff) but the code's brilliant, and efficient too.

The header file for it has examples of the use in a 'unit-test'
section, but even that's not obvious.

Basically, we use it either for splitting strings with characters, as
in:

typedef stlsoft::string_tokeniser<std::string, char> s2c_tok_t;

s2c_tok_t tokens("/usr/bin:/usr/sbin::", ':');

for(s2c_tok_t::const_iterator i = tokens.begin(); i != tokens.end();
++i)
{
std::cout << *i << std::endl;
}

or for splitting them with strings, as in:

typedef stlsoft::string_tokeniser<std::string, std::string> s2s_tok_t;

s2s_tok_t tokens("[][]/usr/bin[]/usr/sbin", "[]");

for(s2s_tok_t::const_iterator i = tokens.begin(); i != tokens.end();
++i)
{
std::cout << *i << std::endl;
}
HTH

The Raver

Generic Usenet Account wrote:

Is it that I am blurry eyed, or is it indeed that the C++ string class
has no tokenizer method defined? I have defined my own functions [that
I have posted to comp.sources.d], but I would prefer to use the
standard functions, if available.

Thanks
Bhat