473,882 Members | 1,549 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

strtok behavior with multiple consecutive delimiters

Hello, and good whatever daytime is at your place..
please can somebody tell me, what the standard behavior of strtok shall be,
if it encounters two or more consecutive delimiters like in
(checks omitted)

char tst[] = "this\nis\n\nan \nempty\n\n\nli ne";
^^^^ ^^^^^^
char *tok = strtok(tst, "\n");
tok = strtok(NULL, "\n");
and so on..

will the groups of '\n' marked above be consumed one by one or the whole
group together?

Thank you very much
May 6 '06
34 24855
Christopher Benson-Manica wrote:
In comp.lang.c an**@servocomm. freeserve.co.uk wrote:
OTOH By using C++ life becomes more productive, less error prone,
less complicated and more elegant:
Not always... (digression warning)
std::cout << "token: \""<< *iter <<"\"\n";


IMHO this is harder for the programmer to read than

printf( "token: \"%s\"\n", str );


In this case I think it depends what you are used to!

To a certain extent this is a question of religion, but the difference
between the prevailing styles becomes more pronounced with heavily
formatted output:

printf( "%6s %2.2f %-18s:%u\n", val1, val2, val3, val4 );

Accomplishing the same thing with std::cout would be messy.


FWIW I think it might look like this:

std::cout
<< std::setw(6) << val1
<< ' ' << std::fixed << std::setw(2)
<< std::setprecisi on(2) << val2
<< ' ' << std::left << std::setw(18) << val3
<< ":" << val4 << '\n';

regards
Andy Little

May 7 '06 #21
Phlip wrote:

Geometer wrote:
I did :). I just wanted to know if this is the behavior required by the
standard and whether there is a difference betwenn C and C++.


Can we add to the FAQ "Please don't ask about strtok(), because everyone
here is ready to complain about it in endless ways"?

I *like* strtok(). It does what it does, and it does it very well.
You just have to understand how it works. Like so much in C, use
the right tool for job.

--
+----------------------------------------------------------------+
| Charles and Francis Richmond richmond at plano dot net |
+----------------------------------------------------------------+
May 7 '06 #22
an**@servocomm. freeserve.co.uk wrote:
Christopher Benson-Manica wrote:
To a certain extent this is a question of religion, but the difference
between the prevailing styles becomes more pronounced with heavily
formatted output:

printf( "%6s %2.2f %-18s:%u\n", val1, val2, val3, val4 );

Accomplishi ng the same thing with std::cout would be messy.

FWIW I think it might look like this:

std::cout
<< std::setw(6) << val1
<< ' ' << std::fixed << std::setw(2)
<< std::setprecisi on(2) << val2
<< ' ' << std::left << std::setw(18) << val3
<< ":" << val4 << '\n';

Which I'm sure you will admit, is a bit of an abomination!

Thank goodness C++ retains the C standard library for cases like this.

--
Ian Collins.
May 7 '06 #23
jacob navia wrote:
.... snip ...
I compiled your program in C++ using the VS 2005 compiler. The
executable size of that stuff was 180 224 bytes.

Then I compiled Chuck's version using his strtok function using the
lcc-win32 compiler (a C compiler, not a C++ one). The size was
14 645 bytes.


I compiled toksplit without the testing code, using gcc -Os, and
the generated object code was 0x5b bytes long. That's less than
100 bytes of object code.

The point is: measure the routine, not the testing program.

--
"If you want to post a followup via groups.google.c om, don't use
the broken "Reply" link at the bottom of the article. Click on
"show options" at the top of the article, then click on the
"Reply" at the bottom of the article headers." - Keith Thompson
More details at: <http://cfaj.freeshell. org/google/>
Also see <http://www.safalra.com/special/googlegroupsrep ly/>

May 7 '06 #24
Charles Richmond wrote:
I *like* strtok(). It does what it does, and it does it very well.
You just have to understand how it works. Like so much in C, use
the right tool for job.
I would agreed with you except for this:

Ben Pfaff wrote:
* It can only be used once at a time. If a sequence of
strtok() calls is ongoing and another one is started,
the state of the first one is lost. This isn't a
problem for small programs but it is easy to lose track
of such things in hierarchies of nested functions in
large programs. In other words, strtok() breaks
encapsulation.


Another term for "breaks encapsulation" is "refactor-hostile".

I want to refactor freely without worrying that moving a method call into
another one will cause two strtoks to step on each other's toes.

Other than that, yes it's just a tool, and I too will occassionally use it
where it works.

--
Phlip
http://c2.com/cgi/wiki?ZeekLand <-- NOT a blog!!!
May 7 '06 #25
Ian Collins wrote:
std::cout
<< std::setw(6) << val1
<< ' ' << std::fixed << std::setw(2)
<< std::setprecisi on(2) << val2
<< ' ' << std::left << std::setw(18) << val3
<< ":" << val4 << '\n';

Which I'm sure you will admit, is a bit of an abomination!

Thank goodness C++ retains the C standard library for cases like this.


Thank goodness C++ permits you to write custom IO manipulators, to bottle
all that up into something legible.

Try extending printf to accept your own % tags some time...

--
Phlip
http://c2.com/cgi/wiki?ZeekLand <-- NOT a blog!!!
May 7 '06 #26
Phlip wrote:
Ian Collins wrote:

std::cout
<< std::setw(6) << val1
<< ' ' << std::fixed << std::setw(2)
<< std::setprecisi on(2) << val2
<< ' ' << std::left << std::setw(18) << val3
<< ":" << val4 << '\n';


Which I'm sure you will admit, is a bit of an abomination!

Thank goodness C++ retains the C standard library for cases like this.

Thank goodness C++ permits you to write custom IO manipulators, to bottle
all that up into something legible.

Thank goodness the C standard library saves you the trouble of writing
custom IO manipulators, to bottle all that up into something legible.

--
Ian Collins.
May 7 '06 #27
Ian Collins wrote:
Thank goodness the C standard library saves you the trouble of writing
custom IO manipulators, to bottle all that up into something legible.


%*.*e

--
Phlip
http://c2.com/cgi/wiki?ZeekLand <-- NOT a blog!!!
May 7 '06 #28
CBFalconer a écrit :
jacob navia wrote:

... snip ...
I compiled your program in C++ using the VS 2005 compiler. The
executable size of that stuff was 180 224 bytes.

Then I compiled Chuck's version using his strtok function using the
lcc-win32 compiler (a C compiler, not a C++ one). The size was
14 645 bytes.

I compiled toksplit without the testing code, using gcc -Os, and
the generated object code was 0x5b bytes long. That's less than
100 bytes of object code.

The point is: measure the routine, not the testing program.


OK. With lcc-win32 the size of the toksplit routine is 84 bytes,
or 0x54 if you prefer hexa

May 7 '06 #29
<an**@servocomm .freeserve.co.u k> wrote
CBFalconer wrote:
The OP can simply use the following replacement function, which
does not have those objectionable features. The testing code is
longer than the function.
OTOH By using C++ life becomes more productive, less error prone,
less complicated and more elegant:

#include <sstream>
#include <string>
#include <vector>
#include <iostream>

int main()
{

char tst[] = "this\nis\n\nan \nempty\n\n\nli ne";

std::stringstre am s;
s << tst;

Now I get that this is an insertion operator. But why is it aliased with the
bit shift operator? Isn't that a bit counter intuitive?
std::vector<std ::string> tokens;
Then we get a line of pure gibberish. All it is doing is declaring an array
of strings.
while (! s.eof() ){
std::string str;
getline(s,str,' \n');
tokens.push_bac k(str);
}
Now in C, the function feof() will not return true until there has been a
read failure. So is this code correct or not? I suspect that it will fall
over if you terminate the input string with an "\n". But the whole thing is
so encapsulated and wrapped up that it is difficult to tell whehter the code
is bugged or not.

for (std::vector<st d::string>::con st_iterator iter
= tokens.begin();
iter !=tokens.end();
++iter){
Total gibberish. Four lines in the body of a for loop?
std::cout << "token: \""<< *iter <<"\"\n";
And this line isn't too pretty. Strings aren't too bad, but have you ever
tried to format a floating point number using these fancy stream interfaces?
}

}

regards
Andy Little

This is the reason I've given up using C++. The other reason is that
nowadays much of my code has to rin on a parallel computer, and have you
tried passing objects to other processes using the MPI interface?
--
www.personal.leeds.ac.uk/bgy1mm
My web page. Programming goodies.

May 7 '06 #30

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

33
1022
by: Geometer | last post by:
Hello, and good whatever daytime is at your place.. please can somebody tell me, what the standard behavior of strtok shall be, if it encounters two or more consecutive delimiters like in (checks omitted) char tst = "this\nis\n\nan\nempty\n\n\nline"; ^^^^ ^^^^^^ char *tok = strtok(tst, "\n");
0
9931
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9777
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
11108
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10403
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
9557
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
7113
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5978
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4601
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
3226
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.