473,803 Members | 2,972 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

using strtok

I cam across an interesting limitation to the use of strtok.

I have two strings on which I want strtok to operate.
However since strtok has only one memory of the residual string I must
complete one set of operations before starting on the second. This
is inconvenient in the context of my program!

So far the only solution I can see is to write a replacement for strtok
to use on one of the strings. Can anyone offer an alternative?

--
_ _______________ _______________ ___________
/ \._._ |_ _ _ /' Orpheus Internet Services
\_/| |_)| |(/_|_|_/ 'Internet for Everyone'
_______ | ___________./ http://www.orpheusinternet.co.uk
Mar 10 '07 #1
14 3326
On Mar 10, 12:35 pm, Mr John FO Evans <m...@orpheusma il.co.ukwrote:
I cam across an interesting limitation to the use of strtok.

I have two strings on which I want strtok to operate.
However since strtok has only one memory of the residual string I must
complete one set of operations before starting on the second. This
is inconvenient in the context of my program!

So far the only solution I can see is to write a replacement for strtok
to use on one of the strings. Can anyone offer an alternative?
Not really, unless you'd be willing to use <OT>POSIX/SUS's strtok_r()</
OTand skip on the portability. Otherwise, look at CBFalconer's
replacement toksplit(), discussed at
http://groups.google.com/group/comp....58085dd57c3a5b.
--
WYCIWYG - what you C is what you get

Mar 10 '07 #2

Mr John FO Evans wrote:
I cam across an interesting limitation to the use of strtok.

I have two strings on which I want strtok to operate.
However since strtok has only one memory of the residual string I must
complete one set of operations before starting on the second. This
is inconvenient in the context of my program!

So far the only solution I can see is to write a replacement for strtok
to use on one of the strings. Can anyone offer an alternative?
POSIX specifies a strtok_r that was designed to work around the
reentrancy issue of strtok. If you want full portability however,
you'll have to roll your own version. It's not difficult and can be
done in completely standard C. CBFalconer periodically publishes his
toksplit function to this group. Use Google Group's search facility to
locate the source.

Mar 10 '07 #3
Mr John FO Evans wrote:
>
I cam across an interesting limitation to the use of strtok.

I have two strings on which I want strtok to operate.
However since strtok has only one memory of the residual string I
must complete one set of operations before starting on the second.
This is inconvenient in the context of my program!

So far the only solution I can see is to write a replacement for
strtok to use on one of the strings. Can anyone offer an
alternative?
Try this:

/* ------- file toksplit.c ----------*/
#include "toksplit.h "

/* copy over the next token from an input string, after
skipping leading blanks (or other whitespace?). The
token is terminated by the first appearance of tokchar,
or by the end of the source string.

The caller must supply sufficient space in token to
receive any token, Otherwise tokens will be truncated.

Returns: a pointer past the terminating tokchar.

This will happily return an infinity of empty tokens if
called with src pointing to the end of a string. Tokens
will never include a copy of tokchar.

A better name would be "strtkn", except that is reserved
for the system namespace. Change to that at your risk.

released to Public Domain, by C.B. Falconer.
Published 2006-02-20. Attribution appreciated.
Revised 2006-06-13
*/

const char *toksplit(const char *src, /* Source of tokens */
char tokchar, /* token delimiting char */
char *token, /* receiver of parsed token */
size_t lgh) /* length token can receive */
/* not including final '\0' */
{
if (src) {
while (' ' == *src) src++;

while (*src && (tokchar != *src)) {
if (lgh) {
*token++ = *src;
--lgh;
}
src++;
}
if (*src && (tokchar == *src)) src++;
}
*token = '\0';
return src;
} /* toksplit */

#ifdef TESTING
#include <stdio.h>

#define ABRsize 6 /* length of acceptable token abbreviations */

/* ---------------- */

static void showtoken(int i, char *tok)
{
putchar(i + '1'); putchar(':');
puts(tok);
} /* showtoken */

/* ---------------- */

int main(void)
{
char teststring[] = "This is a test, ,, abbrev, more";

const char *t, *s = teststring;
int i;
char token[ABRsize + 1];

puts(teststring );
t = s;
for (i = 0; i < 4; i++) {
t = toksplit(t, ',', token, ABRsize);
showtoken(i, token);
}

puts("\nHow to detect 'no more tokens' while truncating");
t = s; i = 0;
while (*t) {
t = toksplit(t, ',', token, 3);
showtoken(i, token);
i++;
}

puts("\nUsing blanks as token delimiters");
t = s; i = 0;
while (*t) {
t = toksplit(t, ' ', token, ABRsize);
showtoken(i, token);
i++;
}
return 0;
} /* main */

#endif
/* ------- end file toksplit.c ----------*/

/* ------- file toksplit.h ----------*/
#ifndef H_toksplit_h
# define H_toksplit_h

# ifdef __cplusplus
extern "C" {
# endif

#include <stddef.h>

/* copy over the next token from an input string, after
skipping leading blanks (or other whitespace?). The
token is terminated by the first appearance of tokchar,
or by the end of the source string.

The caller must supply sufficient space in token to
receive any token, Otherwise tokens will be truncated.

Returns: a pointer past the terminating tokchar.

This will happily return an infinity of empty tokens if
called with src pointing to the end of a string. Tokens
will never include a copy of tokchar.

released to Public Domain, by C.B. Falconer.
Published 2006-02-20. Attribution appreciated.
*/

const char *toksplit(const char *src, /* Source of tokens */
char tokchar, /* token delimiting char */
char *token, /* receiver of parsed token */
size_t lgh); /* length token can receive */
/* not including final '\0' */

# ifdef __cplusplus
}
# endif
#endif
/* ------- end file toksplit.h ----------*/

--
<http://www.cs.auckland .ac.nz/~pgut001/pubs/vista_cost.txt>
<http://www.securityfoc us.com/columnists/423>

"A man who is right every time is not likely to do very much."
-- Francis Crick, co-discover of DNA
"There is nothing more amazing than stupidity in action."
-- Thomas Matthews

--
Posted via a free Usenet account from http://www.teranews.com

Mar 10 '07 #4

"matevzb" <ma*****@gmail. comwrote in message
news:11******** **************@ s48g2000cws.goo glegroups.com.. .
>So far the only solution I can see is to write a replacement for strtok
to use on one of the strings. Can anyone offer an alternative?
Not really, unless you'd be willing to use <OT>POSIX/SUS's strtok_r()</
OTand skip on the portability. Otherwise, look at CBFalconer's
replacement toksplit(), discussed at
are POSIX sources available?

Mar 11 '07 #5
"Servé Laurijssen" wrote:
"matevzb" <ma*****@gmail. comwrote in message
>>So far the only solution I can see is to write a replacement for
strtok to use on one of the strings. Can anyone offer an alternative?

Not really, unless you'd be willing to use <OT>POSIX/SUS's strtok_r()<
/OTand skip on the portability. Otherwise, look at CBFalconer's
replacement toksplit(), discussed at

are POSIX sources available?
Anything published here, as ready to go, will either run on POSIX
or there will be much wailing, teeth gnashing and berating from the
regulars. We deal with portable code here. Which, in turn, is why
POSIX is off-topic.

Please do not remove attribution lines for material you quote.
Those are the initial lines that say "Joe wrote:" or similar.

--
Chuck F (cbfalconer at maineline dot net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home .att.net>

--
Posted via a free Usenet account from http://www.teranews.com

Mar 11 '07 #6
On Mar 11, 3:59 pm, "Servé Laurijssen" <s...@n.tkwrote :
"matevzb" <mate...@gmail. comwrote in message

news:11******** **************@ s48g2000cws.goo glegroups.com.. .
So far the only solution I can see is to write a replacement for strtok
to use on one of the strings. Can anyone offer an alternative?
Not really, unless you'd be willing to use <OT>POSIX/SUS's strtok_r()</
OTand skip on the portability. Otherwise, look at CBFalconer's
replacement toksplit(), discussed at

are POSIX sources available?
POSIX/SUS, similar to ISO C, is a specification, so the answer would
be no. Source code for specific implementations may be available (e.g.
GNU libc), but whether or not they are portable and/or conform to
POSIX is another question. I'd say you're better off with toksplit().
--
WYCIWYG - what you C is what you get

Mar 11 '07 #7

"CBFalconer " <cb********@yah oo.comwrote in message
news:45******** *******@yahoo.c om...
Anything published here, as ready to go, will either run on POSIX
or there will be much wailing, teeth gnashing and berating from the
regulars. We deal with portable code here. Which, in turn, is why
POSIX is off-topic.
I was just wondering why your function is not considered off-topic but every
time posix is mentioned its off topic. One could mention that to get a
portable version of strtok_r you can strip it from posix.
Please do not remove attribution lines for material you quote.
Those are the initial lines that say "Joe wrote:" or similar.
was mistake sorry
Mar 11 '07 #8
"Servé Laurijssen" <se*@n.tkwrites :
I was just wondering why your function is not considered off-topic but every
time posix is mentioned its off topic. One could mention that to get a
portable version of strtok_r you can strip it from posix.
POSIX is a standard. It's not a collection of source code from
which you can strip anything.
--
int main(void){char p[]="ABCDEFGHIJKLM NOPQRSTUVWXYZab cdefghijklmnopq rstuvwxyz.\
\n",*q="kl BIcNBFr.NKEzjwC IxNJC";int i=sizeof p/2;char *strchr();int putchar(\
);while(*q){i+= strchr(p,*q++)-p;if(i>=(int)si zeof p)i-=sizeof p-1;putchar(p[i]\
);}return 0;}
Mar 11 '07 #9
santosh wrote:
POSIX specifies a strtok_r that was designed to work around the
reentrancy issue of strtok. If you want full portability however,
you'll have to roll your own version. It's not difficult and can be
done in completely standard C.
#include <stddef.h>

char *str_tok_r(char *s1, const char *s2, char **s3);
char *str_chr(const char *s, int c);
size_t str_spn(const char *s1, const char *s2);
size_t str_cspn(const char *s1, const char *s2);

char *str_tok_r(char *s1, const char *s2, char **s3)
{
if (s1 != NULL) {
*s3 = s1;
}
s1 = *s3 + str_spn(*s3, s2);
if (*s1 == '\0') {
return NULL;
}
*s3 = s1 + str_cspn(s1, s2);
if (**s3 != '\0') {
*(*s3)++ = '\0';
}
return s1;
}

size_t str_spn(const char *s1, const char *s2)
{
size_t n;

for (n = 0; *s1 != '\0' && str_chr(s2, *s1) != NULL; ++s1) {
++n;
}
return n;
}

size_t str_cspn(const char *s1, const char *s2)
{
size_t n;

for (n = 0; str_chr(s2, *s1) == NULL; ++s1) {
++n;
}
return n;
}

char *str_chr(const char *s, int c)
{
while (*s != (char)c) {
if (*s == '\0') {
return NULL;
}
++s;
}
return (char *)s;
}

--
pete
Mar 11 '07 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
2914
by: ern | last post by:
//I used strtok() like this: fgets(userCommand, MAX_COMMAND_SIZE, stdin); g_UserCommands = strtok(command, " "); while(g_UserCommands != NULL){ i+=1; g_UserCommands = strtok(NULL, " "); } //Now I want to use strcmp() like this:
1
1362
by: arssa2020 | last post by:
hello, i am writing a GPS app in evc++ 4.0, i am using c-style char* instead of CStrings, i want to parse NMEA sentence such as "GPZDA,162913.48,19,08,2005,,*63" problem is when i use strtok it ignores an empty data field below is the output for the sentence above "GPZDA" "162913.48"
20
17246
by: bubunia2000 | last post by:
Hi all, I heard that strtok is not thread safe. So I want to write a sample program which will tokenize string without using strtok. Can I get a sample source code for the same. For exp: 0.0.0.0--->I want to tokenize the string using delimiter as as dot. Regards
8
1932
by: hu | last post by:
hi, everybody! I'm testing the fuction of strtok(). The environment is WinXP, VC++6.0. Program is simple, but mistake is confusing. First, the below code can get right outcome:"ello world, hello dreams." #include <stdafx.h> #include <string.h> #include <stdio.h> int main()
29
2591
by: Pietro Cerutti | last post by:
Hello, here I have a strange problem with a real simple strtok example. The program is as follows: ### BEGIN STRTOK ### #include <string.h> #include <stdio.h>
11
904
by: Lothar Behrens | last post by:
Hi, I have selected strtok to be used in my string replacement function. But I lost the last token, if there is one. This string would be replaced select "name", "vorname", "userid", "passwort" from "users" order by "users"
4
4774
by: ohaqqi | last post by:
Hi everybody. I haven't programmed anything in about 8 years, I've read up a little bit on C and need to write a shell in C. I want to use strtok() to take an input from a user and parse it into the command and its arguments. for example: copy <file1> <file2> will copy file 2 to file 1, del <file1> will delete a file, etc. The exit command is all I've implemented right now, but even that produces an error when executed...I'm sure I've got a...
11
17180
by: magicman | last post by:
can anyone point me out to its implementation in C before I roll my own. thx
4
8092
by: spiralfire | last post by:
I wrote a translator, that reads a DIMACS graph format and writes to a simpler format... basically DIMACS format is: c comment p type nodes edges //type is alwats edge on my problems, nodes is the number of nodes and edges number of edges e v1 v2 //this means an edge connecting v1 and v2, both integers
0
9703
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9565
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10550
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10317
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10295
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9125
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
6844
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
1
4275
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
2972
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.