strtok and strtok_r

[siddhu]

Dear experts,

As I know strtok_r is re-entrant version of strtok.
strtok_r() is called with s1(lets say) as its first parameter.
Remaining tokens from s1 are obtained by calling strtok_r() with a
null pointer for the first parameter.
My confusion is that this behavior is same as strtok. So I assume
strtok_r must also be using any function static variable to keep the
information about s1. If this is the case then how strtok_r is re-
entrant?
Otherwise how it keeps the information about s1?

Regards,
Siddharth

[Joachim Schmitz]

"Ben Bacarisse" <be********@bsb .me.ukschrieb im Newsbeitrag
news:87******** ****@bsb.me.uk. ..

CBFalconer <cb********@yah oo.comwrites:

>Joachim Schmitz wrote:

>>"Joachim Schmitz" <no*********@sc hmitz-digital.deschri eb:
"Joe Wright" <jo********@com cast.netschrieb :

... snip ...

>>>>>
size_t strlen(const char *s) {
size_t r = 0;
if (s) while (*s++) ++r;
return r;
}

Well, that version of strlen doesn't distinguish between a NULL
and an empty string. This would:
size_t strlen(const char *s) {
size_t r = (size_t)-1;
if (s) while (*s++) ++r;
return ++r;
}

Nonsense, thinko...
if (s) { r=0; while (*s++) ++r; }
return r;
or simpler
size_t strlen(const char *s) {
size_t r = 0;
if (s) while (*s++) ++r;
else r = (size_t)-1; /* and possibly set errno, e.g. to EINVAL */
return r;
}

If you bother to stop and think about it, I think you will see that
all four routines do the same thing.

Nope. The third and fourth correctly implement the specification
given for the second (which simply re-implements the behaviour of
the first).

I was quite surprised that my mistake went undetected for a day...

[Ben Bacarisse]

CBFalconer <cb********@yah oo.comwrites:

Ben Bacarisse wrote:

>James Antill <ja***********@ and.orgwrites:

... snip ...

>>>
Are you sure you weren't arguing about:

strcpy(dst, src);
vs.
memcpy(dst, src, strlen(src) + 1);

...as I find it hard to believe that Richard was arguing that
strcpy would be faster (possibly on certain platforms it's very
close, but not faster).

I was surprised to find it was. I decided to write a small test,
and for short strings, strdup written with strcpy is faster than
strdup written with memcpy (they both use strlen first to find
out how much to copy). The cut-off point on my laptop is at 164
bytes without any optimisation. With -O2 the cutoff goes down
to 16 bytes, but for small strings, strcpy seems faster. The
traditional YMMV seems to fall short of the mark here -- it
almost certainly will.

Just for fun, since you have the test code available, try comparing
it against the following small piece. Use the same optimization
levels etc.

char *dupstr(const char *s) {
char *p, *q;

if (p = q = malloc(1 + strlen(s)))
while (*p++ = *s++) continue;
return q;
} /* dupstr, untested */

which needs "#include <string.h>" and "#include <stdlib.h".

Without optimisation, worse than strcpy (as one would imagine). With
-O2 almost exactly the same as strcpy.

--
Ben.

[CBFalconer]

Joachim Schmitz wrote:

"CBFalconer " <cb********@yah oo.comschrieb:

.... snip ...

>

>Well, I was only considering the standard strlen specification,
and returning zero for a NULL input (i.e. treating NULL as a zero
length string). I think. I totally missed the urge to return an
error indication for NULL input (which I don't consider aa good
idea, since using it requires non-std accepting code).

Hmm, but how is returning 0 for input NULL any better than
returning (site_t)-1? What else should be returned on a NULL
input? What else should happen instead?

assert(s==NULL) ;

<

is the only other sensible other option I see, as it assures the
program to aborts right there with a somewhat usefull error
message.

Remember, we are discussing an implementation of strlen. The
point, to me, is that an empty string (normally represented by a
pointer to '\0' char) can be equally well represented by NULL,
saving some memory. This is usable as long as the purpose of the
accessing code is to make or use a copy of the string. It fails
miserably if the idea is to modify the string, but in that case the
modifying code will be what blows up.

However, returning a value of "(size_t)-1" is hopeless. This
requires that every call to strlen be immediately followed by a
test of the result, and doesn't allow the (possible) minor
advantage of representing an empty string by NULL.

--
Chuck F (cbfalconer at maineline dot net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home .att.net>

--
Posted via a free Usenet account from http://www.teranews.com

[Keith Thompson]

CBFalconer <cb********@yah oo.comwrites:
[...]

Remember, we are discussing an implementation of strlen. The
point, to me, is that an empty string (normally represented by a
pointer to '\0' char) can be equally well represented by NULL,
saving some memory. This is usable as long as the purpose of the
accessing code is to make or use a copy of the string. It fails
miserably if the idea is to modify the string, but in that case the
modifying code will be what blows up.

[...]

No, an empty string cannot *equally* well be represented by a null
pointer, at least not in standard C. You can do that if you like, as
long as all the code that uses these strings specifically handles that
special case. But you can't use any of the standard functions that
take string arguments unless you wrap each call in a check for a null
pointer.

#include <stdio.h>
int main(void)
{
char *s = NULL;
printf("s = \"%s\"\n", s);
puts(s);
return 0;
}

On one implementation this prints:

s = "(null)"
Segmentation fault (core dumped)

Such an approach also makes it difficult to distinguish between an
empty string and a nonexistent string.

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"

[CBFalconer]

Keith Thompson wrote:

CBFalconer <cb********@yah oo.comwrites:
[...]

>Remember, we are discussing an implementation of strlen. The
point, to me, is that an empty string (normally represented by a
pointer to '\0' char) can be equally well represented by NULL,
saving some memory. This is usable as long as the purpose of the
accessing code is to make or use a copy of the string. It fails
miserably if the idea is to modify the string, but in that case the
modifying code will be what blows up.

[...]

No, an empty string cannot *equally* well be represented by a null
pointer, at least not in standard C. You can do that if you like,
as long as all the code that uses these strings specifically
handles that special case. But you can't use any of the standard
functions that take string arguments unless you wrap each call in
a check for a null pointer.

Fair enough. I am guilty of unrestrained use of 'equally well', in
that I was only considering the programs actual use of such, and
ignoring the effect on other std procs.

--
Chuck F (cbfalconer at maineline dot net)
Available for consulting/temporary embedded and systems.
<http://cbfalconer.home .att.net>

--
Posted via a free Usenet account from http://www.teranews.com

[Tor Rustad]

On 17 Sep, 21:00, Richard Heathfield <[e-mail].i nvalidwrote:

[...]

Kind of you, James, and I must admit I find it hard to imagine arguing that
way as well. Unfortunately, I remember that I had some pretty strange
misconceptions about C a decade or so ago, so it's not utterly impossible.
But no, I don't remember this particular debate. Your mod seems plausible,
however. (I can't think of any reason why I'd want to call memcpy in that
way, however.)

I found the thread, it was more than 7 years ago.. you gave me that
lesson. Posting the link to the thread, would be rather humiliating
for me, so I prefer not to! :)

Here is the function R.H. wrote back then:

#include <stdio.h>
#include <stdlib.h>
#include <assert.h>
char *dupstr(char *s)
{
char *t = NULL;
assert(s != NULL);
if(s != NULL) /* assert won't fire if NDEBUG is defined, so check
anyway */
{
t = malloc(strlen(s ) + 1); /* allocate enough bytes to make a
copy
of the string pointed to by s */
if(t != NULL) /* did it work? */
{
strcpy(t, s); /* perform the copy */
}
}
return t; /* returns NULL if the string could not be created */
}

and Ben Pfaff instantly suggested the "char *dupstr(const char *s)"
improvement.