473,804 Members | 3,570 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

tokezing a string

Hi -

I get a seg-fault when I compile and run this simple program.
(seg-fault in first call to strtok). Any clues?
My gcc is "gcc version 4.1.1 20070105 (Red Hat 4.1.1-51)"

#include <string.h>
int main()
{
char *token;
char *line = "LINE TO BE SEPARATED";
char *search = " ";
/* Token will point to "LINE". */
token = strtok(line, search);
/* Token will point to "TO". */
token = strtok(NULL, search);
}

Jan 24 '07
16 2044

"Rolf Magnus" <ra******@t-online.dewrote in message
news:ep******** *****@news.t-online.com...
Chris Theis wrote:

>Even though strtok is not part of the standard library it's still a valid
function call in C++.

Actually, strtok _is_ part of the standard library.
You're of course right! That was a glitch on my part as I assumed the OP was
referring to what was referred to as the standard template lib.

Cheers
Chris
Jan 24 '07 #11
In article <ej************ *************** *****@4ax.com>,
MrAsm <in*****@invali d.comwrote:
>On Wed, 24 Jan 2007 12:33:52 +0100, "Chris Theis"
<ch*********** **@nospam.cern. chwrote:

>>or a more sophisticated version:

//////////////////////////////////////////////////////////////////////////////

inline std::vector<std ::stringTokeniz eString( const std::string& Text,
const std::string& Delimiters )

Very interesting code.

But may I ask:

1. Why are you defining the function as "inline"?
Is "inline" just for simple stuff like a simple accessor (Get/Set) and
similar...?
I would agree that defining it as inline is at best optional.
>2. Why you are returning the string vector?
Would be better to return the string vector as reference in parameter
list, to avoid copy constructors calls?
This is most likely not the case. You just think it will not be as efficient.

Read about return value optimization (RVO).

e.g.

void TokenizeString(
<<< your params >>>
/* out */ std::vector< std::string & Tokens
);
Peronally, I prefer the other style. It is clearer, easier to write, easier
to read, easier to maintain:

output_t theOutput = theFunct(theInp ut);
or
output_t theOutput(theFu nct(theInput));
vs

output_t theOutput; // Argh an unitialised variable !
theFunct(theInp ut, theOutput);

The first case is kind of natural, easy to read. The compiler probably turns
it into the second case. The second case is second nature to C++ programmers
but not quite as natural to read to others. The last case is by far the worst.
Artificially creating an empty object in an undesirable state, then no way to
immediately know if the following line is a bug:

theFunct(theOut put, theInput);

which will compile if for example this is a string transformation function
that takes a string as input and a string as output.

No, even if RVO didn't exist, 99% of the time I would prefer the first style.

Yan

Jan 24 '07 #12

On Jan 24, 4:24 pm, "Chris Theis" <christian.th.. .@nospam.cern.c h>
wrote:
On Jan 24, 12:38 pm, "Amit Gupta" <emaila...@gmai l.comwrote:
[SNIP]
char *token;
char *line = "LINE TO BE SEPARATED";
char *search = " ";
/* Token will point to "LINE". */
token = strtok(line, search);
/* Token will point to "TO". */
token = strtok(NULL, search);
1. Its not C++, Its C.Please do not top post!
I will really take care to release from bad habit of Top Posting,
the only point i want to make is "think twice when you are using strtok
in code."
I was gone thru very bad debuggin hours (at that time i was knowing
very less about multithreaded code and didn't read/understand man page
of strtok carefully)
>
Even though strtok is not part of the standard library it's still a valid
function call in C++.
2.If you are using C++ then try to find std library function.But which std library function would that be?
3.if you are using C then -- Dont use strtok, or use it with caution
and some Extra Checks for Null and related memory stuff.You're absolutely right on this one, although it does not help the OP a bit
with his problem.

Cheers
Chris
Jan 25 '07 #13

"red floyd" <no*****@here.d udewrote in message
news:FT******** ***********@new ssvr25.news.pro digy.net...
Chris Theis wrote:
You might be tempted for example to attempt to call strtok()
>with string object and thus end up doing somethin like this

strtok( line.c_str(), search);

actually, the above should fail to compile, since strtok() takes a char*
as its first argument, and string::c_str() returns a const char *.
You're absolutely right, but I've (unfortunately) seen compilers silently
getting away with it.

Cheers
Chris
Jan 25 '07 #14
"MrAsm" <in*****@invali d.comwrote in message
news:ej******** *************** *********@4ax.c om...
On Wed, 24 Jan 2007 12:33:52 +0100, "Chris Theis"
<ch************ *@nospam.cern.c hwrote:

>>or a more sophisticated version:

//////////////////////////////////////////////////////////////////////////////

inline std::vector<std ::stringTokeniz eString( const std::string& Text,
const std::string& Delimiters )

Very interesting code.

But may I ask:

1. Why are you defining the function as "inline"?
Is "inline" just for simple stuff like a simple accessor (Get/Set) and
similar...?
Actually the reason was somehow historical 'cause I simply copied and pasted
from my personal toolbox file. However, "inline" is simply a hint for the
compiler and it is actually not bound to follow it. With respect to
optimization the compiler is free to decide whether to inline the code or
not as long as it is guaranteed that the behavior does not change. For
example the rumor that virtual functions are never inlined is still going
around, although this doesn't really hold true as it is up to the compiler
and depends on the context
(http://msdn.microsoft.com/msdnmag/issues/0600/c/).
>
2. Why you are returning the string vector?
Would be better to return the string vector as reference in parameter
list, to avoid copy constructors calls?
Yan already answered this in more detail.

Cheers
Chris
Jan 25 '07 #15


On Jan 24, 3:33 am, "Chris Theis" <christian.th.. .@nospam.cern.c h>
wrote:
"Amit Gupta" <emaila...@gmai l.comwrote in
[...]
}or a more sophisticated version:

//////////////////////////////////////////////////////////////////////////////

inline std::vector<std ::stringTokeniz eString( const std::string& Text,
const std::string& Delimiters )
// Tokenize a passed string with respect to the provided delimiters
//
// e.g.
// string Line = "this_dog_i s mine";
// string Delimiters = " ,:_;#";
// vector<stringWo rdList = TokenizeString( Line, Delimiters );
//////////////////////////////////////////////////////////////////////////////
{
std::vector<std ::stringWordLis t;
std::string::si ze_type Begin, End;
std::string Word;

Begin = Text.find_first _not_of( Delimiters ); // skip blanks or whatever
one finds at the beginning
while( Begin != std::string::np os ) {
End = Text.find_first _of( Delimiters, Begin );
if( End == std::string::np os ) { // we'v reached the end without
finding another delimiter
End = Text.length();
}

Word.assign( Text.begin() + Begin, Text.begin() + End );
WordList.push_b ack( Word );
Begin = Text.find_first _not_of( Delimiters, End);
}

return WordList;
-Thanks

Jan 27 '07 #16
In article <ep**********@c ernne03.cern.ch >,
ch************* @nospam.cern.ch says...

[ ... ]
Thus, you might consider doing a simple tokenizer using stringstreams if
applicable:
[ code using stringstream elided ]
or a more sophisticated version:
[ code using find_first_of and find_first_not_ of elided ]

You can combine the two, using stringstreams with delimiters of your
choice. A stream considers something as a delimiter if its associated
locale says that character is whitespace.

One example is at:

http://groups.google.com/group/comp....81e95c03be9041

--
Later,
Jerry.

The universe is a figment of its own imagination.
Jan 27 '07 #17

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

16
6756
by: Krakatioison | last post by:
My sites navigation is like this: http://www.newsbackup.com/index.php?n=000000000040900000 , depending on the variable "n" (which is always a number), it will take me anywhere on the site... this number is always changing as I have hundreds of thousand of pages of text on my site. Problem: - in my opinion this just not only look weird, but the variable "n" (number)
5
31183
by: Stu Cazzo | last post by:
I have the following: String myStringArray; String myString = "98 99 100"; I want to split up myString and put it into myStringArray. If I use this: myStringArray = myString.split(" "); it will split myString up using the delimiter of 1 space so that
9
8005
by: John F Dutcher | last post by:
I use code like the following to retrieve fields from a form: recd = recd.append(string.ljust(form.getfirst("lname",' '),15)) recd.append(string.ljust(form.getfirst("fname",' '),15)) etc., etc. The intent is to finish by assigning the list to a string that I would write to disk: recstr = string.join(recd,'')
9
3700
by: Derek Hart | last post by:
I wish to execute code from a string. The string will have a function name, which will return a string: Dim a as string a = "MyFunctionName(param1, param2)" I have seen a ton of people discuss how reflection does this, but I cannot find the syntax to do this. I have tried several code example off of gotdotnet and other articles. Can somebody please show me the code to do this?
10
8195
by: Angus Leeming | last post by:
Hello, Could someone explain to me why the Standard conveners chose to typedef std::string rather than derive it from std::basic_string<char, ...>? The result of course is that it is effectively impossible to forward declare std::string. (Yes I am aware that some libraries have a string_fwd.h header, but this is not portable.) That said, is there any real reason why I can't derive an otherwise empty
37
4723
by: Kevin C | last post by:
Quick Question: StringBuilder is obviously more efficient dealing with string concatenations than the old '+=' method... however, in dealing with relatively large string concatenations (ie, 20-30k), what are the performance differences (if any with something as trivial as this) between initializing a new instance of StringBuilder with a specified capacity vs. initializing a new instance without... (the final length is not fixed) ie,
2
4786
by: Andrew | last post by:
I have written two classes : a String Class based on the book " C++ in 21 days " and a GenericIpClass listed below : file GenericStringClass.h // Generic String class
2
5077
by: s | last post by:
I'm getting compile errors on the following code: <code> #include <iostream> #include <fstream> #include <list> #include <string> using namespace std;
11
3668
by: Christopher Benson-Manica | last post by:
Let's say I have a std::string, and I want to replace all the ',' characters with " or ", i.e. "A,B,C" -> "A or B or C". Is the following the best way to do it? int idx; while( (idx=str.find_first_of(',')) >= 0 ) { str.replace( idx, 1, "" ); str.insert( idx, " or " ); }
0
9576
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10567
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10323
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10310
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9138
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
0
6847
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5515
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
4291
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
3809
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.