By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
455,513 Members | 1,653 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 455,513 IT Pros & Developers. It's quick & easy.

Parsing a string using istringstream

P: n/a
Hello all, I'm trying to write a function which given a std::string
parses the string by breaking the sentance up by whitespace (\t, ' ',
\n) and returns the result as a vector of strings. Here's what I have
so far:

std::vector<std::string> tokenize (std::string foo)
{
std::istringstream s (foo);
std::vector <std::string> v;
std::string tok;

for (;;) // infinite loop
{
// try to extract token
s >> tok;

// if string was read, push onto vector else break out
of loop
if (s.good())
v.push_back(tok);
else
break;
}

return v;
}

The problem is that now if given a string that doesn't have whitespace
at the end (ex - "this is a string"), the last token will be
passed up because s.good() will return false not when the last
extraction failed, but when there is no further input in the
istringstream. If I restructure the loop as:

while (s.good())
{
s >> tok;
v.push_back(tok);
}

then if there is whitespace after the last token (ex - "this is a
string "), then v will end up with the last token repeated, as
an extraction when there is only whitespace in the istringstream will
not modify the tok variable (so it keeps it's old value which was the
last token successfully read, and then v pushes this onto the end of the
vector).

Any suggestions?

Thanks,

Adam Parkin
Nov 9 '05 #1
Share this Question
Share on Google+
1 Reply


P: n/a
Adam Parkin wrote:
Hello all, I'm trying to write a function which given a std::string
parses the string by breaking the sentance up by whitespace (\t, ' ',
\n) and returns the result as a vector of strings. Here's what I have
so far:

std::vector<std::string> tokenize (std::string foo)
{
std::istringstream s (foo);
std::vector <std::string> v;
std::string tok;

for (;;) // infinite loop
{
// try to extract token
s >> tok;

// if string was read, push onto vector else break out of
loop
if (s.good())
v.push_back(tok);
else
break;
}

return v;
}

The problem is that now if given a string that doesn't have whitespace
at the end (ex - "this is a string"), the last token will be
passed up because s.good() will return false not when the last
extraction failed, but when there is no further input in the
istringstream. If I restructure the loop as:

while (s.good())
{
s >> tok;
v.push_back(tok);
}

then if there is whitespace after the last token (ex - "this is a
string "), then v will end up with the last token repeated, as
an extraction when there is only whitespace in the istringstream will
not modify the tok variable (so it keeps it's old value which was the
last token successfully read, and then v pushes this onto the end of the
vector).

Any suggestions?

Thanks,

Adam Parkin


while (s >> tok)
v.push_back(tok);

john
Nov 9 '05 #2

This discussion thread is closed

Replies have been disabled for this discussion.