By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
446,171 Members | 943 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 446,171 IT Pros & Developers. It's quick & easy.

tokenizing and reading a string by predifined type and width

P: n/a
Hi everyone,
Sometimes I need to tokenize a string and convert it into some value.
For example, "123.45" with predifined width 2/4 and type int/double
should be converted into an int 2 and a double 3.45.
Here is my code:

#include <iostream>
#include <string>

typedef unsigned ResultT;

const ResultT Success = 0;
const ResultT Arg1 = 1u;
const ResultT Arg2 = 1u << 1;
const ResultT Arg3 = 1u << 2;
const ResultT Arg4 = 1u << 3;
const ResultT Arg5 = 1u << 4;
const ResultT Arg6 = 1u << 5;
const ResultT Arg7 = 1u << 6;
const ResultT Arg8 = 1u << 7;
const ResultT Arg9 = 1u << 8;
const ResultT Arg10 = 1u << 9;

template <typename CharT>
std::basic_istream<CharT>& operator>>(std::basic_istream<CharT>& is,
std::basic_string<CharT>& str)
std::getline(is, str);
return is;

typename CharT,
typename T1
ResultT parseFixedFormat(const std::basic_string<CharT>& input,
T1& v1, std::string::size_type w1)
std::basic_istringstream<CharTis(input.substr(0, w1));
is >v1;
return is ? Success : Arg1;

typename CharT,
typename T1,
typename T2
ResultT parseFixedFormat(const std::basic_string<CharT>& input,
T1& v1, std::string::size_type w1,
T2& v2, std::string::size_type w2)
std::string::size_type offset = w1;
std::basic_istringstream<CharTis(input.size() offset ?
input.substr(offset, w2) :
is >v2;
return (is ? Success : Arg2) +
parseFixedFormat(input, v1, w1);

// and so on for more number of arguments...

std::string input("12345.67some string");
int i;
double d;
std::string s;
parseFixedFormat(input, i, 4, d, 4, s, 16);
// i == 1234, d == 5.67, s == "some string"
Question 1:
You will immediately notice that similar codes are repeated over and over
for more number of arguments. I'd like to know if there is simpler way
without reducing the users' conveinience.

Question 2:
operator>>(istream&, string&) in the standard reads a string word by word.
I defined another operator to read a string with whitespaces as a whole.
The overloaded operator >at the top is for that purpose. I suspected it
cause ambiguity but tried anyway, and it works for some reason. I have tried
with VC++ 7.1, gcc 3.4, and Como test page.
What is the reason why it works?
boost::offset_separator provides similar functionality, but it returns
only as a string. It is a nuisance for a user to convert every string
into a value of some type. boost::lexical_cast is out of consideration,
If you lexical_cast<int>("1 ") an exception is thrown. It does not allow
trailing spaces. I also have to handle those cases.

Thanks in advance for your advice.

ES Kim
Jul 4 '06 #1
Share this Question
Share on Google+
2 Replies

P: n/a
Hi Kim,

There is good reference implementation at


Jul 4 '06 #2

P: n/a
"Sanat" <sa**********@gmail.comwrote in message
Hi Kim,

There is good reference implementation at

I have mentioned boost has its own problem in the original post.
Thanks for your attention anyway.

ES Kim
Jul 5 '06 #3

This discussion thread is closed

Replies have been disabled for this discussion.