fileToString

[Phlip]

Newsgroupies:

Everyone (who is anyone) has written a 'fileToString()' function
before:

string fileToString(string fileName)
{
string result;
ifstream in ( fileName.c_str() );
char ch;
while( in.get(ch) ) result += ch;
return result;
}

It takes a file name and returns a string full of its contents. Text
contents assumed.

Here's the question: What's the fastest, or smallest, or smarmiest
possible Standard Library implementation? Could something like
std::copy() apply?

I wouldn't be surprised if mine were acceptably fast; if both ifstream
and std::string are buffered and delta-ed, respectively.

Please specify your library, if your solution pushes the envelop of
common C++ Standard compliance levels.

--
Phlip
http://www.greencheese.org/AliensAreComing
-- "I wasn't using my civil liberties, anyway..." --

[Mike Wahler]

"Phlip" <ph*******@yahoo.com> wrote in message
news:63*************************@posting.google.co m...

Newsgroupies:

Everyone (who is anyone) has written a 'fileToString()' function
before:

string fileToString(string fileName)
{
string result;
ifstream in ( fileName.c_str() );
char ch;
while( in.get(ch) ) result += ch;
return result;
}

It takes a file name and returns a string full of its contents. Text
contents assumed.

Here's the question: What's the fastest,
Can't tell without measuring.
or smallest,
Can't tell without measuring.
or smarmiest
Subjective term. :-)
possible Standard Library implementation? Could something like
std::copy() apply?
std::copy could be used, sure.

I wouldn't be surprised if mine were acceptably fast;
Nor I. Remember your compiler usually gets a crack at
optimizing, and many OS's have clever file buffering
mechanisms. Measure, measure, measure. :-)
if both ifstream
and std::string are buffered and delta-ed, respectively.

Please specify your library, if your solution pushes the envelop of
common C++ Standard compliance levels.

Here's what I'd probably write, if presented with your
specification (error checking omitted):

#include <fstream>
#include <iostream>
#include <sstream>
#include <string>

std::string fileToString(const std::string& name)
{
std::ifstream in(name.c_str());
std::ostringstream oss;
oss << in.rdbuf();
return oss.str();
}

int main()
{
std::cout << fileToString("file.txt") << '\n';
return 0;
}
Josuttis calls this way "probably the fastest way to
copy files with C++ IOStreams". (p 683).

-Mike

[red floyd]

Phlip wrote:

Newsgroupies:

Everyone (who is anyone) has written a 'fileToString()' function
before:

string fileToString(string fileName)
{
string result;
ifstream in ( fileName.c_str() );
char ch;
while( in.get(ch) ) result += ch;
return result;
}

It takes a file name and returns a string full of its contents. Text
contents assumed.

Here's the question: What's the fastest, or smallest, or smarmiest
possible Standard Library implementation? Could something like
std::copy() apply?

I wouldn't be surprised if mine were acceptably fast; if both ifstream
and std::string are buffered and delta-ed, respectively.

Please specify your library, if your solution pushes the envelop of
common C++ Standard compliance levels.

#include <iostream>
#include <string>
#include <iterator>
using namespace std;
string fileToString(string fileName)
{
ifstream in(fileName.c_str());
string s;
if (in)
s.assign(istream_iterator<char>(in), istream_iterator<char>());
return s;
}

[Dietmar Kuehl]

ph*******@yahoo.com (Phlip) wrote:

Here's the question: What's the fastest, or smallest, or smarmiest
possible Standard Library implementation? Could something like
std::copy() apply?

Here is what should be the fastest approach:

std::string fileToString(std::string const& name) {
std::ifstream in(name.c_str());
return std::string(std::istreambuf_iterator<char>(in),
std::istreambuf_iterator<char>());
}

Since this is a pretty specialized call and a few optimizations are
necessary for this to be fast, it is likely that the constructor of the
string internally actually does something like

std::copy(begin, end, std::back_inserter(*this));

.... and the actual optimizations are applied to 'std::copy()'. I'm, however,
not aware of any standard C++ library which currently optimizes stuff like
this to an extreme level: for this to be fast, it would be necessary to
'reserve()' memory before going ahead and actually reading the string. To
figure out the size, in turn, it would be necessary to have a difference
function which is specialized for 'std::istreambuf_iterator<>()' which
takes the underlying code conversion facet into account and which is indeed
used: 'std::istreambuf_iterator' is an input iterator and thus using
'std::distance()' would consume the sequence. On the other hand, it is
definitely doable. Of course, the copies are not necessarily that expensive
and it may be viable to dump the file blockwise into a string.

My guess is that the fastest approach to reading a file into a string using
current standard C++ library implementations involves using an
'std::ostringstream':

std::string fileToString(std::string const& name) {
std::ifstream in(name.c_str());
std::ostringstream out;
out << in.rdbuf();
return out.str();
}

Stream buffer operate block-oriented internally anyway while the iterator
approach requires that the "segmented sequence" optimization is implemented.
--
<mailto:di***********@yahoo.com> <http://www.dietmar-kuehl.de/>
Phaidros eaSE - Easy Software Engineering: <http://www.phaidros.com/>

[tom_usenet]

On Tue, 02 Dec 2003 00:20:42 GMT, red floyd <no*****@here.dude> wrote:

#include <iostream>
#include <string>
#include <iterator>
using namespace std;
string fileToString(string fileName)
{
ifstream in(fileName.c_str());
string s;
if (in)
s.assign(istream_iterator<char>(in), istream_iterator<char>());

That code skips whitespace, and is very slow. Much better would be:

if (in)
s.assign(istreambuf_iterator<char>(in), istreambuf_iterator<char>());

But this relies on s.assign being efficient for input iterators (and
it sometimes isn't). Better would be this semi-standard version:

string fileToString(string fileName)
{
ifstream in(fileName.c_str());
in.seekg(0, ios_base::end);
int pos = in.tellg();
in.seekg(0, ios_base::beg);
if (in)
{
std::vector<char> v(pos); //string might not be contiguous :(
in.read(&v[0], v.size());
return string(&v[0], v.size());
}
else
return string();
}

Tom

C++ FAQ: http://www.parashift.com/c++-faq-lite/
C FAQ: http://www.eskimo.com/~scs/C-faq/top.html

[Alex Vinokur]

"Dietmar Kuehl" <di***********@yahoo.com> wrote in message news:5b**************************@posting.google.c om...
[snip]

My guess is that the fastest approach to reading a file into a string using
current standard C++ library implementations involves using an
'std::ostringstream':

std::string fileToString(std::string const& name) {
std::ifstream in(name.c_str());
std::ostringstream out;
out << in.rdbuf();
return out.str();
}

Stream buffer operate block-oriented internally anyway while the iterator
approach requires that the "segmented sequence" optimization is implemented.

[snip]
Comparative performance tests can be seen at :
http://article.gmane.org/gmane.comp.....perfometer/22
The "ostringstream out; out << in.rdbuf()" method is the fastest if standard C++ library is used (Dietmar is right).
Outside C++ : mmap (http://www.opengroup.org/onlinepubs/...ions/mmap.html) is the fastest method

See also :
http://article.gmane.org/gmane.comp.....perfometer/21
http://groups.google.com/groups?selm....uni-berlin.de
=====================================
Alex Vinokur
mailto:al****@connect.to
http://up.to/alexvn
=====================================

[Phlip]

Dietmar Kuehl wrote:

ph*******@yahoo.com (Phlip) wrote:

Here's the question: What's the fastest, or smallest, or smarmiest
possible Standard Library implementation? Could something like
std::copy() apply?
Here is what should be the fastest approach:

std::string fileToString(std::string const& name) {
std::ifstream in(name.c_str());
return std::string(std::istreambuf_iterator<char>(in),
std::istreambuf_iterator<char>());
}

I like it because it's the smarmiest. But VC++6 dislikes it, and
emites the usual "screaming at a template" error message:

error C2665: 'basic_string<char,struct std::char_traits<char>,class
std::allocator<char> >::basic_string<char,struct
std::char_traits<char>,class std::allocator<char> >' : none of the 7
overloads can convert parameter 1 from type 'class
std::istreambuf_iterator<char,struct std::char_traits<char> >'
My guess is that the fastest approach to reading a file into a string using
current standard C++ library implementations involves using an
'std::ostringstream':

std::string fileToString(std::string const& name) {
std::ifstream in(name.c_str());
std::ostringstream out;
out << in.rdbuf();
return out.str();
}

Stream buffer operate block-oriented internally anyway while the iterator
approach requires that the "segmented sequence" optimization is implemented.

Ding! VC++ liked that one.

Thanks, to you and red floyd, for the suggestions!

--
Phlip

[Phlip]

Mike Wahler wrote:

Can't tell without measuring.
Josuttis disagrees with you - see below. And feel free to Google for
any of my lectures about premature optimization, or guessing.

In this case, we all agree that the C++ Standard Library enforces
certain performance profiles...
std::string fileToString(const std::string& name)
{
std::ifstream in(name.c_str());
std::ostringstream oss;
oss << in.rdbuf();
return oss.str();
} Josuttis calls this way "probably the fastest way to
copy files with C++ IOStreams". (p 683).

I humbly thank the newsgroup for reading Josuttis for me. ;-)

--
Phlip

[tom_usenet]

On Tue, 2 Dec 2003 20:24:10 +0200, "Alex Vinokur" <al****@bigfoot.com>
wrote:

"Dietmar Kuehl" <di***********@yahoo.com> wrote in message news:5b**************************@posting.google.c om...
[snip]

My guess is that the fastest approach to reading a file into a string using
current standard C++ library implementations involves using an
'std::ostringstream':

std::string fileToString(std::string const& name) {
std::ifstream in(name.c_str());
std::ostringstream out;
out << in.rdbuf();
return out.str();
}

Stream buffer operate block-oriented internally anyway while the iterator
approach requires that the "segmented sequence" optimization is implemented.

[snip]
Comparative performance tests can be seen at :
http://article.gmane.org/gmane.comp.....perfometer/22
The "ostringstream out; out << in.rdbuf()" method is the fastest if standard C++ library is used (Dietmar is right).
Outside C++ : mmap (http://www.opengroup.org/onlinepubs/...ions/mmap.html) is the fastest method

Repeated calls to istream::get should be avoided, for performance
reasons (each call constructs a sentry object, checks error states,
etc.). Instead use rdbuf() and streambuf::sgetc. e.g.

std::streambuf* rdbuf = in.rdbuf();
int c;
while ((c = rdbuf->sgetc()) != EOF)
s[i++] = static_cast<unsigned char>(c);

Tom

C++ FAQ: http://www.parashift.com/c++-faq-lite/
C FAQ: http://www.eskimo.com/~scs/C-faq/top.html

[Mike Wahler]

"Phlip" <ph*******@yahoo.com> wrote in message
news:63************************@posting.google.com ...

Mike Wahler wrote:

Can't tell without measuring.
Josuttis disagrees with you - see below.

I don't see anything below which indicates such a disagreement.
As a matter of fact, there cannot be a disagreement, since what
I presented was from his book, and not my own idea.
And feel free to Google for
any of my lectures about premature optimization, or guessing.
What did I guess? I simply repeated what I'd read. Also,
my several admonitions to 'measure' don't indicate advocation
of 'guessing'. Wasn't it you who first asked about 'fastest'
and 'smallest'?

Or have I completely misunderstood you? Clarifications welcome.

In this case, we all agree that the C++ Standard Library enforces
certain performance profiles...

std::string fileToString(const std::string& name)
{
std::ifstream in(name.c_str());
std::ostringstream oss;
oss << in.rdbuf();
return oss.str();
}

Josuttis calls this way "probably the fastest way to
copy files with C++ IOStreams". (p 683).

I humbly thank the newsgroup for reading Josuttis for me. ;-)

You're welcome. :-)

-Mike