By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
424,968 Members | 1,588 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 424,968 IT Pros & Developers. It's quick & easy.

why is my iostream program so much slower than equivalent cstdio program?

P: n/a
I am using gnu g++ version 3.3.2,
trying a simple test to read in and then
write out a large (100,000 line) text file

##########################################
CSTDIO VERSION TO READ/WRITE TEXT FILE:

#include <cstdlib>
#include <cstdio>
using namespace std;

main(int argc, void** argv) {

FILE* in;
if ((in = fopen("zinput","r")) == NULL) { exit(1); }

FILE* out;
if ((out = fopen("zoutput","w")) == NULL) { exit(1); }

char string[200];

while (fgets(string,199,in) != NULL) {
fprintf(out, "%s", string);
}

}

##########################################
EQUIVALENT IOSTREAM VERSION TO READ/WRITE TEXT FILE:

#include <cstdlib>
#include <iostream>
#include <fstream>
using namespace std;

main(int argc, void** argv) {

ifstream input("zinput");;
if ( !input.good() ) { exit(-1); }

ofstream output("zoutput");
if ( !output.good() ) { exit(-1); }

char string[200];

while ( !input.getline(string, 199).eof() ) {
output << string << endl;
}

}

##########################################

The problem is, the CSTDIO version runs
10 TIMES FASTER thant the IOSTREAM version!
Here are results from running the program with
the time command:

CSTDIO VERSION:
real 0.2
user 0.1
sys 0.0

IOSTREAM VERSION:
real 2.6
user 1.0
sys 1.4

Does anyone know why the IOSTREAM version of this
simple program is so much slower than the CSTDIO version?

Jul 22 '05 #1
Share this Question
Share on Google+
20 Replies


P: n/a
"Mark" <go*****@usgs.gov> wrote in message
news:11**********************@f14g2000cwb.googlegr oups.com...
I am using gnu g++ version 3.3.2,
trying a simple test to read in and then
write out a large (100,000 line) text file

##########################################
CSTDIO VERSION TO READ/WRITE TEXT FILE:

#include <cstdlib>
#include <cstdio>
using namespace std;

main(int argc, void** argv) {

FILE* in;
if ((in = fopen("zinput","r")) == NULL) { exit(1); }

FILE* out;
if ((out = fopen("zoutput","w")) == NULL) { exit(1); }

char string[200];

while (fgets(string,199,in) != NULL) {
fprintf(out, "%s", string);
}

}

##########################################
EQUIVALENT IOSTREAM VERSION TO READ/WRITE TEXT FILE:

#include <cstdlib>
#include <iostream>
#include <fstream>
using namespace std;

main(int argc, void** argv) {

ifstream input("zinput");;
if ( !input.good() ) { exit(-1); }

ofstream output("zoutput");
if ( !output.good() ) { exit(-1); }

char string[200];

while ( !input.getline(string, 199).eof() ) {
output << string << endl;
}

}

##########################################

The problem is, the CSTDIO version runs
10 TIMES FASTER thant the IOSTREAM version!
Here are results from running the program with
the time command:

CSTDIO VERSION:
real 0.2
user 0.1
sys 0.0

IOSTREAM VERSION:
real 2.6
user 1.0
sys 1.4

Does anyone know why the IOSTREAM version of this
simple program is so much slower than the CSTDIO version?


These are the results for a test of my own. This was years ago and was for
VC++ 5 or 6.
// Time to read a file with 26000 points total:
// Using ifstream: 11 sec.
// Using FILE: 0.9 sec.

I think I posted this once before and PJ Plauger, who wrote at least that
part of the MS standard library (maybe all of it), said the cause was some
bug or problem in the library. Whether GNU has a similar problem I don't
know. I'd be interested to know if there is a library out there whose
streams are not so inefficient that you are forced to do your file I/O in C.

DW
Jul 22 '05 #2

P: n/a
Mark wrote:
output << string << endl;


std::endl does an implicit flush. I would do this instead,

output << string << '\n';

As long as the file is openned in text mode, the '\n' will convert to
the proper line terminator for the platform.

On my platform, changing the std::endl to '\n' changed 225% slowdown to
61% slowdown. If you wanted to do better, you could try dealing with
the streambufs directly instead of iostreams. It might help.
--
Dave O'Hearn

Jul 22 '05 #3

P: n/a
Thanks for the advice. Now the IOSTREAM version is only twice as slow.
Mark

Jul 22 '05 #4

P: n/a
Mark wrote:
The problem is, the CSTDIO version runs
10 TIMES FASTER thant the IOSTREAM version!
Here are results from running the program with
the time command:

CSTDIO VERSION:
real 0.2
user 0.1
sys 0.0

IOSTREAM VERSION:
real 2.6
user 1.0
sys 1.4

Does anyone know why the IOSTREAM version of this
simple program is so much slower than the CSTDIO version?


Did you turn on optimizations?

-dr
Jul 22 '05 #5

P: n/a
Mark wrote:
I am using gnu g++ version 3.3.2,
trying a simple test to read in and then
write out a large (100,000 line) text file cat main.C #include <cstdlib>
#include <cstdio>

int main(int argc, char* argv[]) {
int value = EXIT_SUCCESS;
if (1 < argc) {
FILE* in = fopen(argv[1], "r");
if (NULL != in) {
FILE* out = fopen("zoutput", "w");
if (NULL != out) {
char string[200];
while (NULL != fgets(string, 199, in)) {
fprintf(out, "%s", string);
}
}
else {
fprintf(stderr, "Failed to open output file: "
"zoutput\n");
value = EXIT_FAILURE;
}
}
else {
fprintf(stderr, "Failed to open input file: %s\n",
argv[1]);
value = EXIT_FAILURE;
}
}
else {
fprintf(stderr, "usage: %s <input file name>\n", argv[0]);
}
return value;
}
g++ -Wall -ansi -pedantic -O3 -o main main.C
time ./main zinput 0.070u 0.050s 0:00.11 109.0% 0+0k 0+0io 183pf+0w rm zoutput
time ./main zinput 0.080u 0.040s 0:00.11 109.0% 0+0k 0+0io 183pf+0w rm zoutput
time ./main zinput 0.070u 0.050s 0:00.11 109.0% 0+0k 0+0io 183pf+0w rm zoutput
cat main.cc #include <cstdlib>
#include <iostream>
#include <fstream>

int main(int argc, char* argv[]) {
int value = EXIT_SUCCESS;
if (1 < argc) {
std::ifstream input(argv[1]);
if (input.good()) {
std::ofstream output("zoutput");
if (output.good()) {
char string[200];
while (!input.getline(string, 199).eof()) {
output << string << '\n';
}
}
else {
std::cerr << "Failed to open output file: zoutput"
<< std::endl;
fprintf(stderr, "Failed to open output file: "
"zoutput\n");
value = EXIT_FAILURE;
}
}
else {
std::cerr << "Failed to open input file: " << argv[1]
<< std::endl;
value = EXIT_FAILURE;
}
}
else {
std::cerr << "usage: " << argv[0] << "<input file name>"
<< std::endl;
}
return value;
}
g++ -Wall -ansi -pedantic -O3 -o main main.cc
time ./main zinput 0.100u 0.060s 0:00.16 100.0% 0+0k 0+0io 217pf+0w rm zoutput
time ./main zinput 0.120u 0.040s 0:00.16 100.0% 0+0k 0+0io 217pf+0w rm zoutput
time ./main zinput 0.120u 0.050s 0:00.16 106.2% 0+0k 0+0io 217pf+0w ls -s zoutput 6416 zoutput g++ --version g++ (GCC) 3.4.0 cat /etc/redhat-release

Red Hat Linux release 8.0 (Psyche)
Jul 22 '05 #6

P: n/a

"Mark" <go*****@usgs.gov> wrote in message news:11**********************@f14g2000cwb.googlegr oups.com...
[snip]
Does anyone know why the IOSTREAM version of this
simple program is so much slower than the CSTDIO version?


Results of comparative performance measurement "cstdio vs. iostream" can be seen at
http://article.gmane.org/gmane.comp.....perfometer/45
http://article.gmane.org/gmane.comp.....perfometer/38
http://groups-beta.google.com/group/...52787bac28942e

--
Alex Vinokur
email: alex DOT vinokur AT gmail DOT com
http://mathforum.org/library/view/10978.html
http://sourceforge.net/users/alexvn


Jul 22 '05 #7

P: n/a

"Alex Vinokur" <al****@big-foot.com> wrote in message news:32*************@individual.net...

"Mark" <go*****@usgs.gov> wrote in message news:11**********************@f14g2000cwb.googlegr oups.com...
[snip]
Does anyone know why the IOSTREAM version of this
simple program is so much slower than the CSTDIO version?


Results of comparative performance measurement "cstdio vs. iostream" can be seen at
http://article.gmane.org/gmane.comp.....perfometer/45
http://article.gmane.org/gmane.comp.....perfometer/38
http://groups-beta.google.com/group/...52787bac28942e


Also
http://article.gmane.org/gmane.comp.....perfometer/71
http://groups-beta.google.com/group/...a0e2e589527647
--
Alex Vinokur
email: alex DOT vinokur AT gmail DOT com
http://mathforum.org/library/view/10978.html
http://sourceforge.net/users/alexvn


Jul 22 '05 #8

P: n/a
Mark wrote:
I am using gnu g++ version 3.3.2,
trying a simple test to read in and then
write out a large (100,000 line) text file

##########################################
CSTDIO VERSION TO READ/WRITE TEXT FILE:

#include <cstdlib>
#include <cstdio>
using namespace std;

main(int argc, void** argv) {

FILE* in;
if ((in = fopen("zinput","r")) == NULL) { exit(1); }

FILE* out;
if ((out = fopen("zoutput","w")) == NULL) { exit(1); }

char string[200];

while (fgets(string,199,in) != NULL) {
fprintf(out, "%s", string);
}

}

##########################################
EQUIVALENT IOSTREAM VERSION TO READ/WRITE TEXT FILE:

#include <cstdlib>
#include <iostream>
#include <fstream>
using namespace std;

main(int argc, void** argv) {

ifstream input("zinput");;
if ( !input.good() ) { exit(-1); }

ofstream output("zoutput");
if ( !output.good() ) { exit(-1); }

char string[200];

while ( !input.getline(string, 199).eof() ) {
output << string << endl;
}

}

##########################################

The problem is, the CSTDIO version runs
10 TIMES FASTER thant the IOSTREAM version!
Here are results from running the program with
the time command:

CSTDIO VERSION:
real 0.2
user 0.1
sys 0.0

IOSTREAM VERSION:
real 2.6
user 1.0
sys 1.4

Does anyone know why the IOSTREAM version of this
simple program is so much slower than the CSTDIO version?

Try also this funny option in your code:

output.sync_with_stdio(true);

while ( !input.getline(string, 199).eof() ) {
output << string << endl;

}

And flush() everything!;)

Jul 22 '05 #9

P: n/a
"GianGuz" <gi*****************@noze.it> wrote in message
Try also this funny option in your code:

output.sync_with_stdio(true);


Won't this make it slower?
Jul 22 '05 #10

P: n/a
GianGuz wrote:
Try also this funny option in your code:

output.sync_with_stdio(true);


Rather:
output.sync_with_stdio(false);

Best
Darek
Jul 22 '05 #11

P: n/a
Right! Sorry for the misleading...I would have write that, void!

Jul 22 '05 #12

P: n/a
Mark wrote:
Thanks for the advice. Now the IOSTREAM version is only twice
as slow.


At that point, it's probably a quality of implementation issue. E.
Robert Tisdale posted results where the iostreams code was faster. I
used the same compiler he did, but on a much older Red Hat, and mine
was 60% slower.

These benchmarks also may not be relevant to real performance, as the
file ends up paged in, and there are no page faults while reading it.
In most real situations, the page faults would dwarf any 60% or even
100% overhead.

--
Dave O'Hearn

Jul 22 '05 #13

P: n/a
output.sync_with_stdio(false) has no effect on runtime

However, using
<< '\n'
instead of
<< endl
did dramtically improve performance, from IOSTREAM
being 10 times slower to being only twice as slow

Mark

Jul 22 '05 #14

P: n/a
This is an interesting evaluation! ;O
I also have a program in which a lot of debugging informations were
flushed into a
synchronized ostream while the program, with its different threads,
runs.
I will try to substitute the endl with '\n' everywhere. I'm curious to
see if also my performance
will be improved!

Jul 22 '05 #15

P: n/a

"Mark" <go*****@usgs.gov> wrote in message news:11**********************@f14g2000cwb.googlegr oups.com...
output.sync_with_stdio(false) has no effect on runtime

However, using
<< '\n'
instead of
<< endl
did dramtically improve performance, from IOSTREAM
being 10 times slower to being only twice as slow

[snip]

Yes, look at C/C++ Performance Tests:
endl vs. "\n/" and '\n'
for
stdout, stderr, cout, cerr, clog, ostringstream, cout-to-file, cerr-to-file, clog-to-file
http://groups-beta.google.com/group/...a3f2c6e789dde7
http://groups-beta.google.com/group/...dc73df3fd7df08
--
Alex Vinokur
email: alex DOT vinokur AT gmail DOT com
http://mathforum.org/library/view/10978.html
http://sourceforge.net/users/alexvn


Jul 22 '05 #16

P: n/a
Dave Rahardja wrote:

Did you turn on optimizations?

This is very important. The C++ library files tend to make lots
of small inline functions (where as the STDIO uses alot of macros
and large monolithic functions). If you've got inlining disabled
it will be a LOT slower.
Jul 22 '05 #17

P: n/a
GianGuz wrote:
This is an interesting evaluation! ;O
I also have a program in which a lot of debugging informations were
flushed into a
synchronized ostream while the program, with its different threads,
runs.
I will try to substitute the endl with '\n' everywhere. I'm curious to
see if also my performance
will be improved!

Hi,

be careful with replacing 'endl' by '\n' when you write debugging
information. If your program crashes, there may be unflushed data in the
stream which won't be written to the hard disk, so valuable debugging
information (which you may need to find the reason for the crash) gets lost!

greetings
Martin
Jul 22 '05 #18

P: n/a
Ok i'll follow your suggestions. By the way a macro that substitutes
endl with '\n' when the
program isn't in debugging mode seems to me a good compromise!
Gianguglielmo

Jul 22 '05 #19

P: n/a
GianGuz wrote:
By the way a macro that substitutes
endl with '\n' when the
program isn't in debugging mode seems to me a good compromise!


I think this is a bad idea. There are other times when buffering and
flushing can cause behavior differences (esp. as regards deadlock in
some programming types) and it would be bad if the problem disappears
when you compile with debugging. I think it's better to just use what
you mean while you program.

I also tend to send debugging information to std::cerr instead of
std::cout. It makes it easier to debug filters, and (correct me if I'm
wrong) std::cerr isn't buffered. (OTOH, I generally use std::endl with
std::cerr anyway, so it wouldn't bite me if it were.)
Jul 22 '05 #20

P: n/a
Adam Peterson wrote:
I also tend to send debugging information to std::cerr instead
of std::cout. It makes it easier to debug filters, and (correct
me if I'm wrong) std::cerr isn't buffered. (OTOH, I generally
use std::endl with std::cerr anyway, so it wouldn't bite me if
it were.)


Correct, std::cerr isn't buffered. You may want to consider std::clog
instead, though. It writes to the same destination as std::cerr, but is
buffered. Using it properly can lead to many fewer flush operations
than using std::cerr, but you do have to remember to flush at the end
of every line. This is easy if you use a wrapper around the stream
though; I have one that inserts timestamps and flushes at the end of a
line.

--
Dave O'Hearn

Jul 22 '05 #21

This discussion thread is closed

Replies have been disabled for this discussion.