fwrite output is not efficient (fast) ?

[Abubakar]

Hi,

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

regards,

...ab

[Richard Bos]

Abubakar <ab*******@gmai l.comwrote:

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing.

That most probably wasn't a C programmer, but a GNU-almost-POSIX-
with-a-bit-of-C-thrown-in-for-appearance's-sake programmer.

Is that true?

As phrased, it is not true. It may be true for some systems, under some
circumstances; but you should never worry about optimisation until you
_know_ that you have to, and don't just think you might.

Richard

[Gordon Burditt]

>recently some C programmer told me that using fwrite/fopen functions

>are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

For some programs I have benchmarked (which were essentially doing
disk-to-disk file copies), using single-character read() and write()
calls to copy files chews up a lot of CPU time. Switching to
getc()/putc()/fopen() calls reduced the CPU used to copy a given
file by a factor of 10 and the wall clock time to finish copying
the file by about 10%. In this particular situation, buffering
helped speed things up a lot.

[Richard Heathfield]

Abubakar said:

Hi,

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

I suggest you ask your friend:

1) why he thinks fopen outputs anything at all to a file;
2) why he thinks the buffering status of a file depends entirely on fopen
or fwrite;
3) what he thinks setbuf and setvbuf are for;
4) what he means by "efficient" .

You may well find the answers to those questions illuminating. On the other
hand, you might not.

1), 2), and 3) should all send him back-pedalling, and 4) should at least
make him stop and think about the real issues here.

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Nate Eldredge]

Abubakar <ab*******@gmai l.comwrites:

Hi,

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

It's true that the output is buffered (usually). Depending on your
situation, that may be good or bad. If you are calling fwrite with
large chunks of data (many kbytes), the buffering will not gain you
much, but a sensible implementation will bypass it and you'll only have
a small amount of overhead. If you are writing small chunks or single
characters at a time (for instance, with fprintf() or putc()), the
buffering can speed things up drastically, since it cuts down on calls
to the operating system, which can be expensive.

It is true in this case that some data may not be written as soon as you
call fwrite(), but this doesn't slow down the data transfer in the long
run. If it's important for some reason that a small bit of data go out
immediately, you can call fflush() or use setvbuf() to change the
buffering mode, but at a cost to overall efficiency.

[s0suk3]

On Oct 24, 2:03*am, Abubakar <ab*******@gmai l.comwrote:

Hi,

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

The purpose of buffering is actually efficiency; transferring a large
block of data to disk at once is faster than transferring it byte by
byte (and by this I don't mean that singe-byte functions such as
getc() and putc() are inefficient; these functions also read/write to
or from a buffer in memory).

This is usually true not only for disk files, but also for I/O devices
in general such as network sockets. In some applications, however, it
can be convenient to develop a more specialized I/O layer model that
can provide more functionality and improve efficiency in specific
situations.

(If you're just worried that the data will remain in the buffer
indefinitely, just call fflush() or close the file.)

Sebastian

[Abubakar]

well he is saying that using the *native* read and write to do the
same task using file descriptors is much faster than the fwrite etc.
He says using the FILE * that is used in case of the fopen/fwrite
buffers data and is not as fast and predictable as read write are.

I have given him the link to this discussion and he is going to be
posting his replies soon hopefully.

On Oct 24, 2:00*pm, Richard Heathfield <[e-mail].i nvalidwrote:

Abubakar said:

Hi,

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

I suggest you ask your friend:

1) why he thinks fopen outputs anything at all to a file;
2) why he thinks the buffering status of a file depends entirely on fopen
or fwrite;
3) what he thinks setbuf and setvbuf are for;
4) what he means by "efficient" .

You may well find the answers to those questions illuminating. On the other
hand, you might not.

1), 2), and 3) should all send him back-pedalling, and 4) should at least
make him stop and think about the real issues here.

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Richard Heathfield]

Abubakar said:

well he is saying that using the *native* read and write to do the
same task using file descriptors is much faster than the fwrite etc.

I suggest you ask your friend the questions that I asked you to ask him.

He says using the FILE * that is used in case of the fopen/fwrite
buffers data and is not as fast and predictable as read write are.

FILE * is a pointer type. It doesn't have a speed.

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[Abubakar]

Well he read the link and now he says that the replies by Nate
Eldredge and the guy whose email starts with "s0s" have said what
proves what he says is right so he says there is no need to reply to
anything. ummmm , i dont know whats up. I am going to be posting more
questions to clear things that he has told me because he has much more
experience in C than me. Thanks for the replies so far, if you guys
have more comments please continue posting.

On Oct 24, 1:49*pm, gordonb.l2...@b urditt.org (Gordon Burditt) wrote:

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

For some programs I have benchmarked (which were essentially doing
disk-to-disk file copies), using single-character read() and write()
calls to copy files chews up a lot of CPU time. *Switching to
getc()/putc()/fopen() calls reduced the CPU used to copy a given
file by a factor of 10 and the wall clock time to finish copying
the file by about 10%. *In this particular situation, buffering
helped speed things up a lot.

[James Kuyper]

Abubakar wrote:

Hi,

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

It's actually backwards, in general. It would be closer to being
accurate to say that they are more efficient because they are buffered
(but event that statement isn't quite true). The purpose of buffering
the data is to take advantage of the fact that, in most cases, it's more
efficient to transfer many bytes at a time. When transferring a large
number of bytes, using unbuffered writes gets the first byte out earlier
than using buffered writes; but gets the last byte out later.

Which approach is better depends upon your application, but in many
contexts the earlier bytes can't (or at least won't) even be used until
after the later bytes have been written, which makes buffered I/O the
clear winner.

Furthermore, C gives you the option of turning off buffering with
setvbuf(). If you do, then the behavior should be quite similar to that
you would get from using system-specific unbuffered I/O functions (such
as write() on Unix-like systems).