fwrite output is not efficient (fast) ?

[Abubakar]

Hi,

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

regards,

...ab

[James Kuyper]

Abubakar wrote:

well he is saying that using the *native* read and write to do the
same task using file descriptors is much faster than the fwrite etc.
He says using the FILE * that is used in case of the fopen/fwrite
buffers data and is not as fast and predictable as read write are.

fwrite() buffers data only if you don't tell it not to, by calling
setvbuf(). When talking about buffered I/O, he's right about the
predictability, if he's talking about predicting when data gets written
to the file. That depends upon the details of the buffering scheme,
which are in general unknown to the user.

However, what he's saying about speed is true only if he's mainly
concerned with the speed with which the first byte reaches a file. If
he's concerned with the speed with which the last byte reaches the file,
buffering is generally faster, at least with sufficiently large files.

[Richard Bos]

James Kuyper <ja*********@ve rizon.netwrote:

Abubakar wrote:

well he is saying that using the *native* read and write to do the
same task using file descriptors is much faster than the fwrite etc.
He says using the FILE * that is used in case of the fopen/fwrite
buffers data and is not as fast and predictable as read write are.

fwrite() buffers data only if you don't tell it not to, by calling
setvbuf(). When talking about buffered I/O, he's right about the
predictability, if he's talking about predicting when data gets written
to the file.

Not even that. The OS may have its own buffers, and so may the drive
firmware. Bottom line, if you want absolute file security, you have to
nail it down to the hardware level. If you don't need that, 99+% of the
time, ISO C <stdio.hfunctio ns are good enough.

Richard

[Richard Heathfield]

Abubakar said:

Well he read the link and now he says that the replies by Nate
Eldredge and the guy whose email starts with "s0s" have said what
proves what he says is right

They have both argued the opposite point of view. They may or may not have
proved /their/ case, but they certainly haven't proved /his/.

so he says there is no need to reply to anything.

Of course not. If he wishes to continue in blissful ignorance, that's his
choice! :-)

--
Richard Heathfield <http://www.cpax.org.uk >
Email: -http://www. +rjh@
Google users: <http://www.cpax.org.uk/prg/writings/googly.php>
"Usenet is a strange place" - dmr 29 July 1999

[William Pursell]

On 24 Oct, 08:03, Abubakar <abubak...@gmai l.comwrote:

recently some C programmer told me that using fwrite/fopen functions
are not efficient because the output that they do to the file is
actually buffered and gets late in writing. Is that true?

No. Sometimes using fwrite() is less efficient than using
(the non-standard function) write(), sometimes it is
more efficient. For example:

fwrite( buf, 1, 8192, fp );

is likely to be less efficient than

write( fd, buf, 8192 );

but

for( i = 0; i < 8192; i++ )
fwrite( buf + i, 1, 1, fp );

is likely to be more efficient than

for( i = 0; i < 8192; i++ )
write( fd, buf + i, 1 );

In the loop, fwrite() is likely to be more efficient
**because of** the buffering. In the first example, fwrite()
is likely to be less efficient because there is (usually)
an extra data move that doesn't occur with write().

However, the portability of fwrite() is a significant
feature to be considered, as is its ease of use. Unless
you have a verified, measured need to replace fwrite()
with write(), don't bother.

[CBFalconer]

Abubakar wrote:

>
well he is saying that using the *native* read and write to do the
same task using file descriptors is much faster than the fwrite etc.
He says using the FILE * that is used in case of the fopen/fwrite
buffers data and is not as fast and predictable as read write are.

I have given him the link to this discussion and he is going to be
posting his replies soon hopefully.

Please do not top-post. Your answer belongs after (or intermixed
with) the quoted material to which you reply, after snipping all
irrelevant material. Your top-posting has lost all continuity from
the thread. See the following links:

<http://www.catb.org/~esr/faqs/smart-questions.html>
<http://www.caliburn.nl/topposting.html >
<http://www.netmeister. org/news/learn2quote.htm l>
<http://cfaj.freeshell. org/google/ (taming google)
<http://members.fortune city.com/nnqweb/ (newusers)

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home .att.net>
Try the download section.

[Malcolm McLean]

"Abubakar" <ab*******@gmai l.comwrote in message news:

>well he is saying that using the *native* read and write to do the
same task using file descriptors is much faster than the fwrite etc.
He says using the FILE * that is used in case of the fopen/fwrite
buffers data and is not as fast and predictable as read write are.

He's right. fwrite() will call the native write functions, which will call
device drivers and the like. As you go down through the levels closer to the
actual hardware, there is more potential for cutting out unnecessary
operations and making things faster. However only if you know what you are
doing (or maybe if the implementation is very slack).
When you move to a new machine it is likely that the optimisations will
become sub-optimal, or the code might not work at all.

Nowadays it is very seldom worth the extra effort and loss of portability in
not calling fwrite(). However that wasn't always true. General-purpose PCs
used to be slower by a factor of 1000 than modern machines, and then you had
to squeeze every last drop of performance out of the machine to make your
games run fast enough.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

[Bartc]

"Malcolm McLean" <re*******@btin ternet.comwrote in message
news:aI******** *************** *******@bt.com. ..

>
"Abubakar" <ab*******@gmai l.comwrote in message news:

>>well he is saying that using the *native* read and write to do the
same task using file descriptors is much faster than the fwrite etc.
He says using the FILE * that is used in case of the fopen/fwrite
buffers data and is not as fast and predictable as read write are.

Nowadays it is very seldom worth the extra effort and loss of portability
in not calling fwrite(). However that wasn't always true. General-purpose
PCs used to be slower by a factor of 1000 than modern machines, and then
you had to squeeze every last drop of performance out of the machine to
make your games run fast enough.

But sometimes modern machines are being asked to do 1000 times more work.
Performance can still be an issue.

--
Bartc

[Malcolm McLean]

"Bartc" <[e-mail] rote in message

But sometimes modern machines are being asked to do 1000 times more work.
Performance can still be an issue.

But disk IO is less likely to be the bottleneck. My programs take up to two
weeks to run on as many processors as I can lay my hands on, however they
only read and write a few kilobytes of data.

--
Free games and programming goodies.
http://www.personal.leeds.ac.uk/~bgy1mm

[Flash Gordon]

Malcolm McLean wrote, On 25/10/08 16:46:

>
"Bartc" <[e-mail] rote in message

>But sometimes modern machines are being asked to do 1000 times more
work. Performance can still be an issue.

But disk IO is less likely to be the bottleneck.

Except when it is. You know absolutely nothing about the type of program
this might be used for and some programs are definitely disk bound even
with the fastest disk sub-systems.

My programs take up to
two weeks to run on as many processors as I can lay my hands on, however
they only read and write a few kilobytes of data.

So yours are not IO bound, that says nothing about the situations the OP
is concerned with.
--
Flash Gordon
If spamming me sent it to sm**@spam.cause way.com
If emailing me use my reply-to address
See the comp.lang.c Wiki hosted by me at http://clc-wiki.net/

[Phil Carmody]

"Malcolm McLean" <re*******@btin ternet.comwrite s:

"Bartc" <[e-mail] rote in message

>But sometimes modern machines are being asked to do 1000 times more
work. Performance can still be an issue.

But disk IO is less likely to be the bottleneck. My programs take up
to two weeks to run on as many processors as I can lay my hands on,
however they only read and write a few kilobytes of data.

If processors increase in speed at a faster rate than
I/O bandwidth does, which happens to be the case, then
I/O can do nothing apart from become more of a bottle-
neck. If you, like me, run /embarassingly parallel/ code,
then more times nothing is nothing, but we're in a very
fortunate minority.

Phil
--
The fact that a believer is happier than a sceptic is no more to the
point than the fact that a drunken man is happier than a sober one.
The happiness of credulity is a cheap and dangerous quality.
-- George Bernard Shaw (1856-1950), Preface to Androcles and the Lion