Uses of setvbuf()

[santosh]

Which situations call for the use of setvbuf(), specifically when you
supply the buffer yourself? What is the advantage of an user specified
buffer over an automatically allocated one? Can we call setvbuf() on
the predefined streams, immediatly upon start of main()? How often is
this function used in typical C programs?

Thanks.

[Eric Sosman]

santosh wrote On 01/05/07 11:46,:

Which situations call for the use of setvbuf(), specifically when you
supply the buffer yourself? What is the advantage of an user specified
buffer over an automatically allocated one? Can we call setvbuf() on
the predefined streams, immediatly upon start of main()? How often is
this function used in typical C programs?

Probably the commonest use of setvbuf() or setbuf() is
to make an output stream line-buffered or unbuffered. The
size of the supplied buffer (if any) isn't much of an issue
in this scenario.

For input or output streams that pass "a lot" of data
and can operate fully-buffered, specifying a "large" buffer
may improve the I/O throughput. How much improvement you
get depends on the system, the C implementation, the program's
I/O behavior, and a lot of other situation-specific details;
the meanings of "a lot" and "large" are also hard to pin down.
Bigger is not always better; "too large" a buffer may actually
hurt performance in some circumstances.

You can apply setvbuf() or setbuf() to any stream, if
you do it soon enough. There's nothing special about the
predefined streams in this regard.

Finally, these functions are really only "suggestion s"
to the I/O library, and the library is at liberty to modify
them or even to ignore them. Implementations differ, and
the implementor (for some strange reason) often believes he
has more knowledge of the underpinnings of the implementation
than does the programmer ...

--
Er*********@sun .com

[santosh]

Eric Sosman wrote:

santosh wrote On 01/05/07 11:46,:

Which situations call for the use of setvbuf(), specifically when you
supply the buffer yourself? What is the advantage of an user specified
buffer over an automatically allocated one? Can we call setvbuf() on
the predefined streams, immediatly upon start of main()? How often is
this function used in typical C programs?

<snip explanation>

Finally, these functions are really only "suggestion s"
to the I/O library, and the library is at liberty to modify
them or even to ignore them.

I can understand ignoring a call by returning a non-zero value but
won't modifying silently the semantics of a call mislead the calling
code? It seems more in keeping with the spirit of C to either honour a
request or to fail.

Anway thanks for your explanation.

Implementations differ, and
the implementor (for some strange reason) often believes he
has more knowledge of the underpinnings of the implementation
than does the programmer ...

In most cases they probably do.

[user923005]

santosh wrote:

Which situations call for the use of setvbuf(), specifically when you
supply the buffer yourself? What is the advantage of an user specified
buffer over an automatically allocated one? Can we call setvbuf() on
the predefined streams, immediatly upon start of main()? How often is
this function used in typical C programs?

Usually, when I call setvbuf() it is to turn off buffering for stdin
and/or stdout. An example of when this is needed is for Winboard
protocol chess engines which exchange text command streams with a chess
protocol server. If the I/O is buffered, a command may not get flushed
and the engine can stall for no apparent reason.

Sometimes, I want to increase the size of the buffer. Sequentially
scanning a file is an example where this can be handy. However, the
buffer size allowed is so tiny that I usually resort to OS specific
routines instead, because the tiny buffers allowed by setvbuf() do not
afford any big boost.

[santosh]

user923005 wrote:

santosh wrote:

Which situations call for the use of setvbuf(), specifically when you
supply the buffer yourself? What is the advantage of an user specified
buffer over an automatically allocated one? Can we call setvbuf() on
the predefined streams, immediatly upon start of main()? How often is
this function used in typical C programs?

Usually, when I call setvbuf() it is to turn off buffering for stdin
and/or stdout.

Turning off buffering for stdin in setvbuf() seems to accomplish little
since it's overruled by the OS's buffering. But I can see that it might
be useful for output and non-interactive input streams.

<snip>

Sometimes, I want to increase the size of the buffer. Sequentially
scanning a file is an example where this can be handy. However, the
buffer size allowed is so tiny that I usually resort to OS specific
routines instead, because the tiny buffers allowed by setvbuf() do not
afford any big boost.

Correct me if I'm wrong but I don't see any limit to the buffer's
allowed size. The value of BUFSIZ is recommended but I don't see why
only tiny buffers are allowed.

[Eric Sosman]

santosh wrote On 01/05/07 12:26,:

Eric Sosman wrote:

>>santosh wrote On 01/05/07 11:46,:

>>>Which situations call for the use of setvbuf(), [...]

Finally, these functions are really only "suggestion s"
to the I/O library, and the library is at liberty to modify
them or even to ignore them.

I can understand ignoring a call by returning a non-zero value but
won't modifying silently the semantics of a call mislead the calling
code? It seems more in keeping with the spirit of C to either honour a
request or to fail.

The older setbuf() function has no way to "fail," since
it is void and can return no value.

setvbuf() returns a success/failure indication, but what
constitutes "failure" seems not to be very well defined. For
example, is it a "failure" if you provide a 32K buffer for an
output stream, and the I/O library happily uses the buffer but
always drains it when 1K characters have accumulated?

I/O systems must bridge a speed gap of seven or so decimal
orders of magnitude, a gap that continues to widen. They also
deal with an ever-changing suite of device characteristics :
streaming devices, multi-pathed devices, devices that appear
and disappear dynamically, devices that require cryptographic
challenge-and-response, ... The thicket of problems faced by
an I/O system is dense, and the tools developed to deal with
it (by cutting, burning, spraying herbicide, going around the
edge, ...) are diverse to the point of being baroque. Atop
this Babel of complexity sits the C library with its simple
model of stream I/O -- another impedance mismatch to be dealt
with. The implementor needs all the freedom the Standard can
grant.

--
Er*********@sun .com

[user923005]

santosh wrote:

user923005 wrote:

santosh wrote:

Which situations call for the use of setvbuf(), specifically when you
supply the buffer yourself? What is the advantage of an user specified
buffer over an automatically allocated one? Can we call setvbuf() on
the predefined streams, immediatly upon start of main()? How often is
this function used in typical C programs?

Usually, when I call setvbuf() it is to turn off buffering for stdin
and/or stdout.

Turning off buffering for stdin in setvbuf() seems to accomplish little
since it's overruled by the OS's buffering. But I can see that it might
be useful for output and non-interactive input streams.

If you don't call it, then the chess engines don't work. If you do
call it then they do. So I would not call that useless. If there were
some compiler that did not honor my buffering request, then I would
switch compilers.

7.19.5.6 The setvbuf function
Synopsis
1 #include <stdio.h>
int setvbuf(FILE * restrict stream,
char * restrict buf,
int mode, size_t size);
Description
2 The setvbuf function may be used only after the stream pointed to by
stream has
been associated with an open file and before any other operation (other
than an
unsuccessful call to setvbuf) is performed on the stream. The argument
mode
determines how stream will be buffered, as follows: _IOFBF causes
input/output to be
fully buffered; _IOLBF causes input/output to be line buffered; _IONBF
causes
input/output to be unbuffered. If buf is not a null pointer, the array
it points to may be
used instead of a buffer allocated by the setvbuf function (see
footnote 230) and the argument size
specifies the size of the array; otherwise, size may determine the size
of a buffer
allocated by the setvbuf function. The contents of the array at any
time are
indeterminate.
Returns
3 The setvbuf function returns zero on success, or nonzero if an
invalid value is given
for mode or if the request cannot be honored.

footnote 230) The buffer has to have a lifetime at least as great as
the open stream, so the stream should be closed
before a buffer that has automatic storage duration is deallocated upon
block exit.

<snip>

Sometimes, I want to increase the size of the buffer. Sequentially
scanning a file is an example where this can be handy. However, the
buffer size allowed is so tiny that I usually resort to OS specific
routines instead, because the tiny buffers allowed by setvbuf() do not
afford any big boost.

Correct me if I'm wrong but I don't see any limit to the buffer's
allowed size. The value of BUFSIZ is recommended but I don't see why
only tiny buffers are allowed.

The thing could theoretically be huge. But Microsoft set a tiny limit
on it (16K or something nearly as absurd if I recall correctly) so I
just write system specific routines when I want to buffer since I am
going to have to do it anyway.

[user923005]

user923005 wrote:

The thing could theoretically be huge. But Microsoft set a tiny limit
on it (16K or something nearly as absurd if I recall correctly) so I
just write system specific routines when I want to buffer since I am
going to have to do it anyway.

From:
http://msdn.microsoft.com/library/de...rt_setvbuf.asp
We have this:

"size
Buffer size in bytes. Allowable range: 2 < size < 32768. Internally,
the value supplied for size is rounded down to the nearest multiple of
2."

Which has caused me a wee bit of grief from time to time.

[aegis]

santosh wrote:

Which situations call for the use of setvbuf(), specifically when you
supply the buffer yourself? What is the advantage of an user specified
buffer over an automatically allocated one? Can we call setvbuf() on
the predefined streams, immediatly upon start of main()? How often is
this function used in typical C programs?

Thanks.

Any situation that calls for change in the characteristics of streams.
Keep in mind though, there are different levels of input/output
typically.

--
aegis

[Al Balmer]

On 5 Jan 2007 08:46:43 -0800, "santosh" <sa*********@gm ail.comwrote:

>Which situations call for the use of setvbuf(), specifically when you
supply the buffer yourself? What is the advantage of an user specified
buffer over an automatically allocated one? Can we call setvbuf() on
the predefined streams, immediatly upon start of main()? How often is
this function used in typical C programs?

Thanks.

As Eric says, controlling the buffering is probably the most common
use. I've found (and here we go off-topic for the group) that in some
implementations , specifying a buffer can give better performance if
you are opening and closing many files with the same file pointer,
apparently because it saves the system allocating a new buffer every
time. In fact, this was the workaround for a bug in one version of
HP-UX, which allocated buffers much faster than it released them,
causing a program to run out of memory. A fun one to track down, since
the error was at the system level, and the program hadn't a clue as to
why it was suddenly kicked out :-)

--
Al Balmer
Sun City, AZ