looking for statefull snprintf

[dimka]

Hello

Here is my situation: I have a char buf[1024]; that is used as a buffer

for a log output. Normally, I do:

rc = snprintf( buf, sizeof(buf), some_format, some_args );

This makes the formatting my output and then I flush the buffer.
In 90% of cases this works well, exept the cases when the format or the

arguments require more then 1024 bytes of formatted input.
There is no easy way to re-run snprintf() form 'last' position.

Suppose that using of asprintf(), another buffer and other kinds of
malloc() is not an option. Suppose the buffer must be preprocessed
before it hits the actual io.

Is there any open/closed libraries that solve the problem? Are there
any discussions held on the subject? Please provide the references.

Thanks
--
Dima

[Andrew Poelstra]

On 2006-08-17, dimka <di************ @gmail.comwrote :

Hello

Here is my situation: I have a char buf[1024]; that is used as a buffer

for a log output. Normally, I do:

rc = snprintf( buf, sizeof(buf), some_format, some_args );

This makes the formatting my output and then I flush the buffer.
In 90% of cases this works well, exept the cases when the format or the

arguments require more then 1024 bytes of formatted input.
There is no easy way to re-run snprintf() form 'last' position.

Suppose that using of asprintf(), another buffer and other kinds of
malloc() is not an option. Suppose the buffer must be preprocessed
before it hits the actual io.

Is there any open/closed libraries that solve the problem? Are there
any discussions held on the subject? Please provide the references.

I'm not sure about libraries, but you could check out the code for
Chuck Falconers ggets() code (Google), or some of the stuff in my
ioutils package (see my sig for link).

Chuck's code will give you some idea on how to read in "unlimited"
amounts of data, and mine has some minimal input parsing you might
want to check out.

--
Andrew Poelstra <http://www.wpsoftware. net/projects>
To reach me by email, use `apoelstra' at the above domain.
"Do BOTH ends of the cable need to be plugged in?" -Anon.

[Ben Pfaff]

"dimka" <di************ @gmail.comwrite s:

Here is my situation: I have a char buf[1024]; that is used as a buffer
for a log output. Normally, I do:

rc = snprintf( buf, sizeof(buf), some_format, some_args );

This makes the formatting my output and then I flush the buffer.
In 90% of cases this works well, exept the cases when the format or the
arguments require more then 1024 bytes of formatted input.
There is no easy way to re-run snprintf() form 'last' position.

The usual solution is allocate a bigger buffer and try again,
instead of trying to restart from the point where the first try
left off, which as you say is not well supported.

The snprintf function returns the size of the buffer you need
(although you need to add 1 to make room for a terminating null).
--
"...Almost makes you wonder why Heisenberg didn't include postinc/dec operators
in the uncertainty principle. Which of course makes the above equivalent to
Schrodinger's pointer..."
--Anthony McDonald

[Ancient_Hacker]

dimka wrote:

Hello

Here is my situation: I have a char buf[1024]; that is used as a buffer

memory is selling nowdays for about $60 a gigabyte. By my calculations
you're allocating
$0.006144 of memory for this buffer. If you're paid $5 an hour, and
it took you 2.5 minutes to type this message, you've wasted $0.208333
just typing in this message. For that money, you could have bought 30
times the memory you allocated.

Hows about you either:

(1) Bump up the memory buffer size. Assuming it's a local variable,
it's not going to tie up the memory for more than a few microseconds.

(2) Write the data directly to the file with printf(), then there's no
need for the temporary buffer.

(3) Estimate the amount of memory needed beforehand, then add a penny.

[CBFalconer]

Ben Pfaff wrote:

"dimka" <di************ @gmail.comwrite s:

>Here is my situation: I have a char buf[1024]; that is used as a
buffer for a log output. Normally, I do:

rc = snprintf( buf, sizeof(buf), some_format, some_args );

This makes the formatting my output and then I flush the buffer.
In 90% of cases this works well, exept the cases when the format
or the arguments require more then 1024 bytes of formatted input.
There is no easy way to re-run snprintf() form 'last' position.

The usual solution is allocate a bigger buffer and try again,
instead of trying to restart from the point where the first try
left off, which as you say is not well supported.

The snprintf function returns the size of the buffer you need
(although you need to add 1 to make room for a terminating null).

>From N869:

7.19.6.5 The snprintf function

Synopsis

[#1]

#include <stdio.h>
int snprintf(char * restrict s, size_t n,
const char * restrict format, ...);

Description

[#2] The snprintf function is equivalent to fprintf, except
that the output is written into an array (specified by
argument s) rather than to a stream. If n is zero, nothing
is written, and s may be a null pointer. Otherwise, output
characters beyond the n-1st are discarded rather than being
written to the array, and a null character is written at the
end of the characters actually written into the array. If
copying takes place between objects that overlap, the
behavior is undefined.

Returns

[#3] The snprintf function returns the number of characters
that would have been written had n been sufficiently large,
not counting the terminating null character, or a negative
value if an encoding error occurred. Thus, the null-
terminated output has been completely written if and only if
the returned value is nonnegative and less than n.

Taking advantage of this you can pretest for buffer overflow:

templgh = snprintf(NULL, 0, format, whatever);
if (templgh + bufindex >= sizeof(buffer) {
flushbuf(buffer ); bufindex = 0;
}
bufindex += snprintf(buffer[bufindex],
sizeof(buffer) - 1 - bufindex,
format, whatever);

--
"The power of the Executive to cast a man into prison without
formulating any charge known to the law, and particularly to
deny him the judgement of his peers, is in the highest degree
odious and is the foundation of all totalitarian government
whether Nazi or Communist." -- W. Churchill, Nov 21, 1943

[Ben Pfaff]

CBFalconer <cb********@yah oo.comwrites:

Ben Pfaff wrote:

>The usual solution is allocate a bigger buffer and try again,
instead of trying to restart from the point where the first try
left off, which as you say is not well supported.

The snprintf function returns the size of the buffer you need
(although you need to add 1 to make room for a terminating null).

[...]

Taking advantage of this you can pretest for buffer overflow:

templgh = snprintf(NULL, 0, format, whatever);
if (templgh + bufindex >= sizeof(buffer) {
flushbuf(buffer ); bufindex = 0;
}
bufindex += snprintf(buffer[bufindex],
sizeof(buffer) - 1 - bufindex,
format, whatever);

You sure, but it's likely to be faster to call snprintf just once
in the common case (and occasionally a second time), instead of
always calling it twice.

Also, I would make the code above a little smarter, so that it
can expand the buffer in case flushing it isn't sufficient to
make enough room.
--
int main(void){char p[]="ABCDEFGHIJKLM NOPQRSTUVWXYZab cdefghijklmnopq rstuvwxyz.\
\n",*q="kl BIcNBFr.NKEzjwC IxNJC";int i=sizeof p/2;char *strchr();int putchar(\
);while(*q){i+= strchr(p,*q++)-p;if(i>=(int)si zeof p)i-=sizeof p-1;putchar(p[i]\
);}return 0;}

[dimka]

This is funny :) Everyone can cite the standard. But this is EXACTLY
the code that
does NOT work.

>
templgh = snprintf(NULL, 0, format, whatever);
if (templgh + bufindex >= sizeof(buffer) {
flushbuf(buffer ); bufindex = 0;
}
bufindex += snprintf(buffer[bufindex],
sizeof(buffer) - 1 - bufindex,
format, whatever);

[Ben Pfaff]

"dimka" <di************ @gmail.comwrite s:

>templgh = snprintf(NULL, 0, format, whatever);
if (templgh + bufindex >= sizeof(buffer) {
flushbuf(buffer ); bufindex = 0;
}
bufindex += snprintf(buffer[bufindex],
sizeof(buffer) - 1 - bufindex,
format, whatever);

This is funny :) Everyone can cite the standard. But this is EXACTLY
the code that
does NOT work.

Perhaps you are using a pre-standard implementation of snprintf.
Some of these implementations just returned -1 when the string
wouldn't fit in the buffer. Then the caller is left to guess how
much room is needed and try again.
--
Peter Seebach on C99:
"[F]or the most part, features were added, not removed. This sounds
great until you try to carry a full-sized printout of the standard
around for a day."

[dimka]

Well, let me explain the problem again.
Lets assume that the buffer is empty, thus no need to flush it.
then

rc = snprintf( buffer, sizeof(buffer)-1, "%s", str );

Works pretty well. No buffer overrun, in the new standard
rc == strlen(str), in old standard rc == -1, thus I have a way
to check that there was the potential overrun.

The problem arises when strlen(str) sizeof(buffer).
In this situation the buffer will contain trunkated version
of str, I have the information that the string was
trunkated, but no way to say "hey, I have flushed the
buffer, please continue from the place where you
stopped". There must to be an alternative implementation
of snprintf() to do exactly what I want.
Ben Pfaff wrote:

"dimka" <di************ @gmail.comwrite s:

templgh = snprintf(NULL, 0, format, whatever);
if (templgh + bufindex >= sizeof(buffer) {
flushbuf(buffer ); bufindex = 0;
}
bufindex += snprintf(buffer[bufindex],
sizeof(buffer) - 1 - bufindex,
format, whatever);

This is funny :) Everyone can cite the standard. But this is EXACTLY
the code that
does NOT work.

Perhaps you are using a pre-standard implementation of snprintf.
Some of these implementations just returned -1 when the string
wouldn't fit in the buffer. Then the caller is left to guess how
much room is needed and try again.
--
Peter Seebach on C99:
"[F]or the most part, features were added, not removed. This sounds
great until you try to carry a full-sized printout of the standard
around for a day."

[Default User]

dimka wrote:

This is funny :)

Please read the information below.
Brian
--
Please don't top-post. Your replies belong following or interspersed
with properly trimmed quotes. See the majority of other posts in the
newsgroup, or:
<http://www.caliburn.nl/topposting.html >