By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
454,484 Members | 1,553 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 454,484 IT Pros & Developers. It's quick & easy.

Expanding buffer in C

P: n/a
Do you remember a previous discussion on this newsgroup as at

http://groups.google.com/group/comp....738c4f91113b4e

It concerned code to manage a buffer, expanding it as needed semi
automatically. I've incorporated suggestions people made and published
the code at

http://codewiki.wikispaces.com/xbuf.c

(For the impatient check the last two links in the table of contents
for the code. The rest is documentation.)

The code is intended to be fast. It's uses include reading an
arbitrary-length line from an input stream - a safe gets() replacement
if you like - but it is more flexible than line reading functions and
is intended to be useful in more cases than just line reading.

--NB. Code and documentation are on a wiki. If you see things that
should be changed feel free to change them.

Apart from the code itself what do you think of the concept of
publishing it on a wiki...?

James
Nov 16 '08 #1
Share this Question
Share on Google+
3 Replies


P: n/a
On Nov 15, 8:12*pm, James Harris <james.harri...@googlemail.com>
wrote:
Do you remember a previous discussion on this newsgroup as at

*http://groups.google.com/group/comp....hread/cb502ce8....

It concerned code to manage a buffer, expanding it as needed semi
automatically. I've incorporated suggestions people made and published
the code at

*http://codewiki.wikispaces.com/xbuf.c

(For the impatient check the last two links in the table of contents
for the code. The rest is documentation.)

The code is intended to be fast. It's uses include reading an
arbitrary-length line from an input stream - a safe gets() replacement
if you like - but it is more flexible than line reading functions and
is intended to be useful in more cases than just line reading.

--NB. Code and documentation are on a wiki. If you see things that
should be changed feel free to change them.

Apart from the code itself what do you think of the concept of
publishing it on a wiki...?

James
Nice presentation. You'll probably get less heap fragmentation for
programs that use lots of buffers if all of them are at set standard
sizes. So your new_size computation would become something like

new_size = current_size;
while (new_size < requested_size)
new_size = new_size * FACTOR;

NB. Your choice of default 3 / 2 for FACTOR is interesting. I've
been using this for many years because the standard 2 seemed
extravagant.
Nov 17 '08 #2

P: n/a
On 18 Nov, 17:53, Gene <gene.ress...@gmail.comwrote:
....
Is there a better way to increase the buffer size? Especially as the
address space fills up should the increase factor be reduced, and how?
Anyone been down this road?

Garbage collectors deal with this when they need to increase the pool
size when not much VM is left. There isn't a single best policy
because behavior depends heavily on the OS.
Agreed. The more I think about this the less I think there is any kind
of one-size-fits-all solution. One advantage of distributing as source
code is that people can vary the code to suit their situation. I've
kept the original code as it was but have documented some alternatives
as comments within the code.

http://codewiki.wikispaces.com/xbuf.c

The commented alternatives are purely illustrative and are untested. I
hope I've got the C syntax etc correct. Either way they should be
enough to illustrate the options to anyone using the code. If you,
dear reader, see an error feel free to fix it or let me know what I've
got wrong and I'll fix it.

James
Nov 21 '08 #3

P: n/a
James Harris wrote:
Gene <gene.ress...@gmail.comwrote:
.... snip ...
>
>Garbage collectors deal with this when they need to increase the
pool size when not much VM is left. There isn't a single best
policy because behavior depends heavily on the OS.

Agreed. The more I think about this the less I think there is any
kind of one-size-fits-all solution. One advantage of distributing
as source code is that people can vary the code to suit their
situation. I've kept the original code as it was but have
documented some alternatives as comments within the code.
In general you should adjust the mechanism to fit the expected
use. As an example, my ggets routine expands the buffer in
constant increments (of 128), because its primary use is expected
to be interactive input, and those strings are not normally
unlimited. However my hashlib expands its table by a factor of two
each time, resulting in constant average overhead, and basically
eliminating consideration of table size. In each case there are
possible penalties.

--
[mail]: Chuck F (cbfalconer at maineline dot net)
[page]: <http://cbfalconer.home.att.net>
Try the download section.
Nov 22 '08 #4

This discussion thread is closed

Replies have been disabled for this discussion.