Inheriting streambuf

[Viktor Lundström]

Hi!
I was planning to wrap a socket inside an iostream, to achieve
something like this:
TCPSocket s(..);
s << "Hello!" << endl;

Information on the web seems to be a bit scarce on how to do this.
I have understood that there is a relation between streambuf and
iostream, where one should be able to extend streambuf and override
the underflow/overflow functions (and do send()/recv() for example).
My concepts are a bit dim.

Here are my questions:
If I inherit streambuf, which functions do I have to override
to achieve buffered IO with arbitrary buffer size?

How do I create an iostream which uses my extended streambuf?

I appreciate all answers.

Viktor Lundström

[Shane Beasley]

Viktor Lundström <d9*****@nada.kth.se> wrote in message news:<Pi*************************************@knat te.nada.kth.se>...

I was planning to wrap a socket inside an iostream, to achieve
something like this:
TCPSocket s(..);
s << "Hello!" << endl;
(Note that sockets are off-topic in this newsgroup, but IOStreams are
dead-on, so I'll help where I can.)

"Socket streams" seem to be of limited use in real programs. Many of
them exist out there -- I've even written one or two -- but what seems
to be a good idea actually turns out to be disastrous.

C++ iostreams are designed to be used with consistent streams of data.
That is, each request for n bytes either retrieves exactly n bytes or
signals an error. There is no provision for a stream which routinely
return less than n bytes, yet that's how TCP operates. Furthermore,
you must account for the possibility that the other end of the socket
is evil or broken. At the very least, getting it right is difficult,
if not impossible. The only foolproof way seems to be to read a single
char at a time, but that (to me) defeats the purpose of using a
stream.

If you wish to continue, you might want to check out Kreft and
Langer's "IOStreams and Locales," a book all about IOStreams (and
locales).
Information on the web seems to be a bit scarce on how to do this.
That's funny, considering how many times it's been done. :) Off the
top of my head:

- Socket++ (http://www.cs.utexas.edu/users/laven...urses/socket++)
is very out of date and usually doesn't compile out of the box, but it
comes with sockbuf and iosockinet classes.

- ACE (http://www.cs.wustl.edu/~schmidt/ACE.html) is very robust,
enterprise caliber, and well supported. It's also bloody huge, but
quite portable. Its ACE_IOStream class is very out of date and in fact
is disabled in the build rules for modern C++ compilers.

- GNU Common C++ (http://www.gnu.org/software/commonc++/) was crap,
at least as of the 1.x release; I started with ACE about when 2.0 came
out. However, it has an ost::tcpstream class.

- Netxx (http://pmade.org/software/netxx/) has its Netxx::Netbuf
class template. I've never used it.
I have understood that there is a relation between streambuf and
iostream, where one should be able to extend streambuf and override
the underflow/overflow functions (and do send()/recv() for example).
My concepts are a bit dim.
I think the important ones are underflow, uflow, overflow, xsgetn,
xsputn, and sync. You can optionally support the locales and
user-defined buffer operations (or you can just use your own internal
buffer -- std::vector<char> or std::string, maybe).
Here are my questions:
If I inherit streambuf, which functions do I have to override
to achieve buffered IO with arbitrary buffer size?
All functions marked "virtual" can be overridden, and any decent C++
reference should give you a list of them. For instance:

<http://dinkumware.com/htm_cpl/streambu.html>

Read the docs, figure out what they do, and override the ones that
apply to your stream.
How do I create an iostream which uses my extended streambuf?

One way:

#include <iostream>

class SockBuf : public std::streambuf {
public:
bool connect (...);
// other socket-related functions here

private: // these are never called directly, so they're private
// override virtual member functions here
};

class SockStream : private SockBuf, public std::iostream {
public:
SockStream () : std::iostream(this) { } // "this" as SockBuf *
SockStream (...) : SockBuf(...), std::iostream(this) { }

using SockBuf::connect;
// other socket-related using declarations here
};

You could technically make both of these into one class; I prefer to
split them up (the "do one thing and do it well" principle).

- Shane

[tom_usenet]

On 24 Sep 2003 23:15:52 -0700, sb******@cs.uic.edu (Shane Beasley)
wrote:

"Socket streams" seem to be of limited use in real programs. Many of
them exist out there -- I've even written one or two -- but what seems
to be a good idea actually turns out to be disastrous.

C++ iostreams are designed to be used with consistent streams of data.
That is, each request for n bytes either retrieves exactly n bytes or
signals an error. There is no provision for a stream which routinely
return less than n bytes, yet that's how TCP operates. Furthermore,
you must account for the possibility that the other end of the socket
is evil or broken. At the very least, getting it right is difficult,
if not impossible. The only foolproof way seems to be to read a single
char at a time, but that (to me) defeats the purpose of using a
stream.

It's fine to use it with sockets as long as you're happy with blocking
IO. IOStreams isn't very good for non-blocking IO though, so
alternative solutions should be sought if that is required. However, I
find blocking IO in a separate thread is generally easiest...

Tom

[Dietmar Kuehl]

Viktor Lundström <d9*****@nada.kth.se> wrote:

Information on the web seems to be a bit scarce on how to do this.
Is it? I'm sure that I have posted at least 10 different stream buffers
to newsgroups, at least 5 of whom were perfectly applicable to your
question (ie. showed the use of the appropriate member functions). Even
though some of this stuff is pretty old by now, it is still applicable.
If I inherit streambuf, which functions do I have to override
to achieve buffered IO with arbitrary buffer size?
The absolute minimum for buffered I/O are:

- 'underflow()' for reading
- 'overflow()' and 'sync()' for writing

To improve performance you might want to also override 'xsgetn()' and
'xsputn()' but I suspect that this is not necessary for a socket
communication: it makes sense when huge blocks of data can be processed
more efficiently than smaller blocks. I don't think this is the case
for sockets although it may be possible to avoid copying the bytes one
time.

What you have to do in your stream buffer for sockets is pretty simple:

- You have to create the socket itself, probably in the constructor or
even earlier from another class.
- You have to create input and output buffers. These are probably best
allocated in the constructor. Initialization of the stream buffer to
become aware of the get buffer is a natural thing to be done in
'underflow()'. Eg.:

int socketbuf::s_bufsize const = 1024;
socketbuf::socketbuf(int fd):
m_fd(fd), m_in(new char[s_bufsize]), m_out(new char[s_bufsize])
{
// set up the output buffer to leave at least one space empty:
setp(m_out, m_out + s_bufsize - 1);
}
socketbuf::~socketbuf() { delete[] m_in; delete[] m_out; }

- 'overflow()' is called when the buffer is full and a character is
attempted to be put into the buffer. In addition, it may be called
with an argument of 'traits_type::eof()', indicating that the buffer
should be flushed. Since there is a one character space left, the
passed character is put there and 'sync()' is called:

socketbuf::int_type socketbuf::overflow(int_type c) {
if (!traits_type::eq_int_type(traits_type::eof(), c)) {
traits_type::assign(*pptr(), traits_type::to_char_type(c));
pbump(1);
}
return sync() == 0? traits_type::not_eof(c): traits_type::eof();
}

- 'sync()' is called to bring the internal and the external stream
into synchronization. This effectively means that the output buffer
is to be flushed:

int socketbuf::sync() {
return pbase() == pptr() ||
write(m_fd, pbase(), pptr() - pbase()) == pptr() - base()? 0: 1;
}

- 'underflow()' is called when a character is to be read but none is
available. It is supposed to fill a buffer (if no buffer is used for
input, you need to implement 'uflow()', too). A real implementation
will move a few characters of the previous buffer to the front of
the new buffer to always allow put back (this is not done here for
brevity):

socketbuf::int_type socketbuf::underflow() {
int rc = read(m_fd, m_in, s_bufsize);
if (rc <= 0)
return traits_type::eof();
setg(m_in, m_in, m_in + rc);
return traits_type::to_int_type(*gptr());
}

The sequence between the first parameter to 'setg()' and the second
parameter is the "put back area" (in this case empty): if you retain
characters for put back, you start reading beginning at the second
parameter.

That's it (... and all of this I have said before, although with
different words, in articles I have posted in the past).
How do I create an iostream which uses my extended streambuf?

The stream classes and 'std::ios' have a constructor taking a pointer
to 'std::streambuf'. So you can simply use these constructors to
create an appropriate stream, eg.:

socketbuf socketbuf(open_socket_somehow());
std::istream socketin(&socketbuf);

Typically, construction of the stream buffer is encapsulated into
a class derived from 'std::istream', 'std::ostream', or
'std::iostream'. I have shown this technique in several articles I
have posted in the past...
--
<mailto:di***********@yahoo.com> <http://www.dietmar-kuehl.de/>
Phaidros eaSE - Easy Software Engineering: <http://www.phaidros.com/>