469,643 Members | 2,064 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,643 developers. It's quick & easy.

std::istream::readsome


What I would like to do is read bytes from a stream, any number and any
time. I would like it to wait until there are any bytes to read.

I want the exact same functionality as cstdio's "fread" but in a
std::istream.

It appeared at first that "readsome" would do exactly what I wanted but
it appears not to work very well at all (at least with gcc!). When
reading from a named pipe on gcc, it returns immediately - no errors,
just immediate return, is that the expected behaviour ?

Oct 28 '06 #1
13 10283
Hello Gianni!
Gianni Mariani wrote:
What I would like to do is read bytes from a stream, any number and any
time. I would like it to wait until there are any bytes to read.
There is no such functionality - neither in the C library no in the C++
library.
At least, these libraries are not required to do anything like this.
I want the exact same functionality as cstdio's "fread" but in a
std::istream.
You can use std::istream::read() for this. This will, however, block
until
either end of the input stream is reached or the requested number of
characters is received - as is the case for fread(), of course. If the
fread() on your system returns differently than described above, it is
either wrong or you are not using it on files. In the latter situation,
you
are relying on system specific features and the C or C++ standards
make no requirements on the behavior of the calls whatsoever.
It appeared at first that "readsome" would do exactly what I wanted
This is rather unlikely: the default implementation of readsome() just
obtains characters which are in the buffer of the underlying stream
buffer.
If there are none, it just returns, indicating that it read no
characers. The
default implementation does never attempt to read any bytes. Classes
derived from std::streambuf can choose to use a different approach but
none of the standard stream buffers does.
When
reading from a named pipe on gcc, it returns immediately - no errors,
just immediate return, is that the expected behaviour ?
Yes: there are no characters, i.e. there is nothing to do. You might
want to check gcount() for the number of characters read.

Good luck, Denise.

Oct 29 '06 #2
Denise Kleingeist wrote:
Hello Gianni!
Gianni Mariani wrote:
>What I would like to do is read bytes from a stream, any number and any
time. I would like it to wait until there are any bytes to read.

There is no such functionality - neither in the C library no in the C++
library.
I beg to differ. std::fread does exactly what I want.
At least, these libraries are not required to do anything like this.
These libraries have been performing exactly this way since I wrote my
first C program over 25 years ago. I have no idea what you talk about.
>
>I want the exact same functionality as cstdio's "fread" but in a
std::istream.

You can use std::istream::read() for this. This will, however, block
until
either end of the input stream is reached or the requested number of
characters is received - as is the case for fread(),
Unfortunately, this is not the case, a call to std::istream::read will
fail if there is not enough data to fulfill the read request. There is
no way for the istream::read call interface to indicate that it has read
fewer than the requested number of bytes so the last block read will
almost always be unreadable.

of course. If the
fread() on your system returns differently than described above, it is
std::fread does operate exactly as I want. The size of the last block
being read is returned. I want to do the same thing with std::istream.

.... snip
>It appeared at first that "readsome" would do exactly what I wanted

This is rather unlikely: the default implementation of readsome() just
obtains characters which are in the buffer of the underlying stream
buffer.
If there are none, it just returns, indicating that it read no
characers. The
default implementation does never attempt to read any bytes. Classes
derived from std::streambuf can choose to use a different approach but
none of the standard stream buffers does.
That's strange since at least 2 compilers I use do exactly the opposite
of what you say, on regular files no less.
>
>When
reading from a named pipe on gcc, it returns immediately - no errors,
just immediate return, is that the expected behaviour ?

Yes: there are no characters, i.e. there is nothing to do. You might
want to check gcount() for the number of characters read.
OK - you have affirmed why keep away from iostreams for C++ io. It's
not only I who can't figure out how to use it properly.
Oct 29 '06 #3

Gianni Mariani wrote in message ...
>
It appeared at first that "readsome" would do exactly what I wanted but
it appears not to work very well at all
Yeah, know what you mean. I used to readsome, then codesome. It was always
the codesome that would "Git 'er done!".

--
Bob <GR
POVrookie
Oct 29 '06 #4
Hello Gianni!
Gianni Mariani wrote:
Denise Kleingeist wrote:
Gianni Mariani wrote:
What I would like to do is read bytes from a stream, any number and any
time. I would like it to wait until there are any bytes to read.
There is no such functionality - neither in the C library no in the C++
library.

I beg to differ. std::fread does exactly what I want.
OK. I'm assuming I know what you want and I'm certain that this doesn't
work
in either C or C++. Since you disagree, apparently my assumption is
wrong.
Now, lets verify or falsify my assumptions: from the description given
and the
mention of pipes, I assume you want to obtain whatever amount of data
is
available from your source as soon as it becomes available. For
example, if
you had read all available character, you want to wait until new
characters
become available. Say you are using a buffer of 64 characters and 25
become
available, you want the read function to return and report that 25
characters
have become available.

Functionality like this can be implemented on many systems supporting C
or
C++ but neither language provides the necessary facilities to set this
up: you
need to resort to platform specific functionality to do so.
At least, these libraries are not required to do anything like this.

These libraries have been performing exactly this way since I wrote my
first C program over 25 years. I have no idea what you talk about.
I'm talking about C and C++ standards and what I assume you want to do.
Neither C nor C++ offer functionality which reads characters and
returns
whatever is there without hitting EOF or an error. Both standards
define their
I/O operations to either read precisely the number of characters
specified in
their read operations, hit an error, or reach EOF. Neither standard
defines any form of reads which wait until there are characters
available and
just return those. ... unless, of course, EOF is reached or an error is
encountered.
If you disagree, please point me at the section of the C or C++
standard which
states otherwise. For C, I'm looking at 7.19.8.1 (The fread function),
for C++ at
27.6.1.3 (lib.istream.unformatted) paragraph 30.
I want the exact same functionality as cstdio's "fread" but in a
std::istream.
You can use std::istream::read() for this. This will, however, block
until
either end of the input stream is reached or the requested number of
characters is received - as is the case for fread(),

Unfortunately, this is not the case, a call to std::istream::read will
fail if there is not enough data to fulfill the read request. There is
no way for the istream::read call interface to indicate that it has read
fewer than the requested number of bytes so the last block read will
almost always be unreadable.
There is a way to find out how many characters std::istream::read() has
read:
std::istream::gcount() provides the number of characters read by the
last
unformatted input function (27.6.1.3, lib.istream.unformatted,
paragraph 1).
of course. If the
fread() on your system returns differently than described above, it is

std::fread does operate exactly as I want. The size of the last block
being read is returned. I want to do the same thing with std::istream.
If you want a function which returns the number of characters read
rather
than using std::istream::gcount() to determine this number, you can use
std::streambuf::sgetn(), e.g.:

std::streamsize n = std::cin.rdbuf()->sgetn(buffer, size);

(see 27.5.2.2.3, lib.streambuf.get.area, paragraph 6, and 27.4.4.2,
lib.basic.ios.members, paragraph 4).
... snip
It appeared at first that "readsome" would do exactly what I wanted
This is rather unlikely: the default implementation of readsome() just
obtains characters which are in the buffer of the underlying stream
buffer.
If there are none, it just returns, indicating that it read no
characers. The
default implementation does never attempt to read any bytes. Classes
derived from std::streambuf can choose to use a different approach but
none of the standard stream buffers does.

That's strange since at least 2 compilers I use do exactly the opposite
of what you say, on regular files no less.
Well, maybe I should have said that std::filebuf is not required to
provide a
different than the default implementation of std::streambuf::in_avail()
which
is the function used to determine the maximum number of characters
available to std::istream::readsome(). In my experience, std::filebuf
does
not provide a different implementation than the default and thus
readsome()
which just read the characters currently stuck in the buffer. Since
your
impression was that readsome() didn't work for gcc, I'd guess that at
least
libstdc++ uses does not override this function.
When
reading from a named pipe on gcc, it returns immediately - no errors,
just immediate return, is that the expected behaviour ?
Yes: there are no characters, i.e. there is nothing to do. You might
want to check gcount() for the number of characters read.

OK - you have affirmed why keep away from iostreams for C++ io. It's
not only I who can't figure out how to use it properly.
I certainly know how to use it properly! ... and I'm certainly not
using stdio
in any code I write although I know that library better than most
others, too.
stdio has too many problems, IMO.

Good luck, Denise.

Oct 29 '06 #5

Hi Denise, I know you're trying to help so I appreciate that but I
don't think you really understand the full extent of the question. Also
the format of your text with the uneven breaks in the lines makes it
hard to read on my newsreader.

Let me rephrase the question.

I want to read all the data from a file until end of file a chunk at a
time using std::istream.

Sounds simple enough right ?

Well there does not seem to be any way to do this easily.

std::istream::read is incapable of reading the last incomplete block.

std::istream::readsome looks like the right answer, but alas it does not
wait for data from pipes.

So, how do you do it ? Read just the bytes from the file block by block
to an uneven block end for any kind of file (including pipes) in a
standard C++ way.
Oct 29 '06 #6
Hi Gianni!
Gianni Mariani wrote:
Hi Denise, I know you're trying to help so I appreciate that but I
don't think you really understand the full extent of the question.
This could indeed be the problem...
Also
the format of your text with the uneven breaks in the lines makes it
hard to read on my newsreader.
I'm sorry about this: I'm using Google Groups to post and
this keeps messing up my formatting.
Let me rephrase the question.
Precisions is always good: no guessing this time :-)
I want to read all the data from a file until end of file a chunk at a
time using std::istream.

Sounds simple enough right ?
That is simple, indeed! Where is your problem then? Here is
the code to do it (plus writing the result to standard out;
of course, you want to add error some error handling on the
program arguments and opening the file):

#include <fstream>
#include <iostream>

int main(int ac, char *av[])
{
std::ifstream in(av[1]);
char buffer[64];
while (in.read(buffer, sizeof(buffer)))
std::cout.write(buffer, sizeof(buffer));
// process the final chunk:
std::cout.write(buffer, in.gcount());
}
Well there does not seem to be any way to do this easily.
Doesn't look too complicated to me...
std::istream::read is incapable of reading the last incomplete block.
Of course, it is not! That would be pretty lame!
std::istream::readsome looks like the right answer, but alas it does not
wait for data from pipes.
std::istream::readsome() is intended to do a non-blocking
read of data which is guaranteed to be available at the time
of the call. Basically, the only data which can normally be
guaranteed is the data in the stream buffer's internal buffer
and the default implementation of the associated functionality
(the stream buffer method in_avail()) just looks at the stream
buffer's buffer. Classes derived from std::streambuf can
override in_avail() to do a better job but few classes really
do. However, this is not what you want for the question above.
So, how do you do it ? Read just the bytes from the file block by block
to an uneven block end for any kind of file (including pipes) in a
standard C++ way.
Of course, the C++ way to do the code I wrote above looks
more like the code below :-)

#include <fstream>
#include <iostream>
int main(int ac, char* av[])
{
std::cout << std::ifstream(av[1]).rdbuf();
}

Good luck, Denise.

Oct 29 '06 #7
Denise Kleingeist wrote:
....
Of course, the C++ way to do the code I wrote above looks
more like the code below :-)

#include <fstream>
#include <iostream>
int main(int ac, char* av[])
{
std::cout << std::ifstream(av[1]).rdbuf();
}
OK, now I get it. Bypass all the nonsense and get to the streambuf.

#include <iostream>

using namespace std;

int main ()
{

streambuf * pbuf = std::cin.rdbuf();
char l_buffer[128];

streamsize i;

while ( i = pbuf->sgetn( l_buffer, sizeof(l_buffer) ) )
{
std::cout.write( l_buffer, i );
std::cout.flush();
}

return 0;
}

This seems to behave like I expect it to.

The only last thing I would like is to be able to covert a non binary
file to a "std::ios::binary" file without closing it. The program reads
input from cin (optionally) and after reading the first few bytes it
determines if the file is a "binary" style file and then starts reading
in binary. The program currently reads past "\r\n" or "\n" or \n" and
"does the right thing". So it would be nice to be able to get a binary
handle on std::cin in particular.

G
Oct 29 '06 #8
Hello Gianni!
Gianni Mariani wrote:
OK, now I get it. Bypass all the nonsense and get to the streambuf.
That certainly wasn't what I was saying but there are many
approaches to obtain the desired goal.
The only last thing I would like is to be able to covert a non binary
file to a "std::ios::binary" file without closing it.
There is no way to do something like this with the standard
I/O function, neither in C nor in C++. I guess, that is part
of the reason why these flags are called "openmode" rather
than "streammode" or something like this. If you are on a
UNIX system there is nothing to do anyway: the file modes
simply don't differ.

Good luck, Denise.

Oct 29 '06 #9
Denise Kleingeist wrote:
Hello Gianni!
Gianni Mariani wrote:
>OK, now I get it. Bypass all the nonsense and get to the streambuf.

That certainly wasn't what I was saying but there are many
approaches to obtain the desired goal.
>The only last thing I would like is to be able to covert a non binary
file to a "std::ios::binary" file without closing it.

There is no way to do something like this with the standard
I/O function, neither in C nor in C++. I guess, that is part
of the reason why these flags are called "openmode" rather
than "streammode" or something like this. If you are on a
UNIX system there is nothing to do anyway: the file modes
simply don't differ.
The code is cross platform. I'd like to stick to as many standard C++
constructs as possible. I suspected that this was a stretch.

according to msdn this is all that it needs to do.
result = _setmode( _fileno( stdin ), _O_BINARY );
Oct 29 '06 #10
On Sun, 29 Oct 2006 13:48:36 +1100 in comp.lang.c++, Gianni Mariani
<gi*******@mariani.wswrote,
>I want to read all the data from a file until end of file a chunk at a
time using std::istream.

Sounds simple enough right ?

Well there does not seem to be any way to do this easily.

std::istream::read is incapable of reading the last incomplete block.
As Denise pointed out to you, std::istream::read can and does read the
last incomplete block.

Oct 29 '06 #11
David Harmon wrote:
On Sun, 29 Oct 2006 13:48:36 +1100 in comp.lang.c++, Gianni Mariani
<gi*******@mariani.wswrote,
....
>
As Denise pointed out to you, std::istream::read can and does read the
last incomplete block.
Yes, she did.

However, this raises another set of questions.

a) I find it outside normal practice for interfaces requiring you to
call to get the result of the previous call. It seems error prone. The
whole "gcount()" thing seems somewhat iffy.

b) If a streambuf can be used to read() (using sgetn), what advantage
(or disadvantages) are there by using "istream::read insteam instead ?
Some prior discussions seem to indicate that sgetn() is "less cooked"
but I can't find anything that describes the intent or if there is any
difference whatsoever.

streambuf::sgetn seems to be best alternative for the app in question
since it's not needing any of the services from the std::istream
interface. The application I have reads a file and while reading it
determines if the file is a "binary" version, in which case it needs to
read the raw data from disk, otherwise it's going to interpret <cror
<cr><lfor <lfas line separators and process each line individually
so there is no need to have a "text" mode.

And while I'm on a rant, I might as well go all out ...

Also, it seems to me that streambuf as a design concept is doing too
much. I'm finding it hard to understand why you need to have the locale
knowledge down at the "raw file" layer. I can understand why you would
have it at the ostream or istream level, but not any lower than that.

As far as I can tell, it appears that the whole locale thing is a little
broken anyway. I would expect to see better support for encodings
especially utf-16 as a type separate to wchar_t, or even utf-32. At
this point, if you need to do anything with unicode and be truly
portable, you have to more-or-less ignore all the encoding specific
support and roll your own (or use an alternative).

So there. std::iostreams are on one hand a very useful beast and
certainly fixed alot of problems when it comes to using "C" stdio,
however we have a new set of problems (albeit not as bad as cstdio).

The holy grail of a standard io libary for me would be somthing that
embodies the entire capabilities of the Win32 and Linux API's for I/O
(mapped files, asynchronous I/O etc) and also provided far better
internationalization support. I would like to see "attribute based"
functionality (hard to describe here), which allows to select resources
based on the resource announcing itself as the resource to select based
for an attribute's setting. This would get away from all this complex
and very specific "locale" support to a model where the resource is
specified by the application and selectable through any number of
system, application or user specific attributes and resources.
OK so I probably went OT as well ...

cheers !
Oct 29 '06 #12
aep

Gianni Mariani wrote:
It appeared at first that "readsome" would do exactly what I wanted but
it appears not to work very well at all (at least with gcc!). When
reading from a named pipe on gcc, it returns immediately - no errors,
just immediate return, is that the expected behaviour ?
Version 5.0.3 of libstdc++ with gcc has a bug in which readsome always
returns 0. This is fixed in later versions. Perhaps this is the issue
you're seeing.

Oct 30 '06 #13
aep wrote:
Gianni Mariani wrote:
>It appeared at first that "readsome" would do exactly what I wanted but
it appears not to work very well at all (at least with gcc!). When
reading from a named pipe on gcc, it returns immediately - no errors,
just immediate return, is that the expected behaviour ?

Version 5.0.3 of libstdc++ with gcc has a bug in which readsome always
returns 0. This is fixed in later versions. Perhaps this is the issue
you're seeing.
Perhaps in the past but not in this case. This one is credited to the
learning curve.
Oct 30 '06 #14

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

3 posts views Thread by Morten Rodal | last post: by
19 posts views Thread by Lionel B | last post: by
2 posts views Thread by Alex Vinokur | last post: by
3 posts views Thread by Mario | last post: by
18 posts views Thread by Amadeus W. M. | last post: by
5 posts views Thread by ashton | last post: by
2 posts views Thread by Karl | last post: by
5 posts views Thread by Noel Mosa | last post: by
16 posts views Thread by vizzz | last post: by
reply views Thread by gheharukoh7 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.