473,320 Members | 2,177 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

is std::ifstream buffered or not?

Hey everyone!

I've got a quick question on whether std::ifstream is buffered or not.
The reason is that I have a homework assignment that requires me to
benchmark copying files using different buffer sizes. I ended up doing
this using std::istream::readsome() and std::ostream::write():

// now try writing to the destination file
std::ifstream sourceStream(sourceFilename.c_str(),
std::ios::binary);
std::ofstream destinationStream(destinationFilename.c_str(),
std::ios::binary | std::ios::trunc);

// determine the size of the file
sourceStream.seekg(0, std::ios::end);
const std::streamsize totalLength = sourceStream.tellg();
sourceStream.seekg(0, std::ios::beg);

// now writing the actual data
char buffer[numberOfBytes];
std::streamsize length = 0;
while(length != totalLength)
{
int l = sourceStream.readsome(buffer, numberOfBytes);
destinationStream.write(buffer, l);
length += l;
}
// now close the stream
sourceStream.close();
destinationStream.close();

Now, the validity of benchmarking this could possibly be in question
because std::istream::readsome() could be doing some buffering
(readahead?) in the back, so using different buffer sizes would
possibly be meaningless.

I understand std::cout and std::cin is buffered, and it makes sense
that they are. However, I do not see why std::ifstream and
std::ofstream would be buffered because the filesystem and even the
harddrive does some buffering. Wouldn't that be meaningless?

Then again, this could depend on the implementation. I'm not familiar
enough with the C++ STL specification.

Can anyone help me out? Any help would be greatly appreciated. :)

Jun 25 '06 #1
2 13936
Karl wrote:
Hey everyone!

I've got a quick question on whether std::ifstream is buffered or not.
It is, typically.
The reason is that I have a homework assignment that requires me to
benchmark copying files using different buffer sizes. I ended up doing
this using std::istream::readsome() and std::ostream::write():

// now try writing to the destination file
std::ifstream sourceStream(sourceFilename.c_str(),
std::ios::binary);
std::ofstream destinationStream(destinationFilename.c_str(),
std::ios::binary | std::ios::trunc);

// determine the size of the file
sourceStream.seekg(0, std::ios::end);
const std::streamsize totalLength = sourceStream.tellg();
sourceStream.seekg(0, std::ios::beg);

// now writing the actual data
char buffer[numberOfBytes];
std::streamsize length = 0;
while(length != totalLength)
{
int l = sourceStream.readsome(buffer, numberOfBytes);
destinationStream.write(buffer, l);
length += l;
}
// now close the stream
sourceStream.close();
destinationStream.close();

Now, the validity of benchmarking this could possibly be in question
because std::istream::readsome() could be doing some buffering
(readahead?) in the back, so using different buffer sizes would
possibly be meaningless.

I understand std::cout and std::cin is buffered, and it makes sense
that they are. However, I do not see why std::ifstream and
std::ofstream would be buffered because the filesystem and even the
harddrive does some buffering. Wouldn't that be meaningless?
Not at all. Buffering is useful at many levels. For iostreams, buffering
is done to reduce the number of virtual function calls made, and to
reduce the number of OS read calls made, both of which improve
performance regardless of any buffering happening at lower levels.
Then again, this could depend on the implementation. I'm not familiar
enough with the C++ STL specification.

Can anyone help me out? Any help would be greatly appreciated. :)


You can disable iostream level buffering like this:
std::ifstream sourceStream;
sourceStream.rdbuf()->pubsetbuf(0, 0);
sourceStream.open(sourceFilename.c_str(), std::ios::binary);

std::ofstream destinationStream;
destinationStream.rdbuf()->pubsetbuf(0, 0);
destinationStream.open(destinationFilename.c_str() ,
std::ios::binary | std::ios::trunc);

Note for it to be sure to work, you need to set up the buffer before
opening the file.

Tom
Jun 26 '06 #2
Tom Widmer wrote:
You can disable iostream level buffering like this:
std::ifstream sourceStream;
sourceStream.rdbuf()->pubsetbuf(0, 0);
sourceStream.open(sourceFilename.c_str(), std::ios::binary);

std::ofstream destinationStream;
destinationStream.rdbuf()->pubsetbuf(0, 0);
destinationStream.open(destinationFilename.c_str() ,
std::ios::binary | std::ios::trunc);

Note for it to be sure to work, you need to set up the buffer before
opening the file.


Thank you Tom, that's perfect! I looked at the pubsetbuf()
documentation and see that that does exactly what I need. Thanks!

Jun 27 '06 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: csvka | last post by:
Hello, I wonder if I could pick your brains. I'm beginning to learn about C++. I have opened a file in my program and I want to read lines from it. I would like this to be done in a separate...
4
by: hall | last post by:
Hi. I ran across a bug in one of my problems and after spending some time tracking it down i found that the problem arose in a piece of code that essentially did this: ----------- ifstream...
3
by: Charlie | last post by:
Dear all, I am currently writting a trace analyzer in C++. It always fails to open a very large input file (3.7Gb). I tried on a simple program, same thing happens:...
10
by: sam | last post by:
Hi, Can anyone tell me how to print a file name from ifstream? the following cout code does not print the filename I created with ifstream preivous: ifstream is; is.open ("text.txt");
12
by: Steven T. Hatton | last post by:
I know of a least one person who believes std::ifstream::read() and std::ofstream::write() are "mistakes". They seem to do the job I want done. What's wrong with them. This is the code I...
0
by: Chris | last post by:
I am reading in image files in a program and I read in the header in ascii mode. The problem is, sometimes tellg () gives me a completely incorrect result and sometimes it is just fine. Here is...
2
by: Assertor | last post by:
Hi, All. (VC++6.0) I found some strange thins when using getline() and seekg() of std::ifstream. After the file position of an open file was shift to the end of the file, seekg() did not...
5
by: Assertor | last post by:
Hi, all. Is there any way to create an instance of std::ifstream using std::string. (through std::ifstream's constructor or assignment operator or iterator, etc...) i.e. std::string str =...
2
by: mpalomas | last post by:
Hi C++ folks, I have trouble to open files whose path contains non-ascii characters with std::ifstream. For instance let's say i just have a file which has Japanese characters either in the...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.