Connecting File Streams

[woessner]

This is not a really well-formed question so please bear with me.

I have a very large binary file (tens of GB). I would like to write a
quick program to take a contiguous piece of the file and save it to
another file. Conceptually, this is very simple:

Open ifstream
Open ofstream
Seek ifstream to beginning of piece
Read ifstream in to buffer
Write buffer to ofstream

I'm not very familiar with streams, but I think even I could handle
that. My concern is the with the buffer. I'm going to want to copy
far more data than will fit in memory. That means I will have to use a
loop to read and write the data in chunks. OK, that's still not a
problem. But it does raise the question of how big to make the buffer.
My gut instinct tells me to make the buffer as big as possible, but
I'm not really sure.

I'm hoping there's a way to avoid this problem, altogether. Is it
possible to "connect" an ifstream and an ofstream? Conceptually, this
is what I'm after:

Open ifstream
Open ofstream
Connect ifstream to ofstream
Seek ifstream to beginning of piece
Read ifstream (automatically writes to ofstream)

My hope is that the stream classes will allocate whatever buffers they
need and that those buffers will be appropriately sized.

I've looked through the Josuttis book a little bit. I read the section
on connecting streams, but that seems to be for synchronization. Does
anybody know if this is possible (and a good idea)? If not, I'm open
to any other suggestions.

Thanks in advance,
Bill

[Yahooooooooo]

streambuf may help to connect these streams

std::streambuf
std::cout.rdbuf

wo******@gmail.com wrote:

This is not a really well-formed question so please bear with me.

I have a very large binary file (tens of GB). I would like to write a
quick program to take a contiguous piece of the file and save it to
another file. Conceptually, this is very simple:

Open ifstream
Open ofstream
Seek ifstream to beginning of piece
Read ifstream in to buffer
Write buffer to ofstream

I'm not very familiar with streams, but I think even I could handle
that. My concern is the with the buffer. I'm going to want to copy
far more data than will fit in memory. That means I will have to use a
loop to read and write the data in chunks. OK, that's still not a
problem. But it does raise the question of how big to make the buffer.
My gut instinct tells me to make the buffer as big as possible, but
I'm not really sure.

I'm hoping there's a way to avoid this problem, altogether. Is it
possible to "connect" an ifstream and an ofstream? Conceptually, this
is what I'm after:

Open ifstream
Open ofstream
Connect ifstream to ofstream
Seek ifstream to beginning of piece
Read ifstream (automatically writes to ofstream)

My hope is that the stream classes will allocate whatever buffers they
need and that those buffers will be appropriately sized.

I've looked through the Josuttis book a little bit. I read the section
on connecting streams, but that seems to be for synchronization. Does
anybody know if this is possible (and a good idea)? If not, I'm open
to any other suggestions.

Thanks in advance,
Bill