Using .Read(buffer, 0, totalSize) with low RAM

[thiago777]

I have only 1GB of RAM so I cannot work with files too big with the ReadtoEnd method.
Here is the code Im trying so that the file would split in pieces of 256MB:

 Try
               Dim BInput As New FileStream(Filename, FileMode.Open, FileAccess.Read)
               Dim Reader As New BinaryReader(BInput)
 
               Dim info As New FileInfo(Filename)
               totalSize = info.Length
               Dim i As Integer = totalSize / Dtblock 'Dtblock is a const 256mb
               Dim completed As Integer = 0
 
               Dim firstround As Boolean = True
 
               If (totalSize > Dtblock) Then
                  'The file is bigger then 256MB
                  'Prepare to split
                  Dim EngineObj As New Engine()
                  ReDim buffer(Dtblock)
 
                  While completed < totalSize
 
                     Reader.Read(buffer, completed, completed + Dtblock)
                     '..
'here I work with the buffer, then I have to save it
  '..
 
                     completed += Dtblock + 1
 
                     If firstround Then 'here I check if I should create or append to output
                        'Write
                        firstround = False
                        Dim BOutput As New FileStream(TxtBoxOutput.Text, FileMode.Create, FileAccess.Write)
                        Dim writer As New BinaryWriter(BOutput)
                        writer.Write(buffer)
                     Else
                        'Append
                        Dim BOutput As New FileStream(TxtBoxOutput.Text, FileMode.Append, FileAccess.Write)
                        Dim writer As New BinaryWriter(BOutput)
                        writer.Write(buffer)
                     End If
 
                  End While
 
               Else
                  'The file is smaller then 256MB
 
                  ReDim buffer(totalSize)
                  Reader.Read(buffer, 0, totalSize)
..
                 'Here I work with the buffer..
..
                  'Write buffer to new file
                  Dim BOutput As New FileStream(TxtBoxOutput.Text, FileMode.Create, FileAccess.Write)
                  Dim writer As New BinaryWriter(BOutput)
                  writer.Write(buffer)
                  'Done, close instances of file streams
                  Reader.Close()
                  writer.Close()
 
               End If
 
               BInput.Dispose()
               MessageBox.Show("Operation finished!")
            Catch ex As Exception
               MessageBox.Show(ex.ToString)
            End Try

I know it might be horrible, but its the only idea I came up with.
anyone knows a better way to accomplish this?

Or if somebody find knows why I always get an exception on the "Reader.Read(buffer, completed, completed + Dtblock)" saying :

"Offset and length for the array lie outside of the valid field, or the number is greater than the number of the elements of the index to at the end of the source listing."

would be even better..

but till now I dont know why that happens because I supposed that the parameters for Read was like this:

Read(<WheretoStore>,<starting point>, <how many to read from starting point>)

correct?

[TRScheel]

I am not sure, but I think the error is because when you read, you move the start location of the read. For example:

The dog jumped over the moon

If you wrote:

Read(myBuffer, 0, 3)

You would get 'The'

If you did it again, you would get ' do'. But if instead you did:

Read(myBuffer, 2, 3)

you would get 'og '

This is all purely speculation, a short test should prove this wrong/right.

[thiago777]

Hey!
thank you very much for your reply.

you were right about the Read's parameter.

I just didnt know that it had an internal pointer that keeps track of the last byte processed ;)

thanks again!

later,
Thiago.

[TRScheel]

No problem, glad to have helped.

[Plater]

For Read the paremters are:
[number of bytes actually read] Read( [name of byte[] you want the data to go into], [starting index in the GIVEN byte[] (the previous parameter)], [total number of bytes to attempt to read] );

so if I wanted to try to read 1024 bytes into a buffer starting at the first index in my array I woul do this:

Byte[] buff=new Byte[1024];
int actualread=-1;
actaulread=mystream.Read(buff,0,buff.Length);
//actualread will be set to how many bytes I actually was able to read, or 0 if end of stream