By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
446,369 Members | 1,207 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 446,369 IT Pros & Developers. It's quick & easy.

How to only read the last 100 bytes from a Binary file?

P: n/a
Hi,

Is it possible to read a file in reverse and only get the last 100 bytes in
the file without reading the whole file from the begining? I have to get info
from files that are in the last 100 bytes of the file and some of these files
are 600Mb -1 GB in size. I am getting "outofMemory.." exceptions on the
largest files and the other files take "forever" to get the last 100 bytes.

This is the code I have currently that works with smaller files:

Dim fileData() As Byte
FilePath = "C:\Temp\1DD04336CA1A45CB81A89167048D2594.abk"
Try
Dim oFile As System.IO.FileInfo
oFile = New System.IO.FileInfo(FilePath)
Dim oFileStream As System.IO.FileStream = oFile.Open
_(IO.FileMode.Open, IO.FileAccess.Read)
Dim lBytes As Long = oFileStream.Length
ReDim fileData(lBytes)
Dim StartReadAt As Long = lBytes - 100
Dim StopReadAt As Long = 100
oFileStream.Read(fileData, StartReadAt, StopReadAt)
oFileStream.Close()
Catch ex As Exception
MsgBox(ex.Message)
End Try

The problem with this code is an Array is created in the size of the file
resulting in upper bounds of maybe 200 000 000, so this is time consuming to
get the last 100 bytes.

Any idéas?
Thanks /Thomas
Feb 1 '07 #1
Share this Question
Share on Google+
6 Replies


P: n/a
Try seeking to oFileStream.Length - 100. Then read from there to the end of
the file. The array size only needs to be 100 bytes. It may take the OS a
while to do the seek, but you won't get the out of memory errors.

Mike Ober.

"ThomasZ" <Th*****@discussions.microsoft.comwrote in message
news:4C**********************************@microsof t.com...
Hi,

Is it possible to read a file in reverse and only get the last 100 bytes
in
the file without reading the whole file from the begining? I have to get
info
from files that are in the last 100 bytes of the file and some of these
files
are 600Mb -1 GB in size. I am getting "outofMemory.." exceptions on the
largest files and the other files take "forever" to get the last 100
bytes.

This is the code I have currently that works with smaller files:

Dim fileData() As Byte
FilePath = "C:\Temp\1DD04336CA1A45CB81A89167048D2594.abk"
Try
Dim oFile As System.IO.FileInfo
oFile = New System.IO.FileInfo(FilePath)
Dim oFileStream As System.IO.FileStream = oFile.Open
_(IO.FileMode.Open, IO.FileAccess.Read)
Dim lBytes As Long = oFileStream.Length
ReDim fileData(lBytes)
Dim StartReadAt As Long = lBytes - 100
Dim StopReadAt As Long = 100
oFileStream.Read(fileData, StartReadAt, StopReadAt)
oFileStream.Close()
Catch ex As Exception
MsgBox(ex.Message)
End Try

The problem with this code is an Array is created in the size of the file
resulting in upper bounds of maybe 200 000 000, so this is time consuming
to
get the last 100 bytes.

Any idéas?
Thanks /Thomas

Feb 1 '07 #2

P: n/a
Ahh, thanks for the "Eye opener"
I used the binaryreader instead and set the "cursor" to the End -100
position and step to the end.

This code worked for me:

Dim fileData(100) As Byte
Dim oFileStream As New FileStream(FilePath, FileMode.Open,
FileAccess.Read)
Dim oBinaryReader As New BinaryReader(oFileStream)
Dim lBytes As Long = oFileStream.Length
oBinaryReader.BaseStream.Position = lBytes - 100
Dim fdi As Integer = 0
For i As Long = 0 To 99
fileData(i) = oBinaryReader.ReadByte()
Next i
oBinaryReader.Close()
oFileStream.Close()

Thanx
/Thomas

"Michael D. Ober" wrote:
Try seeking to oFileStream.Length - 100. Then read from there to the end of
the file. The array size only needs to be 100 bytes. It may take the OS a
while to do the seek, but you won't get the out of memory errors.

Mike Ober.

"ThomasZ" <Th*****@discussions.microsoft.comwrote in message
news:4C**********************************@microsof t.com...
Hi,

Is it possible to read a file in reverse and only get the last 100 bytes
in
the file without reading the whole file from the begining? I have to get
info
from files that are in the last 100 bytes of the file and some of these
files
are 600Mb -1 GB in size. I am getting "outofMemory.." exceptions on the
largest files and the other files take "forever" to get the last 100
bytes.

This is the code I have currently that works with smaller files:

Dim fileData() As Byte
FilePath = "C:\Temp\1DD04336CA1A45CB81A89167048D2594.abk"
Try
Dim oFile As System.IO.FileInfo
oFile = New System.IO.FileInfo(FilePath)
Dim oFileStream As System.IO.FileStream = oFile.Open
_(IO.FileMode.Open, IO.FileAccess.Read)
Dim lBytes As Long = oFileStream.Length
ReDim fileData(lBytes)
Dim StartReadAt As Long = lBytes - 100
Dim StopReadAt As Long = 100
oFileStream.Read(fileData, StartReadAt, StopReadAt)
oFileStream.Close()
Catch ex As Exception
MsgBox(ex.Message)
End Try

The problem with this code is an Array is created in the size of the file
resulting in upper bounds of maybe 200 000 000, so this is time consuming
to
get the last 100 bytes.

Any idéas?
Thanks /Thomas


Feb 1 '07 #3

P: n/a
Out of curiosity, is there a performance difference when seeking to end -100
on a 1K file vs a 10Gb file. I know the old FAT file system would take a
lot longer for the seek on the larger file. Also, why not use ReadBytes and
avoid your read loop?

Dim oBinaryReader As New BinaryReader(New FileStream(FilePath,
FileMode.Open, FileAccess.Read))
oBinaryReader.BaseStream.Position = oBinaryReader.BaseStream.Length - 100
Dim fileData(100) As Byte = oBinaryReader.ReadBytes(100)
oBinaryReader.Close() ' Close both the binary reader and the base
stream

Mike.

"ThomasZ" <Th*****@discussions.microsoft.comwrote in message
news:34**********************************@microsof t.com...
Ahh, thanks for the "Eye opener"
I used the binaryreader instead and set the "cursor" to the End -100
position and step to the end.

This code worked for me:

Dim fileData(100) As Byte
Dim oFileStream As New FileStream(FilePath, FileMode.Open,
FileAccess.Read)
Dim oBinaryReader As New BinaryReader(oFileStream)
Dim lBytes As Long = oFileStream.Length
oBinaryReader.BaseStream.Position = lBytes - 100
Dim fdi As Integer = 0
For i As Long = 0 To 99
fileData(i) = oBinaryReader.ReadByte()
Next i
oBinaryReader.Close()
oFileStream.Close()

Thanx
/Thomas

"Michael D. Ober" wrote:
>Try seeking to oFileStream.Length - 100. Then read from there to the end
of
the file. The array size only needs to be 100 bytes. It may take the OS
a
while to do the seek, but you won't get the out of memory errors.

Mike Ober.

"ThomasZ" <Th*****@discussions.microsoft.comwrote in message
news:4C**********************************@microso ft.com...
Hi,

Is it possible to read a file in reverse and only get the last 100
bytes
in
the file without reading the whole file from the begining? I have to
get
info
from files that are in the last 100 bytes of the file and some of these
files
are 600Mb -1 GB in size. I am getting "outofMemory.." exceptions on the
largest files and the other files take "forever" to get the last 100
bytes.

This is the code I have currently that works with smaller files:

Dim fileData() As Byte
FilePath = "C:\Temp\1DD04336CA1A45CB81A89167048D2594.abk"
Try
Dim oFile As System.IO.FileInfo
oFile = New System.IO.FileInfo(FilePath)
Dim oFileStream As System.IO.FileStream = oFile.Open
_(IO.FileMode.Open, IO.FileAccess.Read)
Dim lBytes As Long = oFileStream.Length
ReDim fileData(lBytes)
Dim StartReadAt As Long = lBytes - 100
Dim StopReadAt As Long = 100
oFileStream.Read(fileData, StartReadAt, StopReadAt)
oFileStream.Close()
Catch ex As Exception
MsgBox(ex.Message)
End Try

The problem with this code is an Array is created in the size of the
file
resulting in upper bounds of maybe 200 000 000, so this is time
consuming
to
get the last 100 bytes.

Any idéas?
Thanks /Thomas



Feb 1 '07 #4

P: n/a
Regardless of the file size it takes about 156 milliseconds to get the last
100 bytes.
Measured with a 100kb file and a 656MB file over the network. This is very
nice since I have to read hundreds of files on a regular basis.

I get this error when trying youre example with ReadBytes(100) "Explicit
initialization is not permitted for arrays declared with explicit bounds."
But the idéa is good and I'll look further into it.

Thanks again
/Thomas

"Michael D. Ober" wrote:
Out of curiosity, is there a performance difference when seeking to end -100
on a 1K file vs a 10Gb file. I know the old FAT file system would take a
lot longer for the seek on the larger file. Also, why not use ReadBytes and
avoid your read loop?

Dim oBinaryReader As New BinaryReader(New FileStream(FilePath,
FileMode.Open, FileAccess.Read))
oBinaryReader.BaseStream.Position = oBinaryReader.BaseStream.Length - 100
Dim fileData(100) As Byte = oBinaryReader.ReadBytes(100)
oBinaryReader.Close() ' Close both the binary reader and the base
stream

Mike.

"ThomasZ" <Th*****@discussions.microsoft.comwrote in message
news:34**********************************@microsof t.com...
Ahh, thanks for the "Eye opener"
I used the binaryreader instead and set the "cursor" to the End -100
position and step to the end.

This code worked for me:

Dim fileData(100) As Byte
Dim oFileStream As New FileStream(FilePath, FileMode.Open,
FileAccess.Read)
Dim oBinaryReader As New BinaryReader(oFileStream)
Dim lBytes As Long = oFileStream.Length
oBinaryReader.BaseStream.Position = lBytes - 100
Dim fdi As Integer = 0
For i As Long = 0 To 99
fileData(i) = oBinaryReader.ReadByte()
Next i
oBinaryReader.Close()
oFileStream.Close()

Thanx
/Thomas

"Michael D. Ober" wrote:
Try seeking to oFileStream.Length - 100. Then read from there to the end
of
the file. The array size only needs to be 100 bytes. It may take the OS
a
while to do the seek, but you won't get the out of memory errors.

Mike Ober.

"ThomasZ" <Th*****@discussions.microsoft.comwrote in message
news:4C**********************************@microsof t.com...
Hi,

Is it possible to read a file in reverse and only get the last 100
bytes
in
the file without reading the whole file from the begining? I have to
get
info
from files that are in the last 100 bytes of the file and some of these
files
are 600Mb -1 GB in size. I am getting "outofMemory.." exceptions on the
largest files and the other files take "forever" to get the last 100
bytes.

This is the code I have currently that works with smaller files:

Dim fileData() As Byte
FilePath = "C:\Temp\1DD04336CA1A45CB81A89167048D2594.abk"
Try
Dim oFile As System.IO.FileInfo
oFile = New System.IO.FileInfo(FilePath)
Dim oFileStream As System.IO.FileStream = oFile.Open
_(IO.FileMode.Open, IO.FileAccess.Read)
Dim lBytes As Long = oFileStream.Length
ReDim fileData(lBytes)
Dim StartReadAt As Long = lBytes - 100
Dim StopReadAt As Long = 100
oFileStream.Read(fileData, StartReadAt, StopReadAt)
oFileStream.Close()
Catch ex As Exception
MsgBox(ex.Message)
End Try

The problem with this code is an Array is created in the size of the
file
resulting in upper bounds of maybe 200 000 000, so this is time
consuming
to
get the last 100 bytes.

Any idéas?
Thanks /Thomas


Feb 2 '07 #5

P: n/a
Got it, Dim as Byte array:

Dim oFileStream As New FileStream(FilePath, FileMode.Open, FileAccess.Read)
Dim oBinaryReader As New BinaryReader(oFileStream)
Dim lBytes As Long = oFileStream.Length
oBinaryReader.BaseStream.Position = lBytes - 100
Dim fileData As Byte() = oBinaryReader.ReadBytes(100)
oBinaryReader.Close()
oFileStream.Close()

Did you mean that oBinaryReader.Close automatically closes oFileStream.Close?

"Michael D. Ober" wrote:
Out of curiosity, is there a performance difference when seeking to end -100
on a 1K file vs a 10Gb file. I know the old FAT file system would take a
lot longer for the seek on the larger file. Also, why not use ReadBytes and
avoid your read loop?

Dim oBinaryReader As New BinaryReader(New FileStream(FilePath,
FileMode.Open, FileAccess.Read))
oBinaryReader.BaseStream.Position = oBinaryReader.BaseStream.Length - 100
Dim fileData(100) As Byte = oBinaryReader.ReadBytes(100)
oBinaryReader.Close() ' Close both the binary reader and the base
stream

Mike.

"ThomasZ" <Th*****@discussions.microsoft.comwrote in message
news:34**********************************@microsof t.com...
Ahh, thanks for the "Eye opener"
I used the binaryreader instead and set the "cursor" to the End -100
position and step to the end.

This code worked for me:

Dim fileData(100) As Byte
Dim oFileStream As New FileStream(FilePath, FileMode.Open,
FileAccess.Read)
Dim oBinaryReader As New BinaryReader(oFileStream)
Dim lBytes As Long = oFileStream.Length
oBinaryReader.BaseStream.Position = lBytes - 100
Dim fdi As Integer = 0
For i As Long = 0 To 99
fileData(i) = oBinaryReader.ReadByte()
Next i
oBinaryReader.Close()
oFileStream.Close()

Thanx
/Thomas

"Michael D. Ober" wrote:
Try seeking to oFileStream.Length - 100. Then read from there to the end
of
the file. The array size only needs to be 100 bytes. It may take the OS
a
while to do the seek, but you won't get the out of memory errors.

Mike Ober.

"ThomasZ" <Th*****@discussions.microsoft.comwrote in message
news:4C**********************************@microsof t.com...
Hi,

Is it possible to read a file in reverse and only get the last 100
bytes
in
the file without reading the whole file from the begining? I have to
get
info
from files that are in the last 100 bytes of the file and some of these
files
are 600Mb -1 GB in size. I am getting "outofMemory.." exceptions on the
largest files and the other files take "forever" to get the last 100
bytes.

This is the code I have currently that works with smaller files:

Dim fileData() As Byte
FilePath = "C:\Temp\1DD04336CA1A45CB81A89167048D2594.abk"
Try
Dim oFile As System.IO.FileInfo
oFile = New System.IO.FileInfo(FilePath)
Dim oFileStream As System.IO.FileStream = oFile.Open
_(IO.FileMode.Open, IO.FileAccess.Read)
Dim lBytes As Long = oFileStream.Length
ReDim fileData(lBytes)
Dim StartReadAt As Long = lBytes - 100
Dim StopReadAt As Long = 100
oFileStream.Read(fileData, StartReadAt, StopReadAt)
oFileStream.Close()
Catch ex As Exception
MsgBox(ex.Message)
End Try

The problem with this code is an Array is created in the size of the
file
resulting in upper bounds of maybe 200 000 000, so this is time
consuming
to
get the last 100 bytes.

Any idéas?
Thanks /Thomas


Feb 2 '07 #6

P: n/a
According to the documentation, BinaryReader.Close() closes both the binary
reader and the underlying stream. That's why you can use a single Binary
Reader and access it's BaseStream property to get the file length and set
your desired position.

Mike.

"ThomasZ" <Th*****@discussions.microsoft.comwrote in message
news:1A**********************************@microsof t.com...
Got it, Dim as Byte array:

Dim oFileStream As New FileStream(FilePath, FileMode.Open,
FileAccess.Read)
Dim oBinaryReader As New BinaryReader(oFileStream)
Dim lBytes As Long = oFileStream.Length
oBinaryReader.BaseStream.Position = lBytes - 100
Dim fileData As Byte() = oBinaryReader.ReadBytes(100)
oBinaryReader.Close()
oFileStream.Close()

Did you mean that oBinaryReader.Close automatically closes
oFileStream.Close?

"Michael D. Ober" wrote:
>Out of curiosity, is there a performance difference when seeking to
end -100
on a 1K file vs a 10Gb file. I know the old FAT file system would take a
lot longer for the seek on the larger file. Also, why not use ReadBytes
and
avoid your read loop?

Dim oBinaryReader As New BinaryReader(New FileStream(FilePath,
FileMode.Open, FileAccess.Read))
oBinaryReader.BaseStream.Position = oBinaryReader.BaseStream.Length - 100
Dim fileData(100) As Byte = oBinaryReader.ReadBytes(100)
oBinaryReader.Close() ' Close both the binary reader and the
base
stream

Mike.

"ThomasZ" <Th*****@discussions.microsoft.comwrote in message
news:34**********************************@microso ft.com...
Ahh, thanks for the "Eye opener"
I used the binaryreader instead and set the "cursor" to the End -100
position and step to the end.

This code worked for me:

Dim fileData(100) As Byte
Dim oFileStream As New FileStream(FilePath, FileMode.Open,
FileAccess.Read)
Dim oBinaryReader As New BinaryReader(oFileStream)
Dim lBytes As Long = oFileStream.Length
oBinaryReader.BaseStream.Position = lBytes - 100
Dim fdi As Integer = 0
For i As Long = 0 To 99
fileData(i) = oBinaryReader.ReadByte()
Next i
oBinaryReader.Close()
oFileStream.Close()

Thanx
/Thomas

"Michael D. Ober" wrote:

Try seeking to oFileStream.Length - 100. Then read from there to the
end
of
the file. The array size only needs to be 100 bytes. It may take the
OS
a
while to do the seek, but you won't get the out of memory errors.

Mike Ober.

"ThomasZ" <Th*****@discussions.microsoft.comwrote in message
news:4C**********************************@microso ft.com...
Hi,

Is it possible to read a file in reverse and only get the last 100
bytes
in
the file without reading the whole file from the begining? I have to
get
info
from files that are in the last 100 bytes of the file and some of
these
files
are 600Mb -1 GB in size. I am getting "outofMemory.." exceptions on
the
largest files and the other files take "forever" to get the last 100
bytes.

This is the code I have currently that works with smaller files:

Dim fileData() As Byte
FilePath = "C:\Temp\1DD04336CA1A45CB81A89167048D2594.abk"
Try
Dim oFile As System.IO.FileInfo
oFile = New System.IO.FileInfo(FilePath)
Dim oFileStream As System.IO.FileStream = oFile.Open
_(IO.FileMode.Open, IO.FileAccess.Read)
Dim lBytes As Long = oFileStream.Length
ReDim fileData(lBytes)
Dim StartReadAt As Long = lBytes - 100
Dim StopReadAt As Long = 100
oFileStream.Read(fileData, StartReadAt, StopReadAt)
oFileStream.Close()
Catch ex As Exception
MsgBox(ex.Message)
End Try

The problem with this code is an Array is created in the size of the
file
resulting in upper bounds of maybe 200 000 000, so this is time
consuming
to
get the last 100 bytes.

Any idéas?
Thanks /Thomas



Feb 2 '07 #7

This discussion thread is closed

Replies have been disabled for this discussion.