473,394 Members | 1,887 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,394 software developers and data experts.

Size of Byte Array for big files?

I am using the following:
Dim fsReadStream As New FileStream(TextBoxVideoFileName.Text, FileMode.Open,
FileAccess.Read)
Dim brReader As New BinaryReader(fsReadStream)
Dim ByteArray() As Byte

ByteArray = brReader.ReadBytes(fsReadStream.Length)

This works fine for smaller files, but crashes with an overflow exception
with a large file (2.5GB). Is the size of the array a problem? How can I
change it? Any other solution?

Thanx,
--
Anil Gupte
www.keeninc.net
www.icinema.com
Aug 21 '07 #1
3 4313
AFAIK arrays are indexed using an integer whose max value allows for 2 Go.

Even for smaller files it's likely better to avoid loading a whole file in
memory unless this is strictly needed and especially if those files are not
limited in size. A common practice is to read a file in "chunks" so that you
consume only the size of a buffer that holds the current "access window" on
file data...

So it would depend what you are doing once the whole file is loaded in
memory....

---
Patrice

"Anil Gupte" <an*******@icinema.coma écrit dans le message de news:
ur**************@TK2MSFTNGP02.phx.gbl...
>I am using the following:
Dim fsReadStream As New FileStream(TextBoxVideoFileName.Text,
FileMode.Open, FileAccess.Read)
Dim brReader As New BinaryReader(fsReadStream)
Dim ByteArray() As Byte

ByteArray = brReader.ReadBytes(fsReadStream.Length)

This works fine for smaller files, but crashes with an overflow exception
with a large file (2.5GB). Is the size of the array a problem? How can I
change it? Any other solution?

Thanx,
--
Anil Gupte
www.keeninc.net
www.icinema.com

Aug 21 '07 #2
"Anil Gupte" <an*******@icinema.comwrote in
news:ur**************@TK2MSFTNGP02.phx.gbl:
This works fine for smaller files, but crashes with an overflow
exception with a large file (2.5GB). Is the size of the array a
problem? How can I change it? Any other solution?
Yes, you're creating a 2.5GB array in memory. I believe there is a 2GB
limit?

You should stream the file using a smaller array as data is needed.
Aug 21 '07 #3
Thanx for your response. How do I "stream the file"?

Best,
--
Anil Gupte
www.keeninc.net
www.icinema.com

"Spam Catcher" <sp**********@rogers.comwrote in message
news:Xn**********************************@127.0.0. 1...
"Anil Gupte" <an*******@icinema.comwrote in
news:ur**************@TK2MSFTNGP02.phx.gbl:
>This works fine for smaller files, but crashes with an overflow
exception with a large file (2.5GB). Is the size of the array a
problem? How can I change it? Any other solution?

Yes, you're creating a 2.5GB array in memory. I believe there is a 2GB
limit?

You should stream the file using a smaller array as data is needed.

Aug 21 '07 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: J?n Sveinsson | last post by:
Hello everyone I have been trying to read and write struct to binary files, I'm using to functions to convert the struct to bytes and bytes to struct, I always receive the following error ...
1
by: Jón Sveinsson | last post by:
Hello everyone I have been trying to read and write struct to binary files, I'm using to functions to convert the struct to bytes and bytes to struct, I always receive the following error ...
2
by: Ron | last post by:
Hello, I am trying to read a list of files from an FTP server (mainframe) to a byte array using sockets as follows, but not getting all the files in a given directory: private readonly static...
4
by: Phillip Ian | last post by:
Version: VS 2005 I took the sample code from help about encrypting and decrypting strings, and changed it to work directly with byte arrays and get the key and IV values from functions I've...
5
by: news.microsoft.com | last post by:
Hello, what is the most performant size for the byte array for reading/writing using 2 filestreams? example code: Dim bytearrayinput(4095) As Byte Dim rdlen As Long = 0 Dim totlen As Long...
5
by: Gerrit | last post by:
Hi all, I'm getting an OutOfMemoryException when I initialize a byte array in C# like this: Byte test = new Byte; I'm using ASP.NET 2.0. In ASP.Net 1.1 it works fine. So what am I doing
2
by: kaczmar2 | last post by:
I have code like so: FileStream fs = new FileStream(locationTextBox.Text, FileMode.Open, FileAccess.Write, FileShare.Read); long size = fs.Length; byte buffer = new byte; // Get file to byte...
9
by: herobeat | last post by:
Hi all, I'm having a hell of a time with declaring a struct to hold some binary data I'm trying to read from some files on disk. What I would like to do is something like this: public struct...
10
by: Chunekit Pong | last post by:
I have a BYTE array - BYTE const* pbBinary I would like to know how many bytes in that byte array but if I do - sizeof(* pbBinary); - then I got 1 but if I do - sizeof( pbBinary); - then I...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.