473,385 Members | 1,907 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

Loading large file into Memory

Hi,

The problem is:
We are having huge amount of binary data something in the range of 500 - 600
MB. The application will be written in .Net( Windows application). It need to display the complete information in graphical view. The problem is how to efficiently load the complete data into memory. If loaded in pieces, what will be the optimized way to do read operation from disk.....
Oct 13 '06 #1
0 1029

Sign in to post your reply or Sign up for a free account.

Similar topics

1
by: Philipp K. Janert, Ph.D. | last post by:
Dear All! I am trying to load a relatively large table (about 1 Million rows) into an sqlite table, which is kept in memory. The load process is very slow - on the order of 15 minutes or so. ...
2
by: Gustaf Liljegren | last post by:
I'm writing a program that will query an XML file with the XmlTextReader class (the question is not specifically about XML, however). This file is very large (maybe 20 MB at most), and during...
2
by: Wizfrog | last post by:
Hello, I'm working with a pretty large XML file, but I really only need to display a few things that requires quite a few transforms. I already limited to the transforms to the data i need to...
4
by: =?Utf-8?B?UGhpbCBKb2huc29u?= | last post by:
Hello, I am working on a peice of code that takes a file and loads it into a SQL Server DB. When I load large files (over 70Mb) I get a system out of memory exception. My machine has 2G RAM ...
5
by: s0suk3 | last post by:
Hi, I wanted to know how cautious it is to do something like: f = file("filename", "rb") f.read() for a possibly huge file. When calling f.read(), and not doing anything with the return...
17
by: byte8bits | last post by:
How does C++ safely open and read very large files? For example, say I have 1GB of physical memory and I open a 4GB file and attempt to read it like so: #include <iostream> #include <fstream>...
0
by: =?Utf-8?B?UGhpbCBKb2huc29u?= | last post by:
Hi, I am working on an ASP.NET 1.1 application that creates thumbnails using the code below. Problem is that the code loads an entire file into memory to instantiate the image (file.Data is a...
6
by: Terry Carroll | last post by:
I am trying to do something with a very large tarfile from within Python, and am running into memory constraints. The tarfile in question is a 4-gigabyte datafile from freedb.org,...
2
by: Kris Kennaway | last post by:
I would like to MIME encode a message from a large file without first loading the file into memory. Assume the file has been pre-encoded on disk (actually I am using encode_7or8bit, so the...
5
by: DR | last post by:
Why is its substantialy slower to load 50GB of gzipped file (20GB gzipped file) then loading 50GB unzipped data? im using System.IO.Compression.GZipStream and its not maxing out the cpu while...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.