473,387 Members | 1,464 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,387 software developers and data experts.

Dealing with very large binary files

Helllo,

What's the general accepted strategy for dealing with very large
binary files in C# ?
I have to do a program that reads some "multi frame bitmap " files
which can reach up to one hundred megs so I need to know how to
optimize reading a file..

Best regards,
Seb

Aug 6 '07 #1
3 2346
se*************@gmail.com wrote:
What's the general accepted strategy for dealing with very large
binary files in C# ?
Very large binary files? Read a little bit at a time.
I have to do a program that reads some "multi frame bitmap " files
which can reach up to one hundred megs so I need to know how to
optimize reading a file..
Oh. I thought you were asking about "very large binary files". These
days, 100MB isn't really all that big. :)

That said, you are likely to find that the most important issue is to
make sure you read enough data at one time. You can use a
BufferedStream to ensure this, but my experience has been that even
FileStream has some built-in buffering (caching at the OS level) that
results in pretty good performance anyway.

Do you have a specific issue that is coming up in which performance is
not satisfactory? Generally, it's better to not waste time optimizing
until you know what performance problem you're trying to solve. Until
you have an actual performance problem, you can't answer that.

Pete
Aug 6 '07 #2
It really depends on what you will do with those files. But you might check
out memory-mapped files.

--
Jonathan Wood
SoftCircuits Programming
http://www.softcircuits.com

<se*************@gmail.comwrote in message
news:11*********************@57g2000hsv.googlegrou ps.com...
Helllo,

What's the general accepted strategy for dealing with very large
binary files in C# ?
I have to do a program that reads some "multi frame bitmap " files
which can reach up to one hundred megs so I need to know how to
optimize reading a file..

Best regards,
Seb
Aug 6 '07 #3
HI,

<se*************@gmail.comwrote in message
news:11*********************@57g2000hsv.googlegrou ps.com...
Helllo,

What's the general accepted strategy for dealing with very large
binary files in C# ?
Depend of your escenario and of the kind of processing you need to do.
I have to do a program that reads some "multi frame bitmap " files
which can reach up to one hundred megs so I need to know how to
optimize reading a file..
And do what with them? As I said depending of your intended action will be
the best way of handling it
Aug 6 '07 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Marc | last post by:
Hello, I have a page where i can upload binary file (using the HTML input type=file approach). This works fine for relatively small files (<6MB)) but when files get bigger (13MB) there is a...
0
by: Sean Newton | last post by:
I am absolutely bewildered by now by the Microsoft.Samples SSPI and Security assemblies. I've been trying to set these up in a very straightforward harness in the way that I'd like to be able to...
11
by: CSN | last post by:
Is it possible to iterate over an array in plpgsql? Something like: function insert_stuff (rel_ids int) .... foreach rel_ids as id insert into table (rel_id, val) values (id, 5);
6
by: Greg | last post by:
I am working on a project that will have about 500,000 records in an XML document. This document will need to be queried with XPath, and records will need to be updated. I was thinking about...
1
by: Patrick | last post by:
Hi, This post is the 'sequel' ;) of the "Data Oriented vs Object Oriented Design" post, but it can be read and treated apart from that one. I will just quote the beginning of my previous message...
2
by: gauravkhanna | last post by:
Hi All I need some help for the below problem: Scenario We need to send large binary files (audio file of about 10 MB or so) from the client machine (.Net Windows based application, located...
2
by: jdev8080 | last post by:
We are looking at creating large XML files containing binary data (encoded as base64) and passing them to transformers that will parse and transform the data into different formats. Basically,...
17
by: byte8bits | last post by:
How does C++ safely open and read very large files? For example, say I have 1GB of physical memory and I open a 4GB file and attempt to read it like so: #include <iostream> #include <fstream>...
16
by: pereges | last post by:
ok so i have written a program in C where I am dealing with huge data(millions and lots of iterations involved) and for some reason the screen tends to freeze and I get no output every time I...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.