473,387 Members | 1,573 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,387 software developers and data experts.

BinaryWritting sequencially

Hello,

I have to proccess big files (around 4GB) and to prevent Out of Memory exceptions I am splitting the file in 256MB blocks. The problem is, as I have to create the file sequencially (block by block), I dont know how to "stick"/"concatenate" the remaining pieces to the last one (or first processed).

I have to "add" the rest of the bytes to the file, anyone knows if this is possible? I am using FileStream and BinaryWriter to do it.

For example, to read I can do this:
[PHP]
Dim BInput As New FileStream(Filename, FileMode.Open, FileAccess.Read)
Dim Reader As New BinaryReader(BInput)
'then inside a loop with variable k:
Reader.Read(mydata, k * Dtblock, Dtblock)
'loop end[/PHP]

(where Dtblock is a constant that indicates 256MB.)

but then I dont know the oposite of the instructions above (to write)

thanks in advance!
Aug 2 '07 #1
1 979
Hello,

I have to proccess big files (around 4GB) and to prevent Out of Memory exceptions I am splitting the file in 256MB blocks. The problem is, as I have to create the file sequencially (block by block), I dont know how to "stick"/"concatenate" the remaining pieces to the last one (or first processed).

I have to "add" the rest of the bytes to the file, anyone knows if this is possible? I am using FileStream and BinaryWriter to do it.

For example, to read I can do this:
[PHP]
Dim BInput As New FileStream(Filename, FileMode.Open, FileAccess.Read)
Dim Reader As New BinaryReader(BInput)
'then inside a loop with variable k:
Reader.Read(mydata, k * Dtblock, Dtblock)
'loop end[/PHP]

(where Dtblock is a constant that indicates 256MB.)

but then I dont know the oposite of the instructions above (to write)

thanks in advance!
Would it work like:

[PHP]Dim BOutput As New FileStream(TxtBoxOutput.Text, FileMode.Append, FileAccess.Write)
Dim Writer As New BinaryWriter(BOutput)
Writer.Write(buffer)[/PHP]

??
Aug 2 '07 #2

Sign in to post your reply or Sign up for a free account.

Similar topics

5
by: Larry Woods | last post by:
Looking for an example of a multi-threaded component that is being used in ASP. Anyone??? TIA, Larry Woods
0
by: Somen | last post by:
Hello Guys and Gals, Thanks for reading this post. I was wondering if you good folks have had some experience using c-jdbc and how it handles writing to databases. To be more specific - if we...
14
by: SD | last post by:
I am thinking about writing a text editor in C for unix sometime soon. I am just doing this to learn more about C. I want to write something like ed.c, a simple line editor. What types of data...
2
by: Paul Evans | last post by:
Hi, I've created a datagrid which works fine accept the id field (the primary key field) comes out in a random order. How do I get the fields to display sequencially? Thanks for your time ...
7
by: aaj | last post by:
Hi all Would anyone be able to offer any unbiased thoughts or advice on reasons to switch to .Net. I have read some of the literature and read other stuff on google, but a so much of it seems to...
5
by: Uwe C. Schroeder | last post by:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Hi, maybe my mind is stuck, but here's something strange. This is the classic "counter" thing, where you can't / won't use sequences....
12
by: Benny Raymond | last post by:
I understand that you would normally want to raise an event on the same thread so that your code blocks until the event has been dealt with... however i've run into a situation where I have a...
1
by: Lyners | last post by:
I am trying to figure out the best way to do this (currently I am having a problem sorting). I have a vb.net program that contains 2 datagrids on a form for the end user. When the user is ready to...
10
by: rola248 | last post by:
hi, I have this code,why is the displayed result "false"? #include <stdio.h> #include <stdlib.h> #include <string.h> int main() { char name="MEHMET"; /* 6 characters */ char...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.