473,385 Members | 1,429 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

Committing Byte Array to Disk

What is the fastest way of writing a Byte array to disk? I have a 512
MB array in memory. I am writing to a 10K RPM drive, and have ample CPU
and RAM. I need to write as much data as possible, as fast as possible.
Currently, I am using the StreamWriter. I am not getting anywhere near
the thru put I would expect. My code is something like:

For i = 0 To (MB.Length - 1)
StreamWriter.Write(MB(i))
StreamWriter.Flush()
Next

Any recommendations on squeezing more performance?

J Wolfgang Goerlich

Mar 22 '06 #1
5 4837
guy
suggest you move the StreamWriter.Flush outside the loop

"jw********@gmail.com" wrote:
What is the fastest way of writing a Byte array to disk? I have a 512
MB array in memory. I am writing to a 10K RPM drive, and have ample CPU
and RAM. I need to write as much data as possible, as fast as possible.
Currently, I am using the StreamWriter. I am not getting anywhere near
the thru put I would expect. My code is something like:

For i = 0 To (MB.Length - 1)
StreamWriter.Write(MB(i))
StreamWriter.Flush()
Next

Any recommendations on squeezing more performance?

J Wolfgang Goerlich

Mar 22 '06 #2
Use BinaryWriter instead.

bw.write(MB)

"guy" <gu*@discussions.microsoft.com> wrote in message
news:2D**********************************@microsof t.com...
suggest you move the StreamWriter.Flush outside the loop

"jw********@gmail.com" wrote:
What is the fastest way of writing a Byte array to disk? I have a 512
MB array in memory. I am writing to a 10K RPM drive, and have ample CPU
and RAM. I need to write as much data as possible, as fast as possible.
Currently, I am using the StreamWriter. I am not getting anywhere near
the thru put I would expect. My code is something like:

For i = 0 To (MB.Length - 1)
StreamWriter.Write(MB(i))
StreamWriter.Flush()
Next

Any recommendations on squeezing more performance?

J Wolfgang Goerlich

Mar 22 '06 #3
Try writing the whole array at one time.
I am not sure if it is faster but it works for me.

Public StatusData(255) As Byte

To Save:
Dim fnum As Integer = FreeFile()
FileOpen(fnum, Application.StartupPath & "\Status.dat",
OpenMode.Binary)
FilePut(fnum, StatusData)
FileClose(fnum)

To Open:
Dim fnum As Integer = FreeFile()
FileOpen(fnum, Application.StartupPath & "\Status.dat",
OpenMode.Binary)
FileGet(fnum, StatusData)
FileClose(fnum)

Mar 22 '06 #4
>

Currently, I am using the StreamWriter. I am not getting anywhere near
the thru put I would expect. My code is something like:

For i = 0 To (MB.Length - 1)
StreamWriter.Write(MB(i))
StreamWriter.Flush()
Next

Any recommendations on squeezing more performance?


Close all other programs which have disk access.

Cor
Mar 22 '06 #5
Much obliged for all the help! I am getting closer. StreamWriter takes
around 60 minutes. FilePut takes about two minutes (and really uses the
CPU!). BinaryWriter takes from 30-60 seconds. This is quite a
performance boost.

Yet, even at 30-seconds or 136 Mb/s, I am getting around half the thru
put from the disks that I expected. I suspect it is time to dig into
this from a hardware perspective.

Thanks again,

J Wolfgang Goerlich

Mar 22 '06 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Sathyaish | last post by:
I am using MCI (winmm.dll) to read, record and playback sound. For now, I am doing this with disk files instead of realtime doing it straight from the memory. If I want to stream/relay/transmit...
1
by: Greg Patrick | last post by:
My problem: I load an some assemblies (strong named) from a byte array using Assembly.Load(byte). They load fine. But one one of them is actually accessed, it's referenced assemblies can't be...
2
by: Dunc | last post by:
Hi, I've got an HttpWebResponse object, all working fine. I'm getting the underlying stream and currently writing it to a file using the ReadToEnd() method. All happy. I want to do a bit of...
3
by: Fireangel | last post by:
I want to cast a class into a byte array. I've seen some examples of this floating around, but they all have simple data members. What happens if I cast something that has a ArrayList or an...
1
by: Taiwo | last post by:
I generated a Typed Dataset class including a base64Binary column. This column was specified as a .NET type of Byte() in the class that was auto-generated. I set the value of this property to New...
4
by: Richard506 | last post by:
If you take this Byte Arra Dim bytes() As Byte = { 207, 224, 135, 161, 253, 233, 111, 110, 99, 111, 100, 105, 110, 103, 32, 69, 120, 97, 109, 112, 108, 101 and write the byte array to disk...
9
by: Charles Law | last post by:
Suppose I have a structure Private Structure MyStruct Dim el1 As Byte Dim el2 As Int16 Dim el3 As Byte End Structure I want to convert this into a byte array where
11
by: Garth Wells | last post by:
I've got code that allows me to render the contents of a byte array to the screen, but I need to write the contents to disk. Suggestions? Thanks
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.