By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
443,990 Members | 1,990 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 443,990 IT Pros & Developers. It's quick & easy.

How to save and read very big Array Value to/from file in VB.NET?

P: n/a
str_Array() 'strings
I need save it to file and next time i can read from file.

--------------------------------------------------------
--------------------------------------------------------
Save Function
I use
str_FromAndToFile=join(str_Array,"VBTAB")
--------------------------------------------------------
Read Function
I use
str_Array=Split(str_FromAndToFile,"VBTAB")
--------------------------------------------------------
--------------------------------------------------------

The problem is more str_Array() huge ,the code run more slowly.

some cost value
(C1.7-512M)
str_Array(i) Read Cost Save Cost.
100000 1.2s 1.5s
1000000 2s 13s
....

Actually I need i=1'000'000'000. :-(
Could anyone help me to improve it or give me some efficiency source code?

THX!

on************@hotmail.com
Nov 20 '05 #1
Share this Question
Share on Google+
4 Replies


P: n/a
Hi

Why are you not using an arraylist and read/write that as normal records
using the streamreader/streamwriter

http://msdn.microsoft.com/library/de...ClassTopic.asp

http://msdn.microsoft.com/library/de...ClassTopic.asp

http://msdn.microsoft.com/library/de...classtopic.asp

I hope this helps?

Cor
Nov 20 '05 #2

P: n/a
* "=?Utf-8?B?b25jZWxvdmVjb2ZmZWU=?=" <on************@discussions.microsoft.com> scripsit:
I need save it to file and next time i can read from file.


'FileOpen', 'FileGet', 'FilePut', ...

--
Herfried K. Wagner [MVP]
<URL:http://dotnet.mvps.org/>
Nov 20 '05 #3

P: n/a
oncelovecoffee,
In addition to the other comments, I would iterate over the array & write
each element as a line.

To read it back I would simply read each line.

I would consider using an ArrayList as they automatically expand as needed.

Something like:

Imports System.IO
Imports System.Text

Dim str_Array As New ArrayList

Dim input As New StreamReader("myfile.in", Encoding.Default)
Dim line As String = input.ReadLine()
Do Until line Is Nothing
str_Array.Add(line)
line = input.ReadLine()
Loop
Dim output As New StreamWriter("myfile.out", False,
Encoding.Default)

For Each line As String In str_Array
output.WriteLine(line)
Next
Actually I need i=1'000'000'000. :-( However if you really have a billion lines, you may want to consider
upgrading to .NET 64 bit edition, as 32 bit .NET only supports 2G of memory.

http://lab.msdn.microsoft.com/vs2005/

Unfortunately the 64 bit edition is only available as Beta 1 currently, you
will need to wait until sometime in 2005 for the release.

Alternatively consider reading only the line you need to work on and write
it.
Dim input As New StreamReader("myfile.in", Encoding.Default)
Dim output As New StreamWriter("myfile.out", False,
Encoding.Default)
Dim line As String = input.ReadLine()
Do Until line Is Nothing
' do something with line
output.WriteLine(line)
line = input.ReadLine()
Loop

Hope this helps
Jay

"oncelovecoffee" <on************@discussions.microsoft.com> wrote in message
news:5A**********************************@microsof t.com... str_Array() 'strings
I need save it to file and next time i can read from file.

--------------------------------------------------------
--------------------------------------------------------
Save Function
I use
str_FromAndToFile=join(str_Array,"VBTAB")
--------------------------------------------------------
Read Function
I use
str_Array=Split(str_FromAndToFile,"VBTAB")
--------------------------------------------------------
--------------------------------------------------------

The problem is more str_Array() huge ,the code run more slowly.

some cost value
(C1.7-512M)
str_Array(i) Read Cost Save Cost.
100000 1.2s 1.5s
1000000 2s 13s
...

Actually I need i=1'000'000'000. :-(
Could anyone help me to improve it or give me some efficiency source code?

THX!

on************@hotmail.com

Nov 20 '05 #4

P: n/a
At last ,I use the policy "Used, Record it",if one of the array member's value is default .then I don't touched it:-)
:EXAMPLE
0 XXX
1 XXX
2 DEFAULT
3 DEFALUT
..
..
..
999999999 XXX
1000000000 DEFALUT

i record

0 XXX
1 XXX
..
..
999999999 XXX

it reduce the size of file and save time to write file.
the worst conditation is I still should record them all if every value changed
:-(

And good news is 1000 times Rnd test shows about 30% of array members's value changed (AVG). :-)
Nov 20 '05 #5

This discussion thread is closed

Replies have been disabled for this discussion.