469,642 Members | 1,267 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,642 developers. It's quick & easy.

How to save and read very big Array Value to/from file in VB.NET?

str_Array() 'strings
I need save it to file and next time i can read from file.

--------------------------------------------------------
--------------------------------------------------------
Save Function
I use
str_FromAndToFile=join(str_Array,"VBTAB")
--------------------------------------------------------
Read Function
I use
str_Array=Split(str_FromAndToFile,"VBTAB")
--------------------------------------------------------
--------------------------------------------------------

The problem is more str_Array() huge ,the code run more slowly.

some cost value
(C1.7-512M)
str_Array(i) Read Cost Save Cost.
100000 1.2s 1.5s
1000000 2s 13s
....

Actually I need i=1'000'000'000. :-(
Could anyone help me to improve it or give me some efficiency source code?

THX!

on************@hotmail.com
Nov 20 '05 #1
4 9246
Hi

Why are you not using an arraylist and read/write that as normal records
using the streamreader/streamwriter

http://msdn.microsoft.com/library/de...ClassTopic.asp

http://msdn.microsoft.com/library/de...ClassTopic.asp

http://msdn.microsoft.com/library/de...classtopic.asp

I hope this helps?

Cor
Nov 20 '05 #2
* "=?Utf-8?B?b25jZWxvdmVjb2ZmZWU=?=" <on************@discussions.microsoft.com> scripsit:
I need save it to file and next time i can read from file.


'FileOpen', 'FileGet', 'FilePut', ...

--
Herfried K. Wagner [MVP]
<URL:http://dotnet.mvps.org/>
Nov 20 '05 #3
oncelovecoffee,
In addition to the other comments, I would iterate over the array & write
each element as a line.

To read it back I would simply read each line.

I would consider using an ArrayList as they automatically expand as needed.

Something like:

Imports System.IO
Imports System.Text

Dim str_Array As New ArrayList

Dim input As New StreamReader("myfile.in", Encoding.Default)
Dim line As String = input.ReadLine()
Do Until line Is Nothing
str_Array.Add(line)
line = input.ReadLine()
Loop
Dim output As New StreamWriter("myfile.out", False,
Encoding.Default)

For Each line As String In str_Array
output.WriteLine(line)
Next
Actually I need i=1'000'000'000. :-( However if you really have a billion lines, you may want to consider
upgrading to .NET 64 bit edition, as 32 bit .NET only supports 2G of memory.

http://lab.msdn.microsoft.com/vs2005/

Unfortunately the 64 bit edition is only available as Beta 1 currently, you
will need to wait until sometime in 2005 for the release.

Alternatively consider reading only the line you need to work on and write
it.
Dim input As New StreamReader("myfile.in", Encoding.Default)
Dim output As New StreamWriter("myfile.out", False,
Encoding.Default)
Dim line As String = input.ReadLine()
Do Until line Is Nothing
' do something with line
output.WriteLine(line)
line = input.ReadLine()
Loop

Hope this helps
Jay

"oncelovecoffee" <on************@discussions.microsoft.com> wrote in message
news:5A**********************************@microsof t.com... str_Array() 'strings
I need save it to file and next time i can read from file.

--------------------------------------------------------
--------------------------------------------------------
Save Function
I use
str_FromAndToFile=join(str_Array,"VBTAB")
--------------------------------------------------------
Read Function
I use
str_Array=Split(str_FromAndToFile,"VBTAB")
--------------------------------------------------------
--------------------------------------------------------

The problem is more str_Array() huge ,the code run more slowly.

some cost value
(C1.7-512M)
str_Array(i) Read Cost Save Cost.
100000 1.2s 1.5s
1000000 2s 13s
...

Actually I need i=1'000'000'000. :-(
Could anyone help me to improve it or give me some efficiency source code?

THX!

on************@hotmail.com

Nov 20 '05 #4
At last ,I use the policy "Used, Record it",if one of the array member's value is default .then I don't touched it:-)
:EXAMPLE
0 XXX
1 XXX
2 DEFAULT
3 DEFALUT
..
..
..
999999999 XXX
1000000000 DEFALUT

i record

0 XXX
1 XXX
..
..
999999999 XXX

it reduce the size of file and save time to write file.
the worst conditation is I still should record them all if every value changed
:-(

And good news is 1000 times Rnd test shows about 30% of array members's value changed (AVG). :-)
Nov 20 '05 #5

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

4 posts views Thread by Andras Gilicz | last post: by
1 post views Thread by Andy Jacobs | last post: by
2 posts views Thread by Duy Nguyen | last post: by
3 posts views Thread by Stefano | last post: by
3 posts views Thread by =?Utf-8?B?Um9nZWxpbw==?= | last post: by
11 posts views Thread by Lamer | last post: by
10 posts views Thread by Gilles Ganault | last post: by
reply views Thread by gheharukoh7 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.