"Jarry" <Ha***********@gmail.comwrote in message
news:11*********************@b28g2000cwb.googlegro ups.com...
>I have two arrays to load, taking one line to an entry from two 770,000
line files: so I have two arrays of 770000. But this can take upwards
of 5 minutes, which is simply too long. I use a stream reader to
readline from the textfiles, using the syntax
For i = 1 to 770000
myArray(i) = myStreamReader.readLine()
Next
How can I speed this process up, but try to keep the arrays that size?
All ideas welcomed.
Thanks,
Jarry
I had a similar problem when implementing a comprehensive "log file" class
for my software. When in Verbose mode, the logs can easily expand to over
100mb pretty quickly. The solution I came up with was to write each line of
the log file fixed width, i.e. say, 128 characters in width. Then I don't
have to "readline" (which needs to scan for a newline character), I can just
suck in 128 characters at a time, or any multiple of 128 characters. The
resulting log file is larger than it would have been without fixed width,
but I can easily display it using Virtual Mode in a list view pretty much in
real-time.
If I were you, I would at least read a large chunk of the file at a time and
do your processing in memory rather than from disk. Get your stream reader
to read in 10mb at a time into a byte array and then split the byte array by
newline into your "myArray". You'll have tricky cases around the edges of
each chunk but it will be an order of magnitude quicker to do it that way.