Hi all....
I need to work with large text files....i know there is a lot of ways under
net to work with them....
i want speed, High Percentage of correctness
so i was thinking ...but not sure the best way to handle text files for
parsing into reports, or into SqlDatabase
or Where Ever else
my question is;
I can build a class which i did under win32 low level file io that only
stores the line offsets in a Array, which is quit fast i was able to load a
million plus line file in less than 6 seconds , so most files that are only
in the megabytes were a blink of a eye. then the class wrapper moves
pointers (not the data on disk) i only accessed disk when data was required,
worked pretty good.
under net i can load the whole file into a single dim array
i can calculate the array dim prior to load so there are only 2 sizing , the
creation of the array and at the end of load, sizing down to what i actually
used...
which is the best way to handle the files in .NET
Just load the The Array/collection with the rows in the file
or
Just load the line offsets into a array and process rows as required
Tks MJ