processing a very large file[100+MB]. THe processing taking place here is
on 30-50MB chunks of the file.
Performance is extremely important here.
+ I initialise a StringBuilder object with the result of the function
System.Text.Encoding.GetString to convert the byte[] input to string.
[potentially gobbling up an extraordinary amount of RAM]
myEncoding.getString(fileInput);
+ The StringBuilder is converted to a string, capitalisation taken
out[ToLower], and split[String.Split()] on the new line character '\n' which
is placed in a string array.
fileLines = SB.ToString.ToLower().Split(newline);
At the moment this is taking up to 40-50% of the programs running time to
perform, I'd really like to get this down as low as possible.
Im guessing not converting to a Byte array would save quite a bit of time,
as well as finding a better way to split the string into lines. But I've had
no luck at all finding anything.
Thankyou.
