Suggestions to reduce memory use when splitting a string

[klineb]

Good Day,

I have written and utility to convert our DOS COBOL data files to a SQL
Server database. Part of the process requires parsing each line into a
sql statement and validting the data to keep the integrity of the
database. We are parsing roughl 81 files and range in size 1 kb to 65
MB files (Average of 400,000 lines in the larger files).

I have written this utility with VB.NET 2003 and when I parse all of
the files I run out of memory. The following functions seems to be the
main source of my leak. Any help optimizing this code is appreciated.

Public Function SplitDelimitedL ine(ByVal CurrentLine As StringBuilder,
_
ByRef SplitString() As String) As Boolean
'7-25-2005
'BJK
'
'--removed the use of Char replaced with: CurrentLine.Cha rs(i)
'
'---------------------------------------
Dim i As Integer
Dim CountDelimiter As Boolean
Dim Total As Integer
Dim lbResult As Boolean
Dim Section As New StringBuilder
Dim liLen As Integer
Dim liCommaPos As Integer
Dim liDQuotePos As Integer

Try
'We want to count the delimiter unless it is within the
text qualifier
CountDelimiter = True
Total = 0
liLen = CurrentLine.Len gth - 1
For i = 0 To liLen
Select Case CurrentLine.Cha rs(i)
Case gsDoubleQoute
If CountDelimiter Then
CountDelimiter = False
Else
CountDelimiter = True
End If
Case gsComma
If CountDelimiter Then
' Add current section to collection
SplitString(Tot al) = Section.ToStrin g.Trim
Section = Nothing
Section = New StringBuilder
Total = Total + 1
Else
Section.Append( CurrentLine.Cha rs(i))
End If
Case Else
Section.Append( CurrentLine.Cha rs(i))
End Select
Next
' Get the last field - as most files will not have an
ending delimiter
If CountDelimiter Then
' Add current section to collection
SplitString(Tot al) = Section.ToStrin g
End If
lbResult = True
Catch ex As Exception
ps_LastErrSourc e = ex.Source
ps_LastErrDesc = ex.ToString
lbResult = False
Dim loSB As New StringBuilder(p s_LastErrDesc)
UpdateLog(loSB)
End Try
Return lbResult
End Function

This function is stored in a class and is called from other function
within this class.
Thanks
Brian

[klineb]

Thank you everybody for your suggestions.

This really is not a one time issue. We still have 60 DOS clients to
convert. All of the clients are running different hardware so buying
memory is not really an option. I will try all of you suggestions and
post my results back.

Does anyone know if running some of the class in a seperate process
would help any?

Thanks again for all of the feedback.

Brian