Performance of Mid$ and possible alternatives

[Killer42]

Hi all.

I have a string and need to extract various (sometimes overlapping) parts based solely on position, regardless of contents. Does anyone know the fastest possible way to do this, in VB6?

I have been considering whether I might even try building an assembler (DLL) routine, but would prefer not to have to - it has been quite a while since I played with assembler.

I have some code which makes lots of decisions based on various chunks of a string. If I were hitting the same position repeatedly, I would just extract it once to a variable and use that. But I have to keep checking different positions, based on records types. At the moment I'm taking the simple and obvious route of using Mid$ to access each chunk.

What I really probably comes under the heading of unobtainable perfection. In the programming language "Natural", you can "redefine" a field, which simply assigns another variable overlapping the specified portion. It's kind of like a Mid$ which is set up at design time. Absolutely wonderful for this kind of thing, as you don't need to do any sort of function call or calculation at runtime.

I'm doing this for millions upon millions of text records each time I run, so what I'm looking for is some way, any way, to improve on the performance of the Mid$ in chopping up the string.

[kadghar]

I have some code which makes lots of decisions based on various chunks of a string. If I were hitting the same position repeatedly, I would just extract it once to a variable and use that. But I have to keep checking different positions, based on records types. At the moment I'm taking the simple and obvious route of using Mid$ to access each chunk.

I think thats the solution, extract each part of the string to an array and work with the array. It will take you a little time to get that array and it'll be much faster than working with Mid$

I made this little test and the results are below.

Sub compare()
Dim Str1 As String
Dim Str2 As String
Dim mStr1() As String
Dim Sin1 As Single
Dim Sin2 As Single
Dim Sin3 As Single
Dim i As Integer
Dim j As Long
Str1 = "Now i'll write here whatever it comes to my mind just to make this little test."
 
ReDim mStr1(1 To Len(Str1))
'Lets see how long it takes to save it into an array
Sin1 = Timer
    For i = 1 To Len(Str1)
        mStr1(i) = Mid(Str1, i, 1)
    Next
Sin1 = Timer - Sin1
 
'Now lets write the string upside down 1'000'000 times
'and take the time using the old mid$
 
Sin2 = Timer
For j = 1 To 1000000
    Str2 = ""
    For i = Len(Str1) To 1 Step -1
        Str2 = Str2 & Mid(Str1, i, 1)
    Next
Next
Sin2 = Timer - Sin2
 
'and take the time using the array
 
Sin3 = Timer
For j = 1 To 1000000
    Str2 = ""
    For i = Len(Str1) To 1 Step -1
        Str2 = Str2 & mStr1(i)
    Next
Next
Sin3 = Timer - Sin3
 
MsgBox ("Array time: " & Sin1 & Chr(10) & "Mid$ Method: " & Sin2 & Chr(10) & "Array Method: " & Sin3)
 
End Sub

Sin1 = 0.0001875
Sin2 = 40.28319
Sin3 = 18.29994

This wont make your code much faster, but at least will reduce the time of your operations to one half.

Hope that helps.
(oh, and im in the job's computer, i think almost every other pc can make it faster.)

[Killer42]

Thanks kadghar, I'll have a play with this at lunch time.

Hm... if an array of String is faster, an array of Byte might be even faster still.

However, in this case I suspect I may lose too much time copying to the array. You see, the problem is I'm not chopping up the same string very many times. I'm reading millions of records and doing this sort of general processing... (all values here are just made up for illustration purposes, I forget the actual details right now)

strRecordType = Mid$(strRecord, 5, 8)
Select Case strRecordType
  Case TYPE_1 ' (a Const value)
    strDetails1 = Mid$(strRecord, 18, 15)
    strDetails2 = Mid$(strRecord, 56, 4)
  Case TYPE_2 ' (another Const value)
    strDetails1 = Mid$(strRecord, 16, 6) ' Overlaps above 18,15
    strDetails2 = Mid$(strRecord, 50, 2) ' Doesn't overlap anything
  '
  '
  '
End Select
 

So you see, I am selectively grabbing many different chunks, but generally only once per record (based on the type encoded within the record). So the sort of techniques I might normally use, like assigning all of them to variables at the start then using those variables, actually increase the processing in this case.

I don't have much experience with Byte arrays, but I gather they can be faster than strings at times. The problem is that whenever I need to check a value, I expect I'll have to loop through a section of the array, which seems inefficient. I really miss the processing available in Natural. In that (it's predominantly a mainframe language) I could just say IF ARRAY1(10:15) = ARRAY2(20:25) THEN BLAH BLAH.

[kadghar]

I see what you mean, and yes... of course making a loop through each character of a string takes longer than chopping a section only once.

Since you have n different records to chop in m different parts, you'll always have m x n different substrings, and anytime you need one of this values for the first time, you'll have to obtain it (using a mid$ i guess)... i dont see an easy way out because you're not using the any substring that much times.

(but remembering your 39 phrases of wisdom, i'd add a 40th one: dont tell anyone something is imposible while he is doing it)

Good Luck.