String Building

[Giles]

When assembling an HTML string from a database before sending it to the
client (ie keeping the connection as short as possible), adding to an
existing string (strOut=strOut & strNextLine) takes time and can defeat the
object.
Is putting each line into an array, and re-dimming the array by +1 each time
quicker? Is there a better way?
Thanks
Giles

[Bob Barrows [MVP]]

Giles wrote:

When assembling an HTML string from a database before sending it to
the client (ie keeping the connection as short as possible), adding
to an existing string (strOut=strOut & strNextLine) takes time and
can defeat the object.
Is putting each line into an array, and re-dimming the array by +1
each time quicker? Is there a better way?

Depends on the size of the string. For large strings, putting the fragments
into an array and using Join to create the string will usually be much
quicker than concatenation.
However, you are much better off initially dimming the array to the size it
needs to be rather than redimming it every time a string fragment is added,
even if you initially dim it to a size that is larger than what you need.
Repeating the ReDim operation for every string fragment will undo all the
gains you achieved when you eliminated the concatenation.

Bob Barrows
--
Microsoft MVP -- ASP/ASP.NET
Please reply to the newsgroup. The email account listed in my From
header is my spam trap, so I don't check it very often. You will get a
quicker response by posting to the newsgroup.

[Bob Barrows [MVP]]

Giles wrote:

When assembling an HTML string from a database before sending it to
the client (ie keeping the connection as short as possible), adding
to an existing string (strOut=strOut & strNextLine) takes time and
can defeat the object.
Is putting each line into an array, and re-dimming the array by +1
each time quicker? Is there a better way?
Thanks
Giles

And if you're really interested in minimizing connection time, look into
using GetRows and GetString:
http://www.aspfaq.com/show.asp?id=2467
--
Microsoft MVP -- ASP/ASP.NET
Please reply to the newsgroup. The email account listed in my From
header is my spam trap, so I don't check it very often. You will get a
quicker response by posting to the newsgroup.

[Dave Anderson]

Giles wrote:

Is putting each line into an array, and re-dimming the array
by +1 each time quicker? Is there a better way?

Yes. Array.push() is especially convenient for avoiding this kind of
nonsense.

But I suppose you mean in VBScript. I suspect, as Bob did, that one of the
recordset methods will do what you need.
--
Dave Anderson

Unsolicited commercial email will be read at a cost of $500 per message. Use
of this email address implies consent to these terms.

[Anthony Jones]

"Giles" <gi***@nospam.com> wrote in message
news:Ox*************@TK2MSFTNGP02.phx.gbl...

When assembling an HTML string from a database before sending it to the
client (ie keeping the connection as short as possible), adding to an
existing string (strOut=strOut & strNextLine) takes time and can defeat the object.
Is putting each line into an array, and re-dimming the array by +1 each time quicker? Is there a better way?
Thanks
Giles

I guess at the end of all that you would do:-

Response.Write strOut

So why not replace:-

strOut = strOut & strNextLine

with:-

Response.Write strNextLine

Using GetRows as Bob suggests is what will minimise connection time.
(Not that I'm a big fan of GetRows myself <bg> )

Anthony.

[Giles]

> "Giles" <gi***@nospam.com> wrote in message

When assembling an HTML string from a database before sending it to the
client (ie keeping the connection as short as possible), adding to an
existing string (strOut=strOut & strNextLine) takes time and can defeat the

object.
Is putting each line into an array, and re-dimming the array by +1 each

time

quicker? Is there a better way?
Thanks
Giles

Anthony wrote:
So why not replace:-
strOut = strOut & strNextLine
with:-
Response.Write strNextLine

Doesn't response.write wait for the client to "acknowledge receipt" ? so a
slow internet connection could keep the db connection open for a long time -
like really ages with some modems / isp's? My server-fuhrer threatened me
with severe whipping for doing that, which is why I changed to making a
string first, closing the conn, then dumping the string. However, now I am
tying up the processor (can't win!).
Bob's suggestion about dimming first (perhaps with a COUNT to ensure the
right size) and GetRows/GetString seems like a top banana to me

[Dave Anderson]

Giles wrote:

Doesn't response.write wait for the client to "acknowledge
receipt" ? so a slow internet connection could keep the db
connection open for a long time - like really ages with
some modems / isp's?
No.

Response.Write actually puts the text into the Response Buffer, which is
flushed (sent to the client) once the entire document is assembled.
Connection speed is irrelevant.

You might be able to imapct performance by turning off buffering or using
explicit Response.Flush calls, but those are atypical things to do.
Bob's suggestion about dimming first (perhaps with a COUNT
to ensure the right size) and GetRows/GetString seems like
a top banana to me

If the recordset is big enough that you are noticing this, use GetRows or
GetString. They are orders of magnitude faster than VBScript arrays and/or
string concatenation.
--
Dave Anderson

Unsolicited commercial email will be read at a cost of $500 per message. Use
of this email address implies consent to these terms.

[Anthony Jones]

"Giles" <gi***@nospam.com> wrote in message
news:eX**************@TK2MSFTNGP05.phx.gbl...

"Giles" <gi***@nospam.com> wrote in message

When assembling an HTML string from a database before sending it to the
client (ie keeping the connection as short as possible), adding to an
existing string (strOut=strOut & strNextLine) takes time and can

defeat the

object.
Is putting each line into an array, and re-dimming the array by +1 each time

quicker? Is there a better way?
Thanks
Giles

Anthony wrote:
So why not replace:-
strOut = strOut & strNextLine
with:-
Response.Write strNextLine

Doesn't response.write wait for the client to "acknowledge receipt" ? so a
slow internet connection could keep the db connection open for a long

time - like really ages with some modems / isp's? My server-fuhrer threatened me
with severe whipping for doing that, which is why I changed to making a
string first, closing the conn, then dumping the string. However, now I am
tying up the processor (can't win!).

Like Dave said with buffering turned on then Response.Write doesn't actually
involve actually sending anything. So using Response.Write will avoid the
inefficient allocation/dealloaction of memory that string concatenation
incurrs.
Bob's suggestion about dimming first (perhaps with a COUNT to ensure the
right size) and GetRows/GetString seems like a top banana to me

An initial Redim of the array is unnecessary GetRows returns an array of the
correct dimensions. Hence if performance is paramount then GetRows coupled
with techniques that avoid excessive string concatenation (such as the Join
function or Response.Write) are what you will be needing.

Anthony.

[Bob Barrows [MVP]]

Anthony Jones wrote:

An initial Redim of the array is unnecessary GetRows returns an array
of the correct dimensions. Hence if performance is paramount then
GetRows coupled with techniques that avoid excessive string
concatenation (such as the Join function or Response.Write) are what
you will be needing.

Oops, you lost sight of the original problem, which was creating a large
html string. GetRows will quickly get the data into an array, upon which he
can close the connection and process the data in the array. I don't believe
that Join can be used on a multidimensional array ...
If he is now going to build a large string using the data in the getrows
array, then yes, an array initially dimmed to the proper size may still be
the best solution.

However, I am going along with your (and Dave's) earlier suggestion to
simply response.write the string fragments built from looping through the
getrows array, rather than concatenating them before writing them. I was not
sure about the real reason for creating the large string when I wrote my
initial response.

--
Microsoft MVP - ASP/ASP.NET
Please reply to the newsgroup. This email account is my spam trap so I
don't check it very often. If you must reply off-line, then remove the
"NO SPAM"

[Giles]

"Bob Barrows [MVP]" wrote <snip>

However, I am going along with your (and Dave's) earlier suggestion to
simply response.write the string fragments built from looping through the
getrows array, rather than concatenating them before writing them. I was
not sure about the real reason for creating the large string when I wrote
my initial response.

Thanks everyone, insights much appreciated. I guess I had been misinformed
about response.write, and had been making life unnecessarily difficult for
myself.
Giles

[Dave Anderson]

Giles wrote:

Thanks everyone, insights much appreciated. I guess I had
been misinformed about response.write, and had been making
life unnecessarily difficult for myself.

If you are still concerned about keeping connections open, keep in mind that
you can do any of these things (Response.Write, rs.GetRows, rs.GetString,
even the dreaded concatenation) with disconnected recordsets. That will ease
the concerns of your "server-fuhrer".

--
Dave Anderson

Unsolicited commercial email will be read at a cost of $500 per message. Use
of this email address implies consent to these terms.

[Justin Piper]

On Tue, 16 May 2006 13:00:01 -0500, Giles <gi***@nospam.com> wrote:

When assembling an HTML string from a database before sending it to the
client (ie keeping the connection as short as possible), adding to an
existing string (strOut=strOut & strNextLine) takes time and can defeat
the
object.
Is putting each line into an array, and re-dimming the array by +1 each
time
quicker? Is there a better way?

I ran some profiling tests of all the ways I could think of offhand to
assemble a string. Using the concatenation operator (&) is by far the
slowest. In my tests this method took over a hundred times as long as
using the Join() function on a statically-dimensioned array.

Here are the results I got:

Method Time
----------------------------------------- ------------
Concatenation operator 51.1457s
Write to ADODB.Stream object 2.582813s
Join Scripting.Dictionary Items() array 0.2144531s
Join dynamically-dimensioned array 0.0609375s
Join statically-dimensioned array 0.046875s

Note that I doubled the size of the dynamically-dimensioned array each
time it filled. Increasing the array's size by one element at a time would
perform considerably worse.

I am surprised by how poorly the Stream object fared. I don't use it very
often, so perhaps I am using it sub-optimally. I have included the script
I used in my tests below.

Option Explicit

Function Prof(f)
Dim start, finish, i, total
total = 0
For i = 0 To 9
start = Timer()
f
finish = Timer()
total = total + finish - start
Next
Prof = total/10
End Function

Sub ProfConcat()
Dim i, s
s = String(100, "s")
For i = 1 To 10000
s = s & String(100, "s")
Next
End Sub
WScript.Echo "Concat" & vbTab & Prof(GetRef("ProfConcat")) & "s"

Sub ProfDict()
Dim i, s, d
Set d = CreateObject("Scripting.Dictionary")
For i = 1 To 10000
d.Add d.Count, String(100, "s")
Next
s = Join(d.Items(), "")
End Sub
WScript.Echo "Dict" & vbTab & Prof(GetRef("ProfDict")) & "s"

Sub ProfDim()
Dim i, s, a(10000)
For i = 1 To 10000
a(i) = String(100, "s")
Next
s = Join(a, "")
End Sub
WScript.Echo "Dim" & vbTab & Prof(GetRef("ProfDim")) & "s"

Sub ProfReDim()
Dim i, s, a()
ReDim a(1)
For i = 1 To 10000
If UBound(a) < i Then ReDim Preserve a(i*2)
a(i) = String(100, "s")
Next
s = Join(a, "")
End Sub
WScript.Echo "ReDim" & vbTab & Prof(GetRef("ProfReDim")) & "s"

Sub ProfStream()
Dim i, s, o
Set o = CreateObject("ADODB.Stream")
o.Open
For i = 1 To 10000
o.WriteText String(100, "s")
Next
o.Position = 0
s = o.ReadText()
End Sub
WScript.Echo "Stream" & vbTab & Prof(GetRef("ProfStream")) & "s"
--
Justin Piper
Bizco Technologies
http://www.bizco.com/

[Justin Piper]

On Wed, 17 May 2006 12:28:38 -0500, Justin Piper <jp****@bizco.com> wrote:

In my tests this method took over a hundred times as long as using the
Join() function on a statically-dimensioned array.

Whoops, make that over a /thousand/ times as long.
--
Justin Piper
Bizco Technologies
http://www.bizco.com/

[Anthony Jones]

>

I am surprised by how poorly the Stream object fared. I don't use it very
often, so perhaps I am using it sub-optimally. I have included the script
I used in my tests below.

That'll be due to the encoding the stream object will be performing on the
string. It doesn't just dump the unicode string given into the storage but
encodes it to the currently selected codepage.

Anthony.

[Justin Piper]

On Wed, 17 May 2006 15:59:03 -0500, Anthony Jones <An*@yadayadayada.com>
wrote:

I am surprised by how poorly the Stream object fared.

That'll be due to the encoding the stream object will be performing on
the
string. It doesn't just dump the unicode string given into the storage
but
encodes it to the currently selected codepage.

Ah, that makes sense. In that case, I suspect it doesn't check whether the
text needs to be re-encoded before doing so, as explicitly setting the
locale to "en-us" and the Stream object's codepage to "windows-1252"
didn't have any effect on performance.

--
Justin Piper
Bizco Technologies
http://www.bizco.com/

[Anthony Jones]

"Justin Piper" <jp****@bizco.com> wrote in message
news:op***************@luxembourg.psg.bizcotech.co m...

On Wed, 17 May 2006 15:59:03 -0500, Anthony Jones <An*@yadayadayada.com>
wrote:

I am surprised by how poorly the Stream object fared.
That'll be due to the encoding the stream object will be performing on
the
string. It doesn't just dump the unicode string given into the storage
but
encodes it to the currently selected codepage.

Ah, that makes sense. In that case, I suspect it doesn't check whether the
text needs to be re-encoded before doing so, as explicitly setting the
locale to "en-us" and the Stream object's codepage to "windows-1252"
didn't have any effect on performance.

All strings in ASP are unicode. Setting the Stream object's encoding to
"UTF-16" might make a difference since it shouldn't have to encode the
string.
--
Justin Piper
Bizco Technologies
http://www.bizco.com/

[Justin Piper]

On Thu, 18 May 2006 09:59:06 -0500, Anthony Jones <An*@yadayadayada.com>
wrote:

All strings in ASP are unicode. Setting the Stream object's encoding to
"UTF-16" might make a difference since it shouldn't have to encode the
string.

No such luck. The docs say that "unicode" is the default value of
..Charset, and explicitly setting it to "utf-16" or "utf-8" unfortunately
doesn't yield any performance gains.

--
Justin Piper
Bizco Technologies
http://www.bizco.com/