Abysmal performance of Scripting.Dictionary object on 2003 server

[Rodd Snook]

I have an application which makes extensive use of the
Scripting.Dictionary object. I'm not doing anything silly like putting
them outside the page scope -- just creating quite a few of them and
stuffing quite a lot of data (from and MS SQL database) into them.

On Windows 2000 server, everything is fine. If the data structures get
really big it slows down, but for normal operation it's no problem.

Recently our hosting provider moved to Windows Server 2003, and everything
went to the dogs. Pages that used to load in 2-3 seconds can now take in
excess of 30 seconds to load. I've done extensive profiling, and it seems
that the dictionary is to blame. Specifically, putting data into them.
Creating them is fine, getting results from the database is fine, adding
the key/value pairs is a nightmare.

The extreme difference in performance makes me think that the server is
out of memory and thrashing, but the provider insists that there is
plenty of free memory, and that my application is the only one with a
problem. Maybe I'm wrong, and it's a data marshalling issue, not a memory
allocation issue.

The paranoid side of me says that ASP is intentionally "unoptimised" in
2003 to "encourage" people to move to .NET.

Has anyone else seen this, or have any ideas what causes it?

Rodd

[Ken Schaefer]

Are you adding a lot of key/value pairs? Or adding a lot of data for a few
pairs?

I also use Scripting.Dictionary in several Classic ASP applications (mostly
for holding a list of potential user errors, and then accessing them when
producing "error summary" for the user), both on Windows 2000 and Windows
2003, and have noticed no perceptable difference. However, I'm not adding
much data (just a short error message), though potentially might add 25-30
key/value pairs.

Cheers
Ken

"Rodd Snook" <us****@nospam.snookums.port5.com> wrote in message
news:pa****************************@nospam.snookum s.port5.com...
: I have an application which makes extensive use of the
: Scripting.Dictionary object. I'm not doing anything silly like putting
: them outside the page scope -- just creating quite a few of them and
: stuffing quite a lot of data (from and MS SQL database) into them.
:
: On Windows 2000 server, everything is fine. If the data structures get
: really big it slows down, but for normal operation it's no problem.
:
: Recently our hosting provider moved to Windows Server 2003, and everything
: went to the dogs. Pages that used to load in 2-3 seconds can now take in
: excess of 30 seconds to load. I've done extensive profiling, and it seems
: that the dictionary is to blame. Specifically, putting data into them.
: Creating them is fine, getting results from the database is fine, adding
: the key/value pairs is a nightmare.
:
: The extreme difference in performance makes me think that the server is
: out of memory and thrashing, but the provider insists that there is
: plenty of free memory, and that my application is the only one with a
: problem. Maybe I'm wrong, and it's a data marshalling issue, not a memory
: allocation issue.
:
: The paranoid side of me says that ASP is intentionally "unoptimised" in
: 2003 to "encourage" people to move to .NET.
:
: Has anyone else seen this, or have any ideas what causes it?
:
: Rodd
:

[Michael D. Kersey]

Ken Schaefer wrote:

Are you adding a lot of key/value pairs? Or adding a lot of data for a few
pairs?

I also use Scripting.Dictionary in several Classic ASP applications (mostly
for holding a list of potential user errors, and then accessing them when
producing "error summary" for the user), both on Windows 2000 and Windows
2003, and have noticed no perceptable difference. However, I'm not adding
much data (just a short error message), though potentially might add 25-30
key/value pairs.

Cheers
Ken

"Rodd Snook" <us****@nospam.snookums.port5.com> wrote in message
news:pa****************************@nospam.snookum s.port5.com...
: I have an application which makes extensive use of the
: Scripting.Dictionary object. I'm not doing anything silly like putting
: them outside the page scope -- just creating quite a few of them and
: stuffing quite a lot of data (from and MS SQL database) into them.
:
: On Windows 2000 server, everything is fine. If the data structures get
: really big it slows down, but for normal operation it's no problem.
:
: Recently our hosting provider moved to Windows Server 2003, and everything
: went to the dogs. Pages that used to load in 2-3 seconds can now take in
: excess of 30 seconds to load. I've done extensive profiling, and it seems
: that the dictionary is to blame. Specifically, putting data into them.
: Creating them is fine, getting results from the database is fine, adding
: the key/value pairs is a nightmare.
:
: The extreme difference in performance makes me think that the server is
: out of memory and thrashing, but the provider insists that there is
: plenty of free memory, and that my application is the only one with a
: problem. Maybe I'm wrong, and it's a data marshalling issue, not a memory
: allocation issue.
:
: The paranoid side of me says that ASP is intentionally "unoptimised" in
: 2003 to "encourage" people to move to .NET.
:
: Has anyone else seen this, or have any ideas what causes it?
:
: Rodd

Scripting.Dictionary object had (has?) some threading issues; Eric
Lippert of Microsoft wrote it while learning about threading:
http://blogs.msdn.com/ericlippert/ar.../19/53054.aspx

There he talks about problems that can occur when using that object.

Possible cures:
There is a component, called "Lookup Table object", written by Microsoft
programmers who wanted better performance than the Dictionary object
provided. Perhaps you could use it.

BTW why use the Scripting.Dictionary object when you have Session and
Application variables available?

Good Luck,
Michael D. Kersey

[Ken Schaefer]

"Michael D. Kersey" <md******@hal-pc.org> wrote in message
news:ug**************@TK2MSFTNGP10.phx.gbl...
: BTW why use the Scripting.Dictionary object when you have Session and
: Application variables available?

Because you might run into scoping issues. Session/Application objects are
global, and there is no way to make them "local" (or reduce their scope in
any way).

If I use a Script.Dictionary component, then I can limit the scope of that
component (eg to just a routine, or a page). I do not need to worry about
accidently over-writing some value on another page, or in another routine.

Also, I'm wary of adding lots of unnecessary stuff to Session or Application
objects. That makes your app unportable. I have plenty of applications that
I've having issues porting to .NET (or adding new features using .NET)
because of excessive overuse of Session variables, which makes it difficult
to run part of the app using ASP, and part using ASP.NET

Cheers
Ken
"Michael D. Kersey" <md******@hal-pc.org> wrote in message
news:ug**************@TK2MSFTNGP10.phx.gbl...
: Ken Schaefer wrote:
: > Are you adding a lot of key/value pairs? Or adding a lot of data for a
few
: > pairs?
: >
: > I also use Scripting.Dictionary in several Classic ASP applications
(mostly
: > for holding a list of potential user errors, and then accessing them
when
: > producing "error summary" for the user), both on Windows 2000 and
Windows
: > 2003, and have noticed no perceptable difference. However, I'm not
adding
: > much data (just a short error message), though potentially might add
25-30
: > key/value pairs.
: >
: > Cheers
: > Ken
: >
: > "Rodd Snook" <us****@nospam.snookums.port5.com> wrote in message
: > news:pa****************************@nospam.snookum s.port5.com...
: > : I have an application which makes extensive use of the
: > : Scripting.Dictionary object. I'm not doing anything silly like putting
: > : them outside the page scope -- just creating quite a few of them and
: > : stuffing quite a lot of data (from and MS SQL database) into them.
: > :
: > : On Windows 2000 server, everything is fine. If the data structures get
: > : really big it slows down, but for normal operation it's no problem.
: > :
: > : Recently our hosting provider moved to Windows Server 2003, and
everything
: > : went to the dogs. Pages that used to load in 2-3 seconds can now take
in
: > : excess of 30 seconds to load. I've done extensive profiling, and it
seems
: > : that the dictionary is to blame. Specifically, putting data into them.
: > : Creating them is fine, getting results from the database is fine,
adding
: > : the key/value pairs is a nightmare.
: > :
: > : The extreme difference in performance makes me think that the server
is
: > : out of memory and thrashing, but the provider insists that there is
: > : plenty of free memory, and that my application is the only one with a
: > : problem. Maybe I'm wrong, and it's a data marshalling issue, not a
memory
: > : allocation issue.
: > :
: > : The paranoid side of me says that ASP is intentionally "unoptimised"
in
: > : 2003 to "encourage" people to move to .NET.
: > :
: > : Has anyone else seen this, or have any ideas what causes it?
: > :
: > : Rodd
: Scripting.Dictionary object had (has?) some threading issues; Eric
: Lippert of Microsoft wrote it while learning about threading:
: http://blogs.msdn.com/ericlippert/ar.../19/53054.aspx
:
: There he talks about problems that can occur when using that object.
:
: Possible cures:
: There is a component, called "Lookup Table object", written by Microsoft
: programmers who wanted better performance than the Dictionary object
: provided. Perhaps you could use it.
:
: BTW why use the Scripting.Dictionary object when you have Session and
: Application variables available?
:
: Good Luck,
: Michael D. Kersey

[Nikolay]

Probably, solution described in

http://support.microsoft.com/default...b;en-us;840875

Nick

"Rodd Snook" <us****@nospam.snookums.port5.com> wrote in message
news:pa****************************@nospam.snookum s.port5.com...

I have an application which makes extensive use of the
Scripting.Dictionary object. I'm not doing anything silly like putting
them outside the page scope -- just creating quite a few of them and
stuffing quite a lot of data (from and MS SQL database) into them.

On Windows 2000 server, everything is fine. If the data structures get
really big it slows down, but for normal operation it's no problem.

Recently our hosting provider moved to Windows Server 2003, and everything
went to the dogs. Pages that used to load in 2-3 seconds can now take in
excess of 30 seconds to load. I've done extensive profiling, and it seems
that the dictionary is to blame. Specifically, putting data into them.
Creating them is fine, getting results from the database is fine, adding
the key/value pairs is a nightmare.

The extreme difference in performance makes me think that the server is
out of memory and thrashing, but the provider insists that there is
plenty of free memory, and that my application is the only one with a
problem. Maybe I'm wrong, and it's a data marshalling issue, not a memory
allocation issue.

The paranoid side of me says that ASP is intentionally "unoptimised" in
2003 to "encourage" people to move to .NET.

Has anyone else seen this, or have any ideas what causes it?

Rodd

[Rodd Snook]

On Wed, 26 May 2004 11:54:36 +0400, Nikolay wrote:

Probably, solution described in

http://support.microsoft.com/default...b;en-us;840875

Nick

Well, the performance hit isn't writing to the network because all my
responses are buffered, and I've done profiling which points the finger
squarely at the dictionary object. This article about RDS sounds closer
http://support.microsoft.com/default.aspx?kbid=555093
but I'm not using RDS (which goes over HTTP, hence the problem)

However, it sounds like 2003 server has some serious issues. I'm going to
move the application to another hosting provider who can supply win2k
boxes.

Rodd

[Rodd Snook]

On Tue, 25 May 2004 09:13:33 -0500, Michael D. Kersey wrote:

Ken Schaefer wrote:

Are you adding a lot of key/value pairs? Or adding a lot of data for a few
pairs?

I also use Scripting.Dictionary in several Classic ASP applications (mostly
for holding a list of potential user errors, and then accessing them when
producing "error summary" for the user), both on Windows 2000 and Windows
2003, and have noticed no perceptable difference. However, I'm not adding
much data (just a short error message), though potentially might add 25-30
key/value pairs.

Cheers
Ken

"Rodd Snook" <us****@nospam.snookums.port5.com> wrote in message
news:pa****************************@nospam.snookum s.port5.com...
: I have an application which makes extensive use of the
: Scripting.Dictionary object. I'm not doing anything silly like putting
: them outside the page scope -- just creating quite a few of them and
: stuffing quite a lot of data (from and MS SQL database) into them.
:
: On Windows 2000 server, everything is fine. If the data structures get
: really big it slows down, but for normal operation it's no problem.
:
: Recently our hosting provider moved to Windows Server 2003, and everything
: went to the dogs. Pages that used to load in 2-3 seconds can now take in
: excess of 30 seconds to load. I've done extensive profiling, and it seems
: that the dictionary is to blame. Specifically, putting data into them.
: Creating them is fine, getting results from the database is fine, adding
: the key/value pairs is a nightmare.
:
: The extreme difference in performance makes me think that the server is
: out of memory and thrashing, but the provider insists that there is
: plenty of free memory, and that my application is the only one with a
: problem. Maybe I'm wrong, and it's a data marshalling issue, not a memory
: allocation issue.
:
: The paranoid side of me says that ASP is intentionally "unoptimised" in
: 2003 to "encourage" people to move to .NET.
:
: Has anyone else seen this, or have any ideas what causes it?
:
: Rodd Scripting.Dictionary object had (has?) some threading issues; Eric
Lippert of Microsoft wrote it while learning about threading:
http://blogs.msdn.com/ericlippert/ar.../19/53054.aspx

There he talks about problems that can occur when using that object.

Possible cures:
There is a component, called "Lookup Table object", written by Microsoft
programmers who wanted better performance than the Dictionary object
provided. Perhaps you could use it.

I was aware of the threading issues, and would like to move away from
dictionary objects so that I can more easily cache my data in session and
application scope. However, I have a lot of code that would need to be
rewritten, and no time to do it. Also, the current hosting environment
doesn't allow for the installation on 3rd-party components in most
situations. The code is also part of a product which is used outside of my
controlled hosting environment, and hence needs to be as portable as
possible.

BTW why use the Scripting.Dictionary object when you have Session and
Application variables available?

What Ken said. Plus, I have a good many of them organised in hierarchies
-- i.e. the values in some of the dictionary objects are other dictionary
objects. To answer his original question, I probably have two objects with
about 50 key/value pairs, but the values of each of these are themselves
dictionary objects with about 5-10 key/value pairs. Before you say "Lord!
That's an obscene amount.", I know it, and a lot of it could be cached at
the application level (in some other kind of data structure, or maybe even
as HTML), but the point is that the code performed acceptably on Win2k.

At the moment my preferred solution (apart from moving to another hosting
provider who has Win2k) is to rewrite the whole thing in .NET, or maybe
ASP/JScript or PHP (there, I said it!). Something with native support for
sparse, associative arrays and regular expressions.

Rodd

[Michael D. Kersey]

Rodd Snook wrote:

On Tue, 25 May 2004 09:13:33 -0500, Michael D. Kersey wrote: <snipped>

BTW why use the Scripting.Dictionary object when you have Session and
Application variables available?

What Ken said. Plus, I have a good many of them organised in hierarchies
-- i.e. the values in some of the dictionary objects are other dictionary
objects. To answer his original question, I probably have two objects with
about 50 key/value pairs, but the values of each of these are themselves
dictionary objects with about 5-10 key/value pairs. Before you say "Lord!
That's an obscene amount.", I know it, and a lot of it could be cached at
the application level (in some other kind of data structure, or maybe even
as HTML), but the point is that the code performed acceptably on Win2k.

So a total of ~500-1000 items: not many, although we know nothing of
their content and, if they're built up and then discarded with each
page, that's a lot of overhead.

Are these dictionary objects running at low, medium(pooled) or high
(Isolated) protection level or running remotely?

Are they saved in the Session or Application objects or to files or a
database between page executions, or are they created and rebuilt with
each page execution?
At the moment my preferred solution (apart from moving to another hosting
provider who has Win2k) is to rewrite the whole thing in .NET, or maybe
ASP/JScript or PHP (there, I said it!). Something with native support for
sparse, associative arrays and regular expressions.

That would be Perl ( http://www.ActiveState.com/ ).

[Rodd Snook]

On Fri, 28 May 2004 00:20:47 -0500, Michael D. Kersey wrote:

Rodd Snook wrote:

On Tue, 25 May 2004 09:13:33 -0500, Michael D. Kersey wrote: <snipped>

BTW why use the Scripting.Dictionary object when you have Session and
Application variables available?

What Ken said. Plus, I have a good many of them organised in hierarchies
-- i.e. the values in some of the dictionary objects are other dictionary
objects. To answer his original question, I probably have two objects with
about 50 key/value pairs, but the values of each of these are themselves
dictionary objects with about 5-10 key/value pairs. Before you say "Lord!
That's an obscene amount.", I know it, and a lot of it could be cached at
the application level (in some other kind of data structure, or maybe even
as HTML), but the point is that the code performed acceptably on Win2k.

So a total of ~500-1000 items: not many, although we know nothing of
their content and, if they're built up and then discarded with each
page, that's a lot of overhead.

Are these dictionary objects running at low, medium(pooled) or high
(Isolated) protection level or running remotely?

That's a good question, but I'm not sure of the answer. The environment
that's giving problems is a shared server at a hosting provider, so I
can't check on the configuration. I'm also not familiar with the
configuration of Win2k3 and IIS6. Apparently each customer "slot" has it's
own application pool, so I'm guessing that the ASP application is
isolated. The Dictionary object is in-process by default in NT4 and Win2k,
and I would imagine that is also the case on Win2k3, but I may be wrong.

Are they saved in the Session or Application objects or to files or a
database between page executions, or are they created and rebuilt with
each page execution?

They are rebuilt from the database with each page execution. I agree that
this is a lot of overhead, but the application performs acceptably on
Win2k. I would cache them at the application level if it weren't for the
threading limitation of the object.

At the moment my preferred solution (apart from moving to another hosting
provider who has Win2k) is to rewrite the whole thing in .NET, or maybe
ASP/JScript or PHP (there, I said it!). Something with native support for
sparse, associative arrays and regular expressions.

That would be Perl ( http://www.ActiveState.com/ ).

PHP and JScript have these features too, but are semantically closer to
VBScript and thus the porting would be easier.