How to share objects in an ASP.NET application ???

[Tony Fonager]

I am currently developing a statistics system in ASP.NET, and need to share
information about the customers websites, in this application.

(I have simplified my code, to make my project easier to explain.)

The simple version of the system is like this : A customer inserts HTML code
on his webpage, which contacts my statistics server each time the customers
website recieves a hit - like a classic "register website traffic" system.

When the customers hit reaches my web application, I need to maintain info
and state about the customers website - therefore I have a "website" class,
which hold information about the customer website (raw hits, unique hits,
website name, hit today ect).

So the first time a customer hits my servers, I need to instantiate an
instance of the website class, and update the information in this class. Next
time I recieve a request for the same customer, I only need to update the
websites information in the instance of that class.

So, here is a picture of how it works today, when the first hit comes in :

1. Customer #100 sends a request to my statistics application.
2. I have no information about customer #100, so I instantiate a new
instance of the "website" class.
3. I update information, and sets the "unique hits" variable to 1 (as this
is the first hit).

After this, all hits coming after the first hit, only updates information in
the "website" class (ie. adds 1 to the "unique hits" variabel) - like in step
3.

I think (and hope) all the above is readable and understandable.

My problem now, is how I share these website classes in the global
application ????

Today I have a shared collection, which hold each instance of the website
class - like this :

public shared websites as new sortedlist

This shared collection can be accessed from all classes in my application,
and actually this works fine.

But, when my collection of websites reaches a certain limit (about 2500
"website" classes), the application starts to become VERY slow, and after
10-15 minutes it totally slows down and becomes useless.

And listen to me - we are talking VERY SLOW here - it runs on dual xeon HP
server, so hardware is enough for testcase ...

My question is now, if I am doing this the right way ??? How are you guys
shared objects among pages in an ASP.NET application ???

Is there are correct way to share objects globally in an ASP.NET application ?
-
Regards,
Tony Fonager, Denmark

[Tony Fonager]

Sorry about the identical post - my mistake!

"Tony Fonager" <fo*****@newsgr oups.nospam> wrote in message
news:14******** *************** ***********@mic rosoft.com...

I am currently developing a statistics system in ASP.NET, and need to share information about the customers websites, in this application.

(I have simplified my code, to make my project easier to explain.)

....

[Bill Borg]

Tony, a few thoughts:

1. If you use a shared variable, you'll get a new instance of that
sortedlist each time IIS spins up a new instance of your app (which it may or
may not decide to do).

2. Tough to tell based on this where your bottleneck is. You might set some
start time/end time breakpoints to see what starts taking so long (e.g.
maintaining the sortedlist, excessive garbage collection due to poor string
handling, etc.). The memory required to store 2500 instances of that class
and the cpu time required to find items in it shouldn't be that
oppressive--I'm guessing the problem is somewhere else (unless you're
supporting a ton of hits/second or are otherwise way underpowered).

3. Regardless, your design isn't built to scale well, because you keep
hanging onto all the web site info. You need a way to flush it periodically
and start over.

4. Why hold onto all the web site information in memory? Why not just write
each "hit" to disk? I presume at some point you have to do that anyway so you
can package it up for your customers. Yes, slightly slower to insert a row
each time, but you'll avoid a whole host of other problems and scalability
should go way up (at least it will be more linear). You could also save up
your writes in cache and persist them when traffic is light, maybe persist
each customers info in a tiny xml file for the day. etc.

5. One way to create a global repository across all instances of your app is
to write a separate web service.

hth,

Bill

"Tony Fonager" wrote:

I am currently developing a statistics system in ASP.NET, and need to share
information about the customers websites, in this application.

(I have simplified my code, to make my project easier to explain.)

The simple version of the system is like this : A customer inserts HTML code
on his webpage, which contacts my statistics server each time the customers
website recieves a hit - like a classic "register website traffic" system.

When the customers hit reaches my web application, I need to maintain info
and state about the customers website - therefore I have a "website" class,
which hold information about the customer website (raw hits, unique hits,
website name, hit today ect).

So the first time a customer hits my servers, I need to instantiate an
instance of the website class, and update the information in this class. Next
time I recieve a request for the same customer, I only need to update the
websites information in the instance of that class.

So, here is a picture of how it works today, when the first hit comes in :

1. Customer #100 sends a request to my statistics application.
2. I have no information about customer #100, so I instantiate a new
instance of the "website" class.
3. I update information, and sets the "unique hits" variable to 1 (as this
is the first hit).

After this, all hits coming after the first hit, only updates information in
the "website" class (ie. adds 1 to the "unique hits" variabel) - like in step
3.

I think (and hope) all the above is readable and understandable.

My problem now, is how I share these website classes in the global
application ????

Today I have a shared collection, which hold each instance of the website
class - like this :

public shared websites as new sortedlist

This shared collection can be accessed from all classes in my application,
and actually this works fine.

But, when my collection of websites reaches a certain limit (about 2500
"website" classes), the application starts to become VERY slow, and after
10-15 minutes it totally slows down and becomes useless.

And listen to me - we are talking VERY SLOW here - it runs on dual xeon HP
server, so hardware is enough for testcase ...

My question is now, if I am doing this the right way ??? How are you guys
shared objects among pages in an ASP.NET application ???

Is there are correct way to share objects globally in an ASP.NET application ?
-
Regards,
Tony Fonager, Denmark

[Tony Fonager]

Bill,

It is not a memory problem (I guess), as my app uses 60 mb, when it totally
slows down ...

I want to update information about each customers website, exactly to leave
out these (slow) SQL updates - thats why I want to store everything in
memory - I then have a seperate thread, that in the background writes back
the information from memory, into a SQL database. So I DO flush data
regularly :-)

I dont understand your comment #1 - what do you mean by that ? IIS should
only have one instance of my app or ?
-
Regards,
Tony Fonager

"Bill Borg" <Bi******@discu ssions.microsof t.com> wrote in message
news:9C******** *************** ***********@mic rosoft.com...

Tony, a few thoughts:

1. If you use a shared variable, you'll get a new instance of that
sortedlist each time IIS spins up a new instance of your app (which it may or may not decide to do).

2. Tough to tell based on this where your bottleneck is. You might set some start time/end time breakpoints to see what starts taking so long (e.g.
maintaining the sortedlist, excessive garbage collection due to poor string handling, etc.). The memory required to store 2500 instances of that class
and the cpu time required to find items in it shouldn't be that
oppressive--I'm guessing the problem is somewhere else (unless you're
supporting a ton of hits/second or are otherwise way underpowered).

3. Regardless, your design isn't built to scale well, because you keep
hanging onto all the web site info. You need a way to flush it periodically and start over.

4. Why hold onto all the web site information in memory? Why not just write each "hit" to disk? I presume at some point you have to do that anyway so you can package it up for your customers. Yes, slightly slower to insert a row
each time, but you'll avoid a whole host of other problems and scalability
should go way up (at least it will be more linear). You could also save up
your writes in cache and persist them when traffic is light, maybe persist
each customers info in a tiny xml file for the day. etc.

5. One way to create a global repository across all instances of your app is to write a separate web service.

hth,

Bill

"Tony Fonager" wrote:

I am currently developing a statistics system in ASP.NET, and need to share information about the customers websites, in this application.

(I have simplified my code, to make my project easier to explain.)

The simple version of the system is like this : A customer inserts HTML code on his webpage, which contacts my statistics server each time the customers website recieves a hit - like a classic "register website traffic" system.
When the customers hit reaches my web application, I need to maintain info and state about the customers website - therefore I have a "website" class, which hold information about the customer website (raw hits, unique hits, website name, hit today ect).

So the first time a customer hits my servers, I need to instantiate an
instance of the website class, and update the information in this class. Next time I recieve a request for the same customer, I only need to update the websites information in the instance of that class.

So, here is a picture of how it works today, when the first hit comes in :
1. Customer #100 sends a request to my statistics application.
2. I have no information about customer #100, so I instantiate a new
instance of the "website" class.
3. I update information, and sets the "unique hits" variable to 1 (as this is the first hit).

After this, all hits coming after the first hit, only updates information in the "website" class (ie. adds 1 to the "unique hits" variabel) - like in step 3.

I think (and hope) all the above is readable and understandable.

My problem now, is how I share these website classes in the global
application ????

Today I have a shared collection, which hold each instance of the website class - like this :

public shared websites as new sortedlist

This shared collection can be accessed from all classes in my application, and actually this works fine.

But, when my collection of websites reaches a certain limit (about 2500
"website" classes), the application starts to become VERY slow, and after 10-15 minutes it totally slows down and becomes useless.

And listen to me - we are talking VERY SLOW here - it runs on dual xeon HP server, so hardware is enough for testcase ...

My question is now, if I am doing this the right way ??? How are you guys shared objects among pages in an ASP.NET application ???

Is there are correct way to share objects globally in an ASP.NET application ?

-
Regards,
Tony Fonager, Denmark

[Steven Cheng[MSFT]]

Hi Tony,

As for the Bill's comment #1, I think he means the asp.net web
application(hos t in its own appdomain) may restart sometimes(for example ,
after some period of idle or configure file/ assembliy changed), and all
the
application scope data(include the static class members will also be
unavailable) after that, new ones will be initialized.

Also, In addition to Static class members, we can also use the
ApplicationStat e or Application Cache to store application level objects.
However, since the asp.net are request/response based and each request
execute in its own workerthread, we need to do concurrency protect on such
global datas. For example, when accessing the ApplicationStat e, we may use
the following statement to protect race condition:

Application.Loc k();
Application["SomeGlobalCoun ter"] =
(int)Applicatio n["SomeGlobalCoun ter"] + 1;
Application.UnL ock();

here is the reference on ASP.NET applicationStat e
#Application State
http://msdn.microsoft.com/library/de...us/cpguide/htm
l/cpconapplicatio nstate.asp

ASP.NET cache is mainly focus on provide cacheing some large and not
frequently changed data, so seems not very useful for your scenario.

In addition, what's the server's OS version? And is the memory useage you
mentioned the asp.net process's memory usage? If the memory is not the
problem, I suspect whether there are any concurency or deadlock problem
since the static variables is accessable throughout the whole application.

Thanks.

Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)

[Tony Fonager]

Steven,

Thanks for your answer ...

I am afraid that my load of static classes is becoming to much for ASP.NET,
even though I am not glad to accept this idea :-)

I have profiled my entire application, and even though I request a page,
which takes maybe 30 seconds to return, the code I have in that particular
page, takes miliseconds to execute, according to the profiler - what happens
for the rest of the 30 seconds, must be inside the ASP.NET framework, which
I cannot profile.

But I am sad to see this - my application is a rewrite of a running JSP
(java) system, and the exact same principles are taken from the java app to
the asp.net app - in java it is running on a single CPU server, and my tests
are running on dual xeon 3 mhz HP server, with tons of memory ... and it
cannot even take 1/5 th of the load as the java version.

I must be doing something totally wrong ....
-
Regards,
Tony

[Steven Cheng[MSFT]]

Hi Tony,

Thanks for your response. Based on your further description, it seems that
there are something being blocked when your page's code executed, since it
'll take about 30 seconds more to return the response.

Since you mentioned that there is a background thread which will regularly
write the inmemory datas into database. Will the accessing to the static
objects in the background thread cause the page's request/response be
blocked? I think you can try stopping the background to see whether the
things will be different.
Also, if you feel it possible , you can also try using the ApplicationStat e
instead of the static class member to store the inmemory datas to see
whether it helps.

In addition, here is a list that shows some asp.net process model related
settings for both IIS5 and IIS6, there're some entries related to the
processs's memory / deadlock time limiation, you can also try adjusting
them to see whether helps.

#Mapping ASP.NET Process Model Settings to IIS 6.0 Application Pool Settings
http://msdn.microsoft.com/library/en...etprocessmodel
settingequivale ncetoapplicatio npoolsettings.a sp?frame=true

Thanks.
Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)

[Tony Fonager]

Steven,

Problem IS solved - how am I going to tell you this - it is embarrassing ;-)

(It was NOT my background thread - it has been disabled all thru the entire
test, so ...)

Here it comes : The slowdown was caused by ENORMOUS viewstate data in the
local browser - actually, when I clicked around in my manager interface
(aspx page), it started slowing down after I had visited a particular page,
with lots and lots of information ...

Information which got posted into the viewstate, as it was bound to a
servercontrol (label) - so what actually happend, was that suddently all my
pages was "carrying aound" a 5-6 MB viewstate, which totally made the
manager interface "non responsive", ..

Arrggggh, I feel like such a beginner :-(

Anyway, on the other side I now know that I can trust .NET again - and the
speed of .NET, as my application is running on lightspeed today, with over
5000 customers in the system - wuhuuu!

Thanks to everybody for their kind help!!!
-
Regards,
Tony

[Steven Cheng[MSFT]]

Hi Tony,

Thanks for your followup. Glad that you've figured out the problem(
especially that you've trust .NET again -:)). Anyway, since the ASP.NET
provide so many new rich features, we may also be careful when using them.
If you meet any other problems on ASP.NET later, please always feel free to
post here.
Thanks again for choosing Microsoft :)
Regards,

Steven Cheng
Microsoft Online Support

Get Secure! www.microsoft.com/security
(This posting is provided "AS IS", with no warranties, and confers no
rights.)