caching large XML files in an ASP.net application

[spammy]

hi all,

im currently writing an ASP.net front end to display data found in an XML
file. the XML file is quite large (700k or so), and so takes a while to
load. it will be changing around every second, so needs to be loaded at
least at that rate. at the moment, the ASP application loads the xml every
refresh - im thinking that this may cause problems when there are 20 clients
each refreshing once per second.

the solution im thinking of is to have a thread accessible to all the ASP
instances that deals with loading and refreshing the XML into a dataset. it
would do this every second and make the dataset available to all ASP
instances to read from.

how would this work? assuming you can create threads as normal in an ASP
application, how do i make it accessible to all of them? by making it
static? which applicaiton thread instance would be in charge of starting and
stopping the XML reading thread? i mean the first should start it and the
last to exit should quit it, but can this be detected? or is a timeout a
best way to quit it (not accessed in a minute -> quit). is this thread a
particularly efficient way of doing this, or would it put another
unnecessary load on to the webserver?

if i dont use a thread, whats the best way to implement the XML reading? it
would have to be a shared dataset, but only updated once every second - how
would this timekeeping be implemented by the various application threads?

thanks in advance, and sorry for the many questions!

spammy

[Janaka]

Spammy

sorry i haven't tried using any of the Threading classes with ASP.NET,
however you might be able to change some other parts of the application to
solve your problem?

first of all is all the data changing in the xml file? If possible i've
found it easier to split out a large xml file into several smaller files and
use xslt on them to make a kind of view of the combined data.

try seeing if its possible to use the Cache object to store the data
(however if the cache has to be cleared every second i don't think this
would give you a large benefit).

J

"spammy" <[e-mail]> wrote in message
news:2h************@uni-berlin.de...

hi all,

im currently writing an ASP.net front end to display data found in an XML
file. the XML file is quite large (700k or so), and so takes a while to
load. it will be changing around every second, so needs to be loaded at
least at that rate. at the moment, the ASP application loads the xml every
refresh - im thinking that this may cause problems when there are 20 clients each refreshing once per second.

the solution im thinking of is to have a thread accessible to all the ASP
instances that deals with loading and refreshing the XML into a dataset. it would do this every second and make the dataset available to all ASP
instances to read from.

how would this work? assuming you can create threads as normal in an ASP
application, how do i make it accessible to all of them? by making it
static? which applicaiton thread instance would be in charge of starting and stopping the XML reading thread? i mean the first should start it and the
last to exit should quit it, but can this be detected? or is a timeout a
best way to quit it (not accessed in a minute -> quit). is this thread a
particularly efficient way of doing this, or would it put another
unnecessary load on to the webserver?

if i dont use a thread, whats the best way to implement the XML reading? it would have to be a shared dataset, but only updated once every second - how would this timekeeping be implemented by the various application threads?

thanks in advance, and sorry for the many questions!

spammy

[spammy]

janaka,

thanks for the response. yes, the whole xml file is changing. further all of
it needs to be made available to each client, as they would be viewing
arbitrary parts of it.

the cache object sounds ideal if the same cache is available to all loading
pages, but like you said it would need to be refreshed each second. if this
refreshing can be done in the backgroud by another thread, then it may work
out well. the question is however - can a thread be set up to do this? ive
been searching groups further and it sounds like asp threading isnt worth
the bother.

has anyone managed to create a working threading ASP application?

spammy

"Janaka" <ja****@magicalia.com> wrote in message
news:OJ**************@TK2MSFTNGP09.phx.gbl...

Spammy

sorry i haven't tried using any of the Threading classes with ASP.NET,
however you might be able to change some other parts of the application to
solve your problem?

first of all is all the data changing in the xml file? If possible i've
found it easier to split out a large xml file into several smaller files and use xslt on them to make a kind of view of the combined data.

try seeing if its possible to use the Cache object to store the data
(however if the cache has to be cleared every second i don't think this
would give you a large benefit).

J

"spammy" <[e-mail]> wrote in message
news:2h************@uni-berlin.de...

hi all,

im currently writing an ASP.net front end to display data found in an XML file. the XML file is quite large (700k or so), and so takes a while to
load. it will be changing around every second, so needs to be loaded at
least at that rate. at the moment, the ASP application loads the xml every refresh - im thinking that this may cause problems when there are 20

clients

each refreshing once per second.

the solution im thinking of is to have a thread accessible to all the ASP instances that deals with loading and refreshing the XML into a dataset.

it

would do this every second and make the dataset available to all ASP
instances to read from.

how would this work? assuming you can create threads as normal in an ASP
application, how do i make it accessible to all of them? by making it
static? which applicaiton thread instance would be in charge of starting

and

stopping the XML reading thread? i mean the first should start it and the last to exit should quit it, but can this be detected? or is a timeout a
best way to quit it (not accessed in a minute -> quit). is this thread a
particularly efficient way of doing this, or would it put another
unnecessary load on to the webserver?

if i dont use a thread, whats the best way to implement the XML reading?

it

would have to be a shared dataset, but only updated once every second -

how

would this timekeeping be implemented by the various application threads?
thanks in advance, and sorry for the many questions!

spammy

[Jason DeFontes]

You can put an object in the cache and tie it's expiration to a file on
the file system, so when the XML file changes it will trigger the
cache's expiration event, and you can reload the data and recreate the
dataset. If you store the dataset in the Application then all your
requests can access the same dataset object.

-Jason

spammy wrote:

janaka,

thanks for the response. yes, the whole xml file is changing. further all of
it needs to be made available to each client, as they would be viewing
arbitrary parts of it.

the cache object sounds ideal if the same cache is available to all loading
pages, but like you said it would need to be refreshed each second. if this
refreshing can be done in the backgroud by another thread, then it may work
out well. the question is however - can a thread be set up to do this? ive
been searching groups further and it sounds like asp threading isnt worth
the bother.

has anyone managed to create a working threading ASP application?

spammy

"Janaka" <ja****@magicalia.com> wrote in message
news:OJ**************@TK2MSFTNGP09.phx.gbl...

Spammy

sorry i haven't tried using any of the Threading classes with ASP.NET,
however you might be able to change some other parts of the application to
solve your problem?

first of all is all the data changing in the xml file? If possible i've
found it easier to split out a large xml file into several smaller files

and

use xslt on them to make a kind of view of the combined data.

try seeing if its possible to use the Cache object to store the data
(however if the cache has to be cleared every second i don't think this
would give you a large benefit).

J

"spammy" <[e-mail]> wrote in message
news:2h************@uni-berlin.de...

hi all,

im currently writing an ASP.net front end to display data found in an
XML
file. the XML file is quite large (700k or so), and so takes a while to
load. it will be changing around every second, so needs to be loaded at
least at that rate. at the moment, the ASP application loads the xml
every
refresh - im thinking that this may cause problems when there are 20

clients

each refreshing once per second.

the solution im thinking of is to have a thread accessible to all the
ASP
instances that deals with loading and refreshing the XML into a dataset.

it

would do this every second and make the dataset available to all ASP
instances to read from.

how would this work? assuming you can create threads as normal in an ASP
application, how do i make it accessible to all of them? by making it
static? which applicaiton thread instance would be in charge of starting

and

stopping the XML reading thread? i mean the first should start it and
the
last to exit should quit it, but can this be detected? or is a timeout a
best way to quit it (not accessed in a minute -> quit). is this thread a
particularly efficient way of doing this, or would it put another
unnecessary load on to the webserver?

if i dont use a thread, whats the best way to implement the XML reading?

it

would have to be a shared dataset, but only updated once every second -

how

would this timekeeping be implemented by the various application
threads?
thanks in advance, and sorry for the many questions!

spammy