Cacheing DB content

[darrel]

In the past, I've used an XML to maintain a bunch of site data and cached
it. I'd only grab a new copy for the cache when the XML file was physically
updated:

-----------------------------------

Dim XMLfile As String
'check to see if the file exists in the cache
If System.Web.HttpContext.Current.Cache(siteID & "menuXML") Is Nothing Then
'read in the XML file
Dim file As New
System.IO.StreamReader(HttpContext.Current.Server. MapPath("/xml/" & siteID &
"/siteMenu.xml"))
XMLfile = file.ReadToEnd
file.Close()
' set the dependency (to only update when file is changed)
Dim depends As New
System.Web.Caching.CacheDependency(HttpContext.Cur rent.Server.MapPath("/xml/"
& siteID & "/siteMenu.xml"))
'add the text to the cache object
System.Web.HttpContext.Current.Cache.Insert(siteID & "menuXML", XMLfile,
depends)
Else
'just use the cached version
XMLfile = CType(System.Web.HttpContext.Current.Cache(siteID & "menuXML"),
String)
End If

-----------------------------------

For a variety of reasons, we're no longer storing the XML as an XML file.
Instead, we're storing the XML in the database.

For now, that means on each page load, we're querying the DB and grabbing
the XML file. Not a huge deal, but it'd be nice to cache it. However, I
don't see a practical way to cache it and only update when the DB is
updated. It seems to me that to see if the DB is updated, I'd have to query
it anyways, so I might as well just grab the XML while I'm at it.

Any thoughts on that? Is it that big of a deal just to grab the data each
and every page load or is there a more elaborate way to handle the caching
in this case?

-Darrel

[Ray Booysen]

I do something similiar, but I create an object of type XmlDocument.

If you can load your XML document into there and cache you shouldn't
have many problems:

System.Xml.XmlDocument MenuDocument = new XmlDocument();
MenuDocument.Load(Server.MapPath("~" + "\\Menu.xml"));

if you can get your XML from the database to stream into an XmlDocument,
you'll be home free. ;)

darrel wrote:

In the past, I've used an XML to maintain a bunch of site data and cached
it. I'd only grab a new copy for the cache when the XML file was physically
updated:

-----------------------------------

Dim XMLfile As String
'check to see if the file exists in the cache
If System.Web.HttpContext.Current.Cache(siteID & "menuXML") Is Nothing Then
'read in the XML file
Dim file As New
System.IO.StreamReader(HttpContext.Current.Server. MapPath("/xml/" & siteID &
"/siteMenu.xml"))
XMLfile = file.ReadToEnd
file.Close()
' set the dependency (to only update when file is changed)
Dim depends As New
System.Web.Caching.CacheDependency(HttpContext.Cur rent.Server.MapPath("/xml/"
& siteID & "/siteMenu.xml"))
'add the text to the cache object
System.Web.HttpContext.Current.Cache.Insert(siteID & "menuXML", XMLfile,
depends)
Else
'just use the cached version
XMLfile = CType(System.Web.HttpContext.Current.Cache(siteID & "menuXML"),
String)
End If

-----------------------------------

For a variety of reasons, we're no longer storing the XML as an XML file.
Instead, we're storing the XML in the database.

For now, that means on each page load, we're querying the DB and grabbing
the XML file. Not a huge deal, but it'd be nice to cache it. However, I
don't see a practical way to cache it and only update when the DB is
updated. It seems to me that to see if the DB is updated, I'd have to query
it anyways, so I might as well just grab the XML while I'm at it.

Any thoughts on that? Is it that big of a deal just to grab the data each
and every page load or is there a more elaborate way to handle the caching
in this case?

-Darrel

[Göran Andersson]

You could cache the data for one minute, or whatever time delay you find
reasonable.

:: At low traffic, the data will be fetched for almost every page, but
that is no problem as there is no strain on the server.

:: At high traffic the data will be fetched at most once a minute.

When you update the data in the database, it will not be more than one
minute before the changes are visible on the site.

Also, the database caches the responses, so most of the time when you
fetch the data you will not even be getting the data from the actual
database, but from the database cache.
darrel wrote:

In the past, I've used an XML to maintain a bunch of site data and cached
it. I'd only grab a new copy for the cache when the XML file was physically
updated:

-----------------------------------

Dim XMLfile As String
'check to see if the file exists in the cache
If System.Web.HttpContext.Current.Cache(siteID & "menuXML") Is Nothing Then
'read in the XML file
Dim file As New
System.IO.StreamReader(HttpContext.Current.Server. MapPath("/xml/" & siteID &
"/siteMenu.xml"))
XMLfile = file.ReadToEnd
file.Close()
' set the dependency (to only update when file is changed)
Dim depends As New
System.Web.Caching.CacheDependency(HttpContext.Cur rent.Server.MapPath("/xml/"
& siteID & "/siteMenu.xml"))
'add the text to the cache object
System.Web.HttpContext.Current.Cache.Insert(siteID & "menuXML", XMLfile,
depends)
Else
'just use the cached version
XMLfile = CType(System.Web.HttpContext.Current.Cache(siteID & "menuXML"),
String)
End If

-----------------------------------

For a variety of reasons, we're no longer storing the XML as an XML file.
Instead, we're storing the XML in the database.

For now, that means on each page load, we're querying the DB and grabbing
the XML file. Not a huge deal, but it'd be nice to cache it. However, I
don't see a practical way to cache it and only update when the DB is
updated. It seems to me that to see if the DB is updated, I'd have to query
it anyways, so I might as well just grab the XML while I'm at it.

Any thoughts on that? Is it that big of a deal just to grab the data each
and every page load or is there a more elaborate way to handle the caching
in this case?

-Darrel

[darrel]

> if you can get your XML from the database to stream into an XmlDocument,

you'll be home free. ;)

So, what does that do...is that making an actual XML document?

If so, how does the cache know when to reload the data?

-Darrel

[darrel]

> You could cache the data for one minute, or whatever time delay you find

reasonable.
Yea, that's probably what I'll do. It's not ideal, though. The XML file
stores the site menu.

So, a page could be deleted, and if someone visits the site for that one
minute, the old XML file will still be pointing to a non-existant page.

Still, since each page might be accessing the same XML up to 3 times (all in
different controls) it probably makes sense to cache it just to save on
that.
Also, the database caches the responses, so most of the time when you
fetch the data you will not even be getting the data from the actual
database, but from the database cache.

Ah, so if I have 3 different calls to get the XML from the DB in three
different controls (all accessing the same class), does caching benefit me
at all, or is that all being done by the DB for me?

-Darrel

[Göran Andersson]

darrel wrote:

You could cache the data for one minute, or whatever time delay you find
reasonable.
Yea, that's probably what I'll do. It's not ideal, though. The XML file
stores the site menu.

So, a page could be deleted, and if someone visits the site for that one
minute, the old XML file will still be pointing to a non-existant page.

Yes, but will have that problem anyway. The user could visit the site
just before the page was deleted, and until the page in the browser gets
refreshed, it will have a menu with an item that points to a
non-existant page.
Still, since each page might be accessing the same XML up to 3 times (all in
different controls) it probably makes sense to cache it just to save on
that.

It definitely does. Even an extremely short cache time, like a few
seconds, would still save some database traffic.

Also, the database caches the responses, so most of the time when you
fetch the data you will not even be getting the data from the actual
database, but from the database cache.

Ah, so if I have 3 different calls to get the XML from the DB in three
different controls (all accessing the same class), does caching benefit me
at all, or is that all being done by the DB for me?

Yes, you would benefit from caching. Even if the database caches the
result, you still have to make the roundtrip to the database server to
get it.

You only need to decide if you benefit enough, or if the performance is
good enough without bothering with the caching.

[darrel]

> Yes, but will have that problem anyway. The user could visit the site just

before the page was deleted, and until the page in the browser gets
refreshed, it will have a menu with an item that points to a non-existant
page.

Touche! (Good point! ;o)

-Darrel