Locking XML/Corruption problems

[darrel]

Followup to a previous related topic now that I have a few more details.

We've been having issues with part of our home-grown CMS.

Whenever a page is edited, and saved, we do a recursive call to the DB and
then spit out an XML file to build the site map, which is then published to
the site.

We ran into an issue that we believe (as we can't replicate it) is happening
when two people manage to hit SAVE at the exact same time, causing two
processes to try writing to the exact same XML file at the same time. This
appears to cause the XML file to only be half-written (my theory that the
second process interupts the first and then they both fail).

This is what we originallly had:

objXMLWriter = New
System.xml.XmlTextWriter(System.Web.HttpContext.Cu rrent.Server.MapPath("/mjb05/xml/"
& siteID & "/siteMenu.xml").ToString, Nothing)

It was suggested that we lock the file as we write it, preventing any
further edits from another process until the first process is done with it:
Dim fs As New
System.IO.FileStream(System.Web.HttpContext.Curren t.Server.MapPath("/mjb05/xml/"
& siteID & "/siteMenu.xml").ToString, IO.FileMode.Open, IO.FileAccess.Write,
IO.FileShare.Read)
objXMLWriter = New System.xml.XmlTextWriter(fs, Nothing)

However, this causes a new problem. Not every time, but usually every other
time or so, when the XML is written, the entire XML file is written
correctly, but then some 'extra crap' is tacked onto the end...typically a
repeat of a few nodes split in the middle of a tag. Which, obviously, is
causing the XML that is being written to be invalid.

Any thoughts on what is happening and/or why and/or a fix/workaround?

The only thing I've been able to come up with is to have the function write
the XML, and then use another function to open the file back up again, and,
then using some string functions, trim off anything after the last closing
node, then resave. That, of course, is a huge hack, so less than idea.

I've also discovered that the problem ONLY happens when i do the recursion.
If I just run through the function once, I can not get it to write the bad
XML. Not sure HOW that is related, but it's something else I've been able to
observe.

Any thoughts appreciated!

-Darrel

[sirfunusa]

Sorry, but it looks like a logic problem. Create a stand-alone VB.NET
app that uses the same logic that writes to your local file system, and
attempt to diagnose. I really doubt that it is "two people at the exact
same time". If it happened more than once, that is impossible.

[darrel]

Sorry, but it looks like a logic problem. Create a stand-alone VB.NET
app that uses the same logic that writes to your local file system, and
attempt to diagnose. I really doubt that it is "two people at the exact
same time". If it happened more than once, that is impossible.

It happened twice. And you are right, no guarantee what, exactly, happened.
As there's no way for us to replicate it.

But, neither here or there...we should be locking the file anyways.

Regarding my current problem with a 'corrupted' XML file, I *FINALLY*
realized what was going on:

Every time I changed the database, and spit out a new XML file, the total
length of the XML file would change. As long as my DB data was INCREASING,
the resultant XML file would be longer than the previous and completely
overwrite the old data.

HOWEVER, if I did anything to REDUCE the amount of data in my DB, the
resultant XML file would be shorter than the previous one written. As such,
it would just 'leave' whatever extra bit of info was there in the file the
last time. We didn't notice this until recently, as most updates were
increasing the DB size (since we're in a massive content migration phase).

IOW, if my XML file is 900 characters long, and I update the DB so the next
XML file is 1000, everything is fine. Then I make another update and reduce
the file to 980 characters. At that point, it dumps all 980 characters back
to the XML file, but leaves those last 20 characters of the previous file,
thereby corrupting it.

Fix? I need to delete the file first and create a brand new, blank file.

WHEW! That was a stumper. ;o)

-Darrel