473,543 Members | 2,565 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

What could cause XML files to not be completely written?

We've had a month long headache with one of our functions in the CMS we've
built.

Whenever a page is updated in our CMS, we do a recursive call to the DB to
get all the pages (and child pages, and child child pages, etc.) and then
create an XML file, and save that to the filesystem. The front end then uses
that XML file along with some XSL to create the site menus, site maps, etc.

We're having a problem where 99 out of 100 times, this works fine. But that
one time, we'll get an incomplete (and therefore invalid) XML file written
to the filesystem. No error. Nothing. Just a bad file.

What could be the possible causes of this? I've included the function that
writes the XML at the bottom of this message if that helps.

At this point, we're looking into two solutions which just seem a bit hacky:

1) write the file, open the file, validate it, if invalid, write again,
repeat a few times and then report an error if you have to.

2) forget the filesystem. Write the XML file as a VARCHAR back into a DB
table and just grab it from there.

Any other thoughts? Things to look out for? Solutions? Theories?

Thanks!

------------------------------------------------------------------

Imports System.IO
Public Class menuWriter
Shared fs As System.IO.FileS tream
'shared objXMLWriter As system.xml.XmlT extWriter

Shared Sub createXMLfileWi thXmlWriter(ByV al parentNode As Integer, ByVal
intLevel As Integer, ByVal rowCount As Integer, ByVal createParentNod e As
Boolean, ByVal siteID As Integer)
Try
' Create the parent node
'dim objXMLWriter As system.xml.XmlT extWriter
' xml writing adapted from
' http://www.developerfusion.com/show/2646/
If
System.IO.Direc tory.Exists(Sys tem.Web.HttpCon text.Current.Se rver.MapPath("/mjb05/xml/"
& siteID)) Then
'directory exists. Do nothing
Else
'directory doesn't exist. Creat directory
System.IO.Direc tory.CreateDire ctory(System.We b.HttpContext.C urrent.Server.M apPath("/mjb05/xml/"
& siteID))
End If

Dim objXMLWriter As System.Xml.XmlT extWriter
Dim path As String =
System.Web.Http Context.Current .Server.MapPath ("/mjb05/xml/" & siteID &
"/siteMenu.xml"). ToString
fs = New FileStream(path , FileMode.Create , FileAccess.Writ e, FileShare.None)
SyncLock fs
objXMLWriter = New System.xml.XmlT extWriter(fs,
System.Text.Enc oding.Default)
objXMLWriter.Fo rmatting = objXMLWriter.Fo rmatting.Indent ed
objXMLWriter.In dentation = 3
objXMLWriter.Wr iteStartDocumen t()
objXMLWriter.Wr iteComment("Cre ated on " & Now())
objXMLWriter.Wr iteStartElement ("menuItems" )
WriteXmlData(ob jXMLWriter, parentNode, siteID)
objXMLWriter.Wr iteEndElement() 'close menuItems node
objXMLWriter.Wr iteEndDocument( ) 'close docment
objXMLWriter.Fl ush()
objXMLWriter.Cl ose()
End SyncLock
Catch ex As Exception
System.Web.Http Context.Current .Response.Write ("Error writing XML file<br
/>")
System.Web.Http Context.Current .Response.Write (ex.Message.ToS tring)
End Try
End Sub

Shared Sub WriteXmlData(By Val objXMLWriter As System.Xml.XmlT extWriter,
ByVal parentNode As Integer, ByVal siteID As Integer, Optional ByVal
intLevel As Int32 = 0)
'retrieve all children of parentNode
' grab the data from the db
Dim DS As New DataSet
Dim strConnect As String
strConnect =
System.Configur ation.Configura tionSettings.Ap pSettings("DBCo nn")
Dim strChk As String
strChk = "SELECT pageID, siteID, parentID, linkText, browserTitle,
contentTitle, hidden, sort, customContent, customContentVa riables,
menuCategory, isCategory, homePage " & _
"FROM WeSiteMenus " & _
"WHERE ParentID = " & parentNode & " AND siteID = " & siteID & " " & _
"ORDER BY homePage DESC, sort ASC"
Dim objConnect As New System.Data.Ole Db.OleDbConnect ion(strConnect)
objConnect.Open ()
Dim objCommand As New System.Data.Ole Db.OleDbCommand (strChk, objConnect)
Dim objOleDbAdapter As New System.Data.Ole Db.OleDbDataAda pter(strChk,
strConnect)
objOleDbAdapter .Fill(DS, "webPages")

' Walk through results whether it's a list or a single contact.
' based off of this tutorial:
' http://www.wwwcoder.com/main/parenti...8/default.aspx
Dim rowCount As Int32 = 0
While rowCount < DS.Tables(0).Ro ws.Count 'ie, if there IS data, then...
objXMLWriter.Wr iteStartElement ("menuItem")

objXMLWriter.Wr iteElementStrin g("pageID",
Trim(DS.Tables( 0).Rows(rowCoun t)("pageID").To String))
objXMLWriter.Wr iteElementStrin g("parentID",
Trim(DS.Tables( 0).Rows(rowCoun t)("parentID"). ToString))
'linkURL...need to actuall add this to the DB!
objXMLWriter.Wr iteElementStrin g("linkURL", "")
objXMLWriter.Wr iteElementStrin g("linkText",
Trim(DS.Tables( 0).Rows(rowCoun t)("linkText"). ToString))
objXMLWriter.Wr iteElementStrin g("browserTitle ",
Trim(DS.Tables( 0).Rows(rowCoun t)("browserTitl e").ToString ))
objXMLWriter.Wr iteElementStrin g("contentTitle ",
Trim(DS.Tables( 0).Rows(rowCoun t)("contentTitl e").ToString ))
objXMLWriter.Wr iteElementStrin g("hidden",
Trim(DS.Tables( 0).Rows(rowCoun t)("hidden").To String))
objXMLWriter.Wr iteElementStrin g("customConten t",
Trim(DS.Tables( 0).Rows(rowCoun t)("customConte nt").ToString ))
objXMLWriter.Wr iteElementStrin g("customConten tVariables",
Trim(DS.Tables( 0).Rows(rowCoun t)("customConte ntVariables").T oString))
objXMLWriter.Wr iteElementStrin g("menuCategory ",
Trim(DS.Tables( 0).Rows(rowCoun t)("menuCategor y").ToString ))
objXMLWriter.Wr iteElementStrin g("isCategory ",
Trim(DS.Tables( 0).Rows(rowCoun t)("isCategory" ).ToString))
objXMLWriter.Wr iteElementStrin g("level", intLevel.ToStri ng)
'now call the subroutine we're in to see if this value has
'any children and increase the indent, and so on...
WriteXmlData(ob jXMLWriter,
Convert.ToInt32 (DS.Tables(0).R ows(rowCount).I tem(0)), siteID, intLevel + 1)
rowCount = rowCount + 1
objXMLWriter.Wr iteEndElement() 'close menuItem node
End While
' clean up
objConnect.Clos e()
objOleDbAdapter .Dispose()
objCommand.Disp ose()
End Sub
End Class
May 11 '06 #1
3 1482
darrel wrote:
We've had a month long headache with one of our functions in the CMS we've
built.

Whenever a page is updated in our CMS, we do a recursive call to the DB to
get all the pages (and child pages, and child child pages, etc.) and then
create an XML file, and save that to the filesystem. The front end then uses
that XML file along with some XSL to create the site menus, site maps, etc.

We're having a problem where 99 out of 100 times, this works fine. But that
one time, we'll get an incomplete (and therefore invalid) XML file written
to the filesystem. No error. Nothing. Just a bad file.
You mean the file just ends in mid-element or something?
The only obvious things that come to mind are a buggy serializer
or a timing problem between the DB and the OS or disk subsystem.
What could be the possible causes of this? I've included the function that
writes the XML at the bottom of this message if that helps.

At this point, we're looking into two solutions which just seem a bit hacky:

1) write the file, open the file, validate it, if invalid, write again,
repeat a few times and then report an error if you have to.

2) forget the filesystem. Write the XML file as a VARCHAR back into a DB
table and just grab it from there.


Easier would be to run an external standalone validator (ie unconnected
to and independent of any other part of your system) which will pass
control onwards if the file is OK, but otherwise terminate the process
and restart it at the file export step.

///Peter
--
XML FAQ: http://xml.silmaril.ie/
May 11 '06 #2
> You mean the file just ends in mid-element or something?

Yep. It'll just cut of mid-element. Randomly anywhere in the document.
The only obvious things that come to mind are a buggy serializer
I'm not sure if I'd know what to look for regarding that. What would that
actually be?
or a timing problem between the DB and the OS or disk subsystem.
Well, I'm grabbing all the data from the DB at once, and then writing it to
the XML file in one swoop, so that probably isn't it. No doubt it could be
the OS or disk subsystem, though. NO idea how to even begin to figure out
what specifically to look at in that context, though.
Easier would be to run an external standalone validator (ie unconnected
to and independent of any other part of your system) which will pass
control onwards if the file is OK, but otherwise terminate the process
and restart it at the file export step.


Whoa. So, I'd be passing something to another app and back again? I'm not
sure if that sounds easier to me ;o)

What kind of external validator are we talking about? Just another web app
that I'd whip up? What would be the advantage of externalizing the validator
(vs just validating what was written in the app itself?)

-Darrel
May 11 '06 #3
darrel wrote:
What kind of external validator are we talking about? Just another web app
that I'd whip up? What would be the advantage of externalizing the validator
(vs just validating what was written in the app itself?)


Any of the standalone parsers will do: it doesn't actually have to
validate if all you want is to test the file's completeness. If we
assume that the file is either complete (and correct) or incomplete
(and that no other mid-way corruption takes place) then a simple
well-formedness check will show if the file is complete or not.

My reason for suggesting an external process was simply to guarantee
that the parse was not being affected by anything else that might have
gone on inside your app.

///Peter
--
XML FAQ: http://xml.silmaril.ie/
May 13 '06 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

92
6343
by: Reed L. O'Brien | last post by:
I see rotor was removed for 2.4 and the docs say use an AES module provided separately... Is there a standard module that works alike or an AES module that works alike but with better encryption? cheers, reed
125
14558
by: Sarah Tanembaum | last post by:
Beside its an opensource and supported by community, what's the fundamental differences between PostgreSQL and those high-price commercial database (and some are bloated such as Oracle) from software giant such as Microsoft SQL Server, Oracle, and Sybase? Is PostgreSQL reliable enough to be used for high-end commercial application? Thanks
4
334
by: Jeff Grundy | last post by:
I am using the FileSystemWatcher to watch a folder for new file. The new files will be very large. The Created event is fired when the new file first appears in a folder. How do I know when the file system is actually finished writing the new file? -- Jeff
121
9920
by: typingcat | last post by:
First of all, I'm an Asian and I need to input Japanese, Korean and so on. I've tried many PHP IDEs today, but almost non of them supported Unicode (UTF-8) file. I've found that the only Unicode support IDEs are DreamWeaver 8 and Zend PHP Studio. DreamWeaver provides full support for Unicode. However, DreamWeaver is a web editor rather...
23
1873
by: JDeats | last post by:
Just spent some time browsing around here: http://msdn.microsoft.com/Longhorn/ I can see the benefits from WinFS (as long as we tag all in-coming data this should be nice, tagging everything could be a tedious process though) but I haven't been able to pick up the practical advantages of XAML. From my assessment, XAML will provide...
37
5921
by: Alan Silver | last post by:
Hello, Newbie here, so please forgive what is probably a basic question ... I see a lot of discussion about "code behind", which if I have understood correctly, means that the script code goes in a separate file from the HTML. Apart from the obvious advantage if you have a separate designer and programmer, are there any other advantages...
98
4496
by: tjb | last post by:
I often see code like this: /// <summary> /// Removes a node. /// </summary> /// <param name="node">The node to remove.</param> public void RemoveNode(Node node) { <...> }
2
3428
by: James | last post by:
CASE 1: The following sentences were copied from the book "Administration Guide Performance". "If more pages have been written to disk, recovery of the database is faster after a system crash because the database manager can rebuild more of the buffer pool from disk instead of having to replay transactions from the database log files." ...
15
2047
by: jim | last post by:
Maybe I'm missing something, but it doesn't look like Microsoft writes a lot of apps in .Net (although they certainly push it for others). What does MS write using pure .Net? If applications like Symantec's antivirus, NeatReciepts or Franklin Covey's PlanPlus for Windows is any guide, .Net applications are slow and clunky. But, maybe the...
0
7341
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
7584
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
0
7677
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
1
5258
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
4884
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3385
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in...
0
3386
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
1809
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
0
626
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.