XmlTextReader is skipping nodes unintentionally

[Geoff Bennett]

While parsing an XML document, my TextReader instance skips nodes. For
example, in this fragment:

<Person Sex="Male" FirstHomeBuyer="No" YearsInCurrentProfession="14">
<RelatedEntityRef RelatedID="118"/>
<PersonName>
<NameTitle Value="Mr"/>
<FirstName>Clint</FirstName>
<OtherName/>
<AlsoKnownAs>Mr C Eastwood</AlsoKnownAs>
<Surname>Eastwood</Surname>
</PersonName>

The parsing code will hit the "Person" element and the "RelatedEntityRef"
element, but it skips "PersonName" and winds up on "NameTitle". My code is
depending on hitting that "PersonName" element so that it can hand off the
outer XML to a PersonName class that knows how to parse the PersonName XML
fragment.

Now, to throw a spanner in the works, this only happens when the XML is
"unformatted". The string I'm getting passed to my web service has no
insignificant whitespace in it. It's all on one line, basically. When it
comes in this way, the code bombs. If I take the same XML, "pretty print" it
in XMLSpy then send it to the web service, it goes through without a problem.

Can anyone give me some indication of what I'm overlooking here?

Thanks.

Geoff.

[Derek Harmon]

"Geoff Bennett" <Ge**********@discussions.microsoft.com> wrote in message news:43**********************************@microsof t.com...

While parsing an XML document, my TextReader instance skips nodes. : : <Person Sex="Male" FirstHomeBuyer="No" YearsInCurrentProfession="14">
<RelatedEntityRef RelatedID="118"/>
<PersonName>
<NameTitle Value="Mr"/> : : The parsing code will hit the "Person" element and the "RelatedEntityRef"
element, but it skips "PersonName" and winds up on "NameTitle". : : Can anyone give me some indication of what I'm overlooking here?

The most common cause of this symptom is that XmlTextReader's
Read( ) method is being called but the processing logic is skipping
handling the node. Often, pretty-printing so that you have insignifi-
cant whitespace nodes will hide this flaw. Most XmlTextReader
logic is performed within a loop of some sort. Lookout for Read( )
appearing inside of the loop (definite warning sign), you want to
ensure Read( ) is happening only once per loop.

What code are you using the read from the XmlTextReader?

There's almost certainly some flow of control that's skipping over
the PersonName element.
Derek Harmon

[Geoff Bennett]

Thanks for replying Derek. The only time I call the XmlTextReader.Read()
method is during the while loop test. Inside the while loop, however,
ReadOuterXml() is called multiple times, but with awareness of what it's
doing.

The stripped down Parse function looks like this:

public override void Parse(string nodeText)
{
XmlTextReader xr = new XmlTextReader(new StringReader(nodeText));
while(xr.Read())
{
switch(xr.Name)
{
case ND_PERSON:
{
if (xr.Depth == 0)
{
string sex = xr.GetAttribute(AT_SEX);
if (sex != null)
Sex = sex;

string previousName = xr.GetAttribute(AT_PREVIOUS_NAME);
if (previousName != null)
PreviousName = previousName;

string firstHomeBuyer =
xr.GetAttribute(AT_FIRST_HOME_BUYER);
if (firstHomeBuyer != null)
FirstHomeBuyer =
LixiParser.ConvertToBoolean(firstHomeBuyer);

string yearsIn = xr.GetAttribute(AT_YEARS_IN_PROFESSION);
if (yearsIn != null)
YearsInCurrentProfession = int.Parse(yearsIn);
}
break;
}

case ND_RELATED_ENTITY_REF:
{
if (xr.Depth == 1)
{
string outerXml = xr.ReadOuterXml();
RelatedEntityRef.Parse(outerXml);
}
break;
}

case ND_PERSON_NAME:
{
if (xr.Depth == 1)
{
string outerXml = xr.ReadOuterXml();
PersonName.Parse(outerXml);
}
break;
}
}
}
xr.Close();
}

There's heaps more in it, but they're all the same as the ND_PERSON_NAME
case, just with different constants and different classes parsing the outer
XML. I'm aware that ReadOuterXml() advances the reader, but it should only
advance it to the next node after the current node's closing tag, surely?

The whole point in the design is each class knows how to parse itself from
an xml fragment passed into it. Each class is aware of the children of its
element, and so it hands off responsibility for parsing the child element to
an agregated child class.

The root class's parsing code starts like this:

public override void Parse(string lixiText)
{
XmlTextReader xr = null;
XmlDocument doc = new XmlDocument();
try
{
doc.LoadXml(lixiText);
xr = new XmlTextReader(new StringReader(doc.OuterXml));
}
catch
{
xr = new XmlTextReader(new StringReader(lixiText));
}
while(xr.Read())

I've just added the try/catch with the XmlDocument class trying to see if it
would have a problem parsing the text - it doesn't. That code won't be
staying there for obvious reasons.

Does this give you anymore information?
"Derek Harmon" wrote:

"Geoff Bennett" <Ge**********@discussions.microsoft.com> wrote in message news:43**********************************@microsof t.com...

While parsing an XML document, my TextReader instance skips nodes.

: :

<Person Sex="Male" FirstHomeBuyer="No" YearsInCurrentProfession="14">
<RelatedEntityRef RelatedID="118"/>
<PersonName>
<NameTitle Value="Mr"/>

: :

The parsing code will hit the "Person" element and the "RelatedEntityRef"
element, but it skips "PersonName" and winds up on "NameTitle".

: :

Can anyone give me some indication of what I'm overlooking here?

The most common cause of this symptom is that XmlTextReader's
Read( ) method is being called but the processing logic is skipping
handling the node. Often, pretty-printing so that you have insignifi-
cant whitespace nodes will hide this flaw. Most XmlTextReader
logic is performed within a loop of some sort. Lookout for Read( )
appearing inside of the loop (definite warning sign), you want to
ensure Read( ) is happening only once per loop.

What code are you using the read from the XmlTextReader?

There's almost certainly some flow of control that's skipping over
the PersonName element.
Derek Harmon

[Derek Harmon]

"Geoff Bennett" <Ge**********@discussions.microsoft.com> wrote in message news:0F**********************************@microsof t.com...

Thanks for replying Derek. The only time I call the XmlTextReader.Read()
method is during the while loop test. Inside the while loop, however,
ReadOuterXml() is called multiple times, but with awareness of what it's
doing.
ReadOuterXml( ) is another Read( ). :-)
while(xr.Read( ))
{ : : case ND_RELATED_ENTITY_REF:
{
if (xr.Depth == 1)
{
string outerXml = xr.ReadOuterXml( );
RelatedEntityRef.Parse( outerXml);
Walking through this can be illuminating (as can setting a breakpoint
on the call to ReadOuterXml( ) and looking at the XmlTextReader's
state before that call, and after the call, but as this newsgroup is a
text medium you'll have to settle for my narrative, sorry.)

In the While loop, xr.Read( ) reads a node. Let's say that node is
<RelatedEntityRef>, a start Element. Name matches the const
string of the case, xr.Depth indicates the depth is 1.

OK, inside this if block, xr.ReadOuterXml( ) is going to read at
least one more node and put it (them) behind us. After it is done,
xr.LocalName is going to say ... what?

Well, there are 2 possibilities. This could be an XML file with
lots of extra whitespace (pretty-printed). If it is, and xr.Read-
OuterXml( ) leaves the XmlTextReader sitting at the position
immediately following <RelatedEntityRef>, then we're sitting
on a plump, juicy Whitespace node. (Extra whitespace covers
up so many errors like this, it's amazing.)

On the other hand, this could be an XML file with all insignificant
whitespace stripped out. In this event, should xr.ReadOuterXml( )
leave the XmlTextReader sitting at the position immediately
following <RelatedEntityRef>, then we're sitting on <PersonName>.

We have more processing to do, though. Let's look at what happens
next. I break out of the switch (there's no goto to the case for
ND_PERSON_NAME so there is no fallthrough). I'm at the end
of the while loop, I go back to the top.

Guess what happens? Read( ) reads another node.

Now, if I was on a Whitespace node, this is probably going to move
me to <PersonName>.

However (and this is the prickly pear in all this), if I was already set
on <PersonName>, where my Name was PersonName and my
NodeType was Element ... then calling Read( ) will promptly move
me to the next node: that's <NameTitle>.
I'm aware that ReadOuterXml() advances the reader, but it should only
advance it to the next node after the current node's closing tag, surely?
Yes, and it has, and that next node is <PersonName>, and then
the Read( ) at the top of the while loop is called to move to the
next node after that.
Does this give you anymore information?

Yes, plenty of information. It's the two reads in a while-loop,
no doubt about it.

Next, what to do? Well, had the ND_RELATED_ENTITY_REF
case fell through (not automatically allowed in C#, unless you
use a goto to the next case), it /might/ have worked. But it's
a malicious little critter to leave in your code like this, and I've
never liked recommending the use of goto's, there's just some-
thing Wirth-less about them.

I think you can use a flag, freePass, that cuts off the Read( )
in the while conditional whenever ReadOuterXml( ) was called.
Then process the XmlNode its left the XmlTextReader on in the
next pass through the while loop.

Here are the changes to your code snippet for this fix,

- - - PersonParse.cs (excerpt)
// . . .
public override void Parse( string nodeText)
{
XmlTextReader xr = new XmlTextReader( new StringReader( nodeText));

// CUT-OFF flag indicating ReadOuterXml called on preceding loop.
bool freePass = false;

// CUT-OFF, ReadOuterXml has already read a node so don't move off it.
while( freePass || xr.Read( ) )
{
// CUT-OFF, ensure flag always down at the start of each loop.
freePass = false;

switch( xr.Name )
{
// . . .
case ND_RELATED_ENTITY_REF:
{
if (xr.Depth == 1)
{
string outerXml = xr.ReadOuterXml( );

// CUT-OFF, raise flag because ReadOuterXml called.
freePass = true;

relatedEntityRef.Parse( outerXml);
}
break;
}

case ND_PERSON_NAME:
{
if (xr.Depth == 1)
{
string outerXml = xr.ReadOuterXml( );

// CUT-OFF, raise flag because ReadOuterXml called.
freePass = true;

personName.Parse( outerXml);
}
break;
}
// . . .
- - -
Good luck,

Derek Harmon

[Geoff Bennett]

Thanks Derek. You've been an absolute legend. I actually went and put some
Debug.WriteLine()'s in to the code and commented out the switch statement. I
noticed that it was once again hitting the PersonName node. So I put the
switch statement back, and wrote out the name and type of the node at the
start and end of the while loop. That's when I picked up exactly what you've
described.

I took a different approach to the solution, putting the responsibility on
moving through the stream on the individual case statements. Some just read
attributes, others read the inner xml and the bulk of them read the outer
xml. In the case of attributes, I explicitly call Read() at the end of the
case. The while loop now checks to see if it's at the end of the stream. I
had considered using a flag, but ended up doing it this way seeing as I was
already doing all those reads anyway. I feel it keeps the body of the switch
more consistent. At least you can see that every case should have a Read() of
some description.

I've only got about another 164 classes to fix now... :)

Thanks for all your help. I wouldn't have figured it out so quickly without
you. I might even make the impending line of death!

"Derek Harmon" wrote:

Walking through this can be illuminating (as can setting a breakpoint
on the call to ReadOuterXml( ) and looking at the XmlTextReader's
state before that call, and after the call, but as this newsgroup is a
text medium you'll have to settle for my narrative, sorry.)

[toxaq]

Derek Harmon wrote:

.... xr.ReadOuterXml( ) is going to read at
least one more node and put it (them) behind us...

Wow, this had me in a head spin. I'd been going very well with
XmltextReader and inadvertantly blamed my woahs on XmlTextReaders
fragment constructor. I even thought I had test code to prove this to
be the case:
eg

// With fragment
XmlParserContext context = new XmlParserContext(null, null,
string.Empty, XmlSpace.None, Encoding.UTF8);
XmlTextReader xtReader = new XmlTextReader(xmlFragment,
XmlNodeType.Document, context);

while (xtReader.Read())
{
if (xtReader.Name.ToLower() == "section")
{
ContentXml += "OuterXML: " +xtReader.ReadOuterXml() + "\n";
}
}

// Straight from document
XmlTextReader xtReader = new XmlTextReader("content.xml");

while (xtReader.Read())
{
if (xtReader.Name.ToLower() == "section")
{
ContentXml += "OuterXML: " +xtReader.ReadOuterXml() + "\n";
}
}

The example straight from the document worked but the one where I was
reading from a fragment (the frament came from another objects
readxml()) wouldn't. It all turned out to be whitespace! What a great
lesson and great code fix.

I had been searching google over and over again for xmltextreader
fragment problem, readouterxml problem etc but I finally found this.

Thanks Derek!

--
toxaq
------------------------------------------------------------------------
Posted via http://www.mcse.ms
------------------------------------------------------------------------
View this thread: http://www.mcse.ms/message1168184.html