XmlDocument.SelectSingleNode() & namespaces - doesn't work

[David Thielen]

Hi;

If there are no namespaces this works fine for me. But if the xml has
namespaces, then I get either no node back or an exception.

Here is the sample xml:
<root xmlns="http://www.test.org"
xmlns:sns="http://www.test.org/sub"
xmlns:mns="http://www.test.org/mini">
<data>
<items>
<item id="1">
<sns:subItem sid="11">dave</sns:subItem>
<sns:subItem sid="12">thielen</sns:subItem>
</item>
<item id="2">
<sns:subItem sid="21">shirley</sns:subItem>
</item>
</items>
<mns:more>dave thielen</mns:more>
</data>
</root>

1. If I do SelectSingleNode("/root/data/items/item/@id") - I get no nodes
returned. This is the case if I create an XmlNamespaceManager or not.

2. If I do SelectSingleNode("/root/data/items/item/sns:subItem/@sid") where
I have a namespace qualified node I get:

2a. If there is no XmlNamespaceManager I get an exception.

2b. If I add the following code:
XmlNamespaceManager context = new XmlNamespaceManager(doc.NameTable);
context.AddNamespace("sns", "http://www.test.org/sub");
node = doc.SelectSingleNode(("/root/data/items/item/sns:subItem/@sid",
context);
Then I get no nodes returned (but at least no exception is thrown).

I tested all of the in XmlSpy and the xpath works there as expected.

So what is going on?

First off, even if I add the namespaces, I still can't get any node
returned. Both those with namespaces and those without evaluate to no nodes.

Second, why does it throw an exception if I don't create a namespace
manager? The namespaces exist in the xml file and by definition, those
mappings should be used. Also, to quote from
http://msdn.microsoft.com/library/de...edprefixes.asp
"If the msbooks namespace was not added to the XmlNamespaceManager, and if it
is not in the name table of the document, then an exception is thrown." So
there should be no exception thrown for namespaces that are defined in the
document.

Help please - this makes no sense to me.

--
thanks - dave

[Kevin Yu [MSFT]]

Hi Dave,

First of all, I would like to confirm my understanding of your issue. From
your description, I understand that you are trying to use SelectSingleNode
to get the attribute. If there is any misunderstanding, please feel free to
let me know.

Since the nodes are under the default namespace, SelectSingleNode require
us to specify the namespace for each node in the Xpath expression including
the default namespace. You can get /root/data/items/item/@id with the
following piece of code.

XmlDocument doc =new XmlDocument();
doc.LoadXml(s);
XmlNamespaceManager nsmgr = new XmlNamespaceManager(doc.NameTable);
nsmgr.AddNamespace("d", "http://www.test.org");
nsmgr.AddNamespace("sns", "http://www.test.org/sub");
nsmgr.AddNamespace("mns", "http://www.test.org/mini");
XmlNode node = doc.SelectSingleNode("//d:root/d:data/d:items/d:item/@id",
nsmgr);

Also, to get to /root/data/items/item/sns:subItem/@sid, Please use the
following code.

XmlNode node =
doc.SelectSingleNode("//d:root/d:data/d:items/d:item/sns:subItem/@sid",
nsmgr);

HTH.

Kevin Yu
=======
"This posting is provided "AS IS" with no warranties, and confers no
rights."

[David Thielen]

Hello;

My problem is that I am not hardcoding the xpath - I have to accept what the
user types in. And I have to accept any legal xpath and it is legal to enter
it with no namespace on the nodes that are in the default namespace.

How can I get this to handle any legal xpath string? (In other words, to
work the same way XmlSpy, dom4j, XSLT, and every other program that accepts
xpath works.)

--
thanks - dave
"Kevin Yu [MSFT]" wrote:

Hi Dave,

First of all, I would like to confirm my understanding of your issue. From
your description, I understand that you are trying to use SelectSingleNode
to get the attribute. If there is any misunderstanding, please feel free to
let me know.

Since the nodes are under the default namespace, SelectSingleNode require
us to specify the namespace for each node in the Xpath expression including
the default namespace. You can get /root/data/items/item/@id with the
following piece of code.

XmlDocument doc =new XmlDocument();
doc.LoadXml(s);
XmlNamespaceManager nsmgr = new XmlNamespaceManager(doc.NameTable);
nsmgr.AddNamespace("d", "http://www.test.org");
nsmgr.AddNamespace("sns", "http://www.test.org/sub");
nsmgr.AddNamespace("mns", "http://www.test.org/mini");
XmlNode node = doc.SelectSingleNode("//d:root/d:data/d:items/d:item/@id",
nsmgr);

Also, to get to /root/data/items/item/sns:subItem/@sid, Please use the
following code.

XmlNode node =
doc.SelectSingleNode("//d:root/d:data/d:items/d:item/sns:subItem/@sid",
nsmgr);

HTH.

Kevin Yu
=======
"This posting is provided "AS IS" with no warranties, and confers no
rights."

[Harry_Crow]

try out SelectSingleNode("//root/data/items/item[@id='1']")
SelectSingleNode("//root/data/items/item[@id='2']")

Harry

"David Thielen" wrote:

Hi;

If there are no namespaces this works fine for me. But if the xml has
namespaces, then I get either no node back or an exception.

Here is the sample xml:
<root xmlns="http://www.test.org"
xmlns:sns="http://www.test.org/sub"
xmlns:mns="http://www.test.org/mini">
<data>
<items>
<item id="1">
<sns:subItem sid="11">dave</sns:subItem>
<sns:subItem sid="12">thielen</sns:subItem>
</item>
<item id="2">
<sns:subItem sid="21">shirley</sns:subItem>
</item>
</items>
<mns:more>dave thielen</mns:more>
</data>
</root>

1. If I do SelectSingleNode("/root/data/items/item/@id") - I get no nodes
returned. This is the case if I create an XmlNamespaceManager or not.

2. If I do SelectSingleNode("/root/data/items/item/sns:subItem/@sid") where
I have a namespace qualified node I get:

2a. If there is no XmlNamespaceManager I get an exception.

2b. If I add the following code:
XmlNamespaceManager context = new XmlNamespaceManager(doc.NameTable);
context.AddNamespace("sns", "http://www.test.org/sub");
node = doc.SelectSingleNode(("/root/data/items/item/sns:subItem/@sid",
context);
Then I get no nodes returned (but at least no exception is thrown).

I tested all of the in XmlSpy and the xpath works there as expected.

So what is going on?

First off, even if I add the namespaces, I still can't get any node
returned. Both those with namespaces and those without evaluate to no nodes.

Second, why does it throw an exception if I don't create a namespace
manager? The namespaces exist in the xml file and by definition, those
mappings should be used. Also, to quote from
http://msdn.microsoft.com/library/de...edprefixes.asp
"If the msbooks namespace was not added to the XmlNamespaceManager, and if it
is not in the name table of the document, then an exception is thrown." So
there should be no exception thrown for namespaces that are defined in the
document.

Help please - this makes no sense to me.

--
thanks - dave

[Martin Honnen]

David Thielen wrote:

My problem is that I am not hardcoding the xpath - I have to accept what the
user types in. And I have to accept any legal xpath and it is legal to enter
it with no namespace on the nodes that are in the default namespace.

While an XPath expression
element-name
is syntactically correct it always selects only elements with the
element name 'element-name' in no namespace, if you want to select
elements in a namespace then you need to bind a prefix to the namespace
URI and use that prefix in the XPath expression, no change to the XML
document is needed for that, see also
<http://www.faqts.com/knowledge_base/view.phtml/aid/34022/fid/616>

--

Martin Honnen --- MVP XML
http://JavaScript.FAQTs.com/

[David Thielen]

Hi;

Tried - it didn't work.

--
thanks - dave
"Harry_Crow" wrote:

try out SelectSingleNode("//root/data/items/item[@id='1']")
SelectSingleNode("//root/data/items/item[@id='2']")

Harry

"David Thielen" wrote:

Hi;

If there are no namespaces this works fine for me. But if the xml has
namespaces, then I get either no node back or an exception.

Here is the sample xml:
<root xmlns="http://www.test.org"
xmlns:sns="http://www.test.org/sub"
xmlns:mns="http://www.test.org/mini">
<data>
<items>
<item id="1">
<sns:subItem sid="11">dave</sns:subItem>
<sns:subItem sid="12">thielen</sns:subItem>
</item>
<item id="2">
<sns:subItem sid="21">shirley</sns:subItem>
</item>
</items>
<mns:more>dave thielen</mns:more>
</data>
</root>

1. If I do SelectSingleNode("/root/data/items/item/@id") - I get no nodes
returned. This is the case if I create an XmlNamespaceManager or not.

2. If I do SelectSingleNode("/root/data/items/item/sns:subItem/@sid") where
I have a namespace qualified node I get:

2a. If there is no XmlNamespaceManager I get an exception.

2b. If I add the following code:
XmlNamespaceManager context = new XmlNamespaceManager(doc.NameTable);
context.AddNamespace("sns", "http://www.test.org/sub");
node = doc.SelectSingleNode(("/root/data/items/item/sns:subItem/@sid",
context);
Then I get no nodes returned (but at least no exception is thrown).

I tested all of the in XmlSpy and the xpath works there as expected.

So what is going on?

First off, even if I add the namespaces, I still can't get any node
returned. Both those with namespaces and those without evaluate to no nodes.

Second, why does it throw an exception if I don't create a namespace
manager? The namespaces exist in the xml file and by definition, those
mappings should be used. Also, to quote from
http://msdn.microsoft.com/library/de...edprefixes.asp
"If the msbooks namespace was not added to the XmlNamespaceManager, and if it
is not in the name table of the document, then an exception is thrown." So
there should be no exception thrown for namespaces that are defined in the
document.

Help please - this makes no sense to me.

--
thanks - dave

[David Thielen]

Hi;

There are a couple of things that don't strike me as right about this.

First, these xpath queries do work in XmlSpy and when using dom4j. And I
understand the difference between no namespace and default namespace - but
the 1.0 spec is a bit unclear on exactly what this means in practice. But it
seems to me that this should work in .NET since it works elsewhere.

Second, the namespaces are defined in the xml file. So why do they need to
be set again using XmlNamespaceManager.AddNamespace()? It should be able to
get those when parsing the xml file.

In my case I can't force these things because I have to handle xml files and
xpath statements given to me by a user in my app. So I can't create a
hard-coded work-around for this.

???

--
thanks - dave
"Martin Honnen" wrote:

David Thielen wrote:

My problem is that I am not hardcoding the xpath - I have to accept what the
user types in. And I have to accept any legal xpath and it is legal to enter
it with no namespace on the nodes that are in the default namespace.

While an XPath expression
element-name
is syntactically correct it always selects only elements with the
element name 'element-name' in no namespace, if you want to select
elements in a namespace then you need to bind a prefix to the namespace
URI and use that prefix in the XPath expression, no change to the XML
document is needed for that, see also
<http://www.faqts.com/knowledge_base/view.phtml/aid/34022/fid/616>

--

Martin Honnen --- MVP XML
http://JavaScript.FAQTs.com/

[Kevin Yu [MSFT]]

Hi Dave,

I'm afraid this is by design in .NET framework. The XmlSpy uses a standard
way in the spec, but in the .NET Xml, we have its own implementation of the
Xpath query.

The XmlNamespaceManager object is designed to serve as a collection of
namespace definitions. It is used by the .NET Framework XML processors to
resolve and manage the scope of namespaces used in XML documents. So you
must use a System.Xml.XmlNamespaceManager object to provide the required
namespace resolution and scope management support.

You can check the following KB articles for more information.

http://support.microsoft.com/?id=313828
http://support.microsoft.com/?id=316913

Kevin Yu
=======
"This posting is provided "AS IS" with no warranties, and confers no
rights."

[David Thielen]

Hi;

Ok - this seems to be required. But it's nuts - this does not match the W3C
standard as far as I can tell. Especially ignoring the namespace settings in
the xml file.

Does anyone have any code to work around this? There have to be others who
have parsed the xml file to pull out the namespace declarations and declare
them programatically to get the standard xpath functionality.

--
thanks - dave
"Kevin Yu [MSFT]" wrote:

Hi Dave,

I'm afraid this is by design in .NET framework. The XmlSpy uses a standard
way in the spec, but in the .NET Xml, we have its own implementation of the
Xpath query.

The XmlNamespaceManager object is designed to serve as a collection of
namespace definitions. It is used by the .NET Framework XML processors to
resolve and manage the scope of namespaces used in XML documents. So you
must use a System.Xml.XmlNamespaceManager object to provide the required
namespace resolution and scope management support.

You can check the following KB articles for more information.

http://support.microsoft.com/?id=313828
http://support.microsoft.com/?id=316913

Kevin Yu
=======
"This posting is provided "AS IS" with no warranties, and confers no
rights."

[Oleg Tkachenko [MVP]]

David Thielen wrote:

First, these xpath queries do work in XmlSpy and when using dom4j.
That means XmlSpy and Dom4J are seriously broken. Your queries must
return nothing on your XML as per XPath 1.0 spec.
And I
understand the difference between no namespace and default namespace - but
the 1.0 spec is a bit unclear on exactly what this means in practice. But it
seems to me that this should work in .NET since it works elsewhere.
It doesn't work with any conformant XSLT processor I'm aware. Actually
it works as expected - returns nothing. That's the most FAQ ever with
XPath. Read "XML Namespaces and How They Affect XPath and XSLT" at
http://msdn.microsoft.com/library/de...ml05202002.asp

Second, the namespaces are defined in the xml file. So why do they need to
be set again using XmlNamespaceManager.AddNamespace()? It should be able to
get those when parsing the xml file.
Because in XML you might define zillion of different namespaces with or
without clashing prefixes and so when you query for an element you must
say explicitly which namespace you are interested in. Take a look at
this XML:
<foo xmlns="urn:bar">
<foo xmlns="urn:foo"/>
</foo>

Which foo element should be selected by "//foo" query? The answer as per
XPath 1.0 spec is - none, because "foo" in XPath 1.0 means element in no
namespace.
In my case I can't force these things because I have to handle xml files and
xpath statements given to me by a user in my app. So I can't create a
hard-coded work-around for this.

Then your app is broken by design. When user query XML she must be able
to define mapping between namespace prefixes used in a query and
namespace URIs.

--
Oleg Tkachenko [XML MVP, MCAD]
http://blog.tkachenko.com

[Oleg Tkachenko [MVP]]

David Thielen wrote:

Ok - this seems to be required. But it's nuts - this does not match the W3C
standard as far as I can tell. Especially ignoring the namespace settings in
the xml file.

It's all according to W3C spec, sorry and every conformant XSLT and
XPath processor behaves this way.

--
Oleg Tkachenko [XML MVP, MCAD]
http://blog.tkachenko.com

[Martin Honnen]

David Thielen wrote:

There are a couple of things that don't strike me as right about this.

First, these xpath queries do work in XmlSpy and when using dom4j. And I
understand the difference between no namespace and default namespace - but
the 1.0 spec is a bit unclear on exactly what this means in practice. But it
seems to me that this should work in .NET since it works elsewhere.

I don't know dom4j and I don't have a version of XML Spy here but it
does not help to say "these XPath queries work somewhere" and do not
work in .NET, of course they work in .NET, it might only be that you do
not get the selection of nodes you expect from your use of those other
applications. If you have an XPath expression with a qualified name e.g.
pf:element-name
then that prefix pf needs to be bound to a namespace URI, otherwise the
XPath expression can't be evaluated. If those tools automatically
provide a binding of prefixes to namespace URIs then fine, but the .NET
API requires you to define those binding yourself if you want to use
XPath expressions with qualified names.

As for elements in the default namespace it is certainly correct
acording to the specs that you can only select them with an XPath
expression using a prefix bound to that default namespace URI, if those
tools return/select nodes in a default namespace by using unqualified
names without a prefix then those tools do not implement XPath 1.0 as is
specified.
--

Martin Honnen --- MVP XML
http://JavaScript.FAQTs.com/

[Oleg Tkachenko [MVP]]

Martin Honnen wrote:

If those tools automatically
provide a binding of prefixes to namespace URIs then fine,

That's unfeasible.

<p:foo xmlns:p="urn:p1">
<p:foo xmlns:p="urn:p2"/>
<p:foo>

Which element should //p:foo select?

--
Oleg Tkachenko [XML MVP, MCAD]
http://blog.tkachenko.com

[Martin Honnen]

Oleg Tkachenko [MVP] wrote:

Martin Honnen wrote:

If those tools automatically provide a binding of prefixes to
namespace URIs then fine,

That's unfeasible.

<p:foo xmlns:p="urn:p1">
<p:foo xmlns:p="urn:p2"/>
<p:foo>

^^
</p:foo>
Which element should //p:foo select?

When evaluating the XPath the prefix p can only be bound to one URI, I
don't know which strategy those tools use to provide those bindings
because I don't know the tools. For instance with Dimitre's XPath
visualizer 1.4 for IE the inner p:foo is selected.

I was not suggesting that there is a consistent and straightforward way
to infer bindings from any instance, I was just acknowledging that some
tools try to do that.
--

Martin Honnen --- MVP XML
http://JavaScript.FAQTs.com/

[David Thielen]

Hello;

Ok, I've just spent the last 2 hours reading all of the urls posted in this
thread, my xml/xpath books, and the W3C docs. Here's how I understand all of
this - please let me know what I am not understanding.

To make this easier I am posting 2 comments. This is on having to use
XmlNamespaceManager.

Ok, for the following xml:
<root xmlns="http://www.test.org" xmlns:sns="http://www.test.org/sub">
<sns:item>dave</sns:item>
</root>

The node root is in the default namespace who's uri is http://www.test.org
and item is in the sns namespace who's uri is http://www.test.org/sub.

Everything I have gone over says that for that xml those nodes are each tied
to those uris. You can have subnodes in the xml that change the default
and/or sns for all subnodes below them - but the xml there is the full and
complete definition of the namespaces for that xml.

So if I use:
XmlNamespaceManager context = new XmlNamespaceManager(xmlDocument);
context.AddNamespace("sns", "http://www.bogus.com");

I've just changed the namespace uri outside of the context of the xml.
Question 1: Am I not in this case mapping the nodes incorrectly according to
the xml spec?

Question 2: If I have a subnode of <sub
xmlns:sns="http://www.test.org/other"> - does this then overrule
XmlNamespaceManager for this node and it's children - or does
XmlNamespaceManager still overrule what is set in the xml file?

Question 3: Since the correct namespaces:URIs are in the xml document, why
does it not at least default to those? Why force a program to make method
calls to tell the XmlDocument object information that exists in the xml file?

thanks - dave

[David Thielen]

Here is the seond part of all this - lets say I have the xml file:

<root xmlns="http://www.test.org" xmlns:sns="http://www.test.org/sub">
<data>
<sns:subItem>dave</sns:subItem>
</data>
</root>

Then postings here have said the xpath to get "dave" is
"/d:root/d:data/sns:subItem" (with context.AddNamespace("d",
"http://www.test.org") setting d).

However, I think it should be "/root/data/sns:subItem"

I understand that root & data are in the default namespace. And the URI of
that namespace is http://www.test.org. But I have seen nothing that requires
a d:root instead of just root. And everything I have read say that root has
the default namespace without it being explicitly included.

So where in the xpath 1.0 spec is this required?

thanks - dave

[Oleg Tkachenko]

David Thielen wrote:

Here is the seond part of all this - lets say I have the xml file:

<root xmlns="http://www.test.org" xmlns:sns="http://www.test.org/sub">
<data>
<sns:subItem>dave</sns:subItem>
</data>
</root>

Then postings here have said the xpath to get "dave" is
"/d:root/d:data/sns:subItem" (with context.AddNamespace("d",
"http://www.test.org") setting d).

However, I think it should be "/root/data/sns:subItem"
Wrong, that selects root element in null namespace, while in your
document root element is in "http://www.test.org" namespace.
You must use prefix to select element in a namepace using XPath 1.0.

I understand that root & data are in the default namespace. And the URI of
that namespace is http://www.test.org. But I have seen nothing that requires
a d:root instead of just root. And everything I have read say that root has
the default namespace without it being explicitly included.

So where in the xpath 1.0 spec is this required?

http://www.w3.org/TR/xpath#node-tests

"A QName in the node test is expanded into an expanded-name using the
namespace declarations from the expression context. This is the same way
expansion is done for element type names in start and end-tags except
that the default namespace declared with xmlns is not used: if the QName
does not have a prefix, then the namespace URI is null (this is the same
way attribute names are expanded). "

--
Oleg Tkachenko [XML MVP, MCAD]
http://blog.tkachenko.com

[Oleg Tkachenko]

David Thielen wrote:

So if I use:
XmlNamespaceManager context = new XmlNamespaceManager(xmlDocument);
context.AddNamespace("sns", "http://www.bogus.com");

I've just changed the namespace uri outside of the context of the xml.
Question 1: Am I not in this case mapping the nodes incorrectly according to
the xml spec?
No, you just changed the mapping. You map sns prefix to different
namespace URI so obviously now you query selects nothing.
Question 2: If I have a subnode of <sub
xmlns:sns="http://www.test.org/other"> - does this then overrule
XmlNamespaceManager for this node and it's children - or does
XmlNamespaceManager still overrule what is set in the xml file?
They aren't connected in any way. It's just mapping. The main rule of
namespaces is: PREFIX DOESN'T MATTER. What matters is only namespace URI
and prefix is just a syntax sugar helping to shorten XML documents.
Question 3: Since the correct namespaces:URIs are in the xml document, why
does it not at least default to those? Why force a program to make method
calls to tell the XmlDocument object information that exists in the xml file?

I don't see how namespace URIs in a document can help you to express
which node you want to select. You probably mean prefixes, but they have
no meaning on their own.

PS. The best ever namespace tutorial is
http://www.jclark.com/xml/xmlns.htm. Highly recommended.

--
Oleg Tkachenko [XML MVP, MCAD]
http://blog.tkachenko.com

[Oleg Tkachenko]

Martin Honnen wrote:

When evaluating the XPath the prefix p can only be bound to one URI, I
don't know which strategy those tools use to provide those bindings
because I don't know the tools. For instance with Dimitre's XPath
visualizer 1.4 for IE the inner p:foo is selected.

That acually looks like old MSXML3 bug. I recall something like that.

--
Oleg Tkachenko [XML MVP, MCAD]
http://blog.tkachenko.com

[David Thielen]

Hi;

I understand what you are saying. And I read the link - I undestand that
explanation.

And I think I understand XmlNamespaceManager - it is overriding the
xmlns:sns=... in my xml file so the one in my file is ignored and instead the
prefix:URI mapping in XmlNamespaceManager is used instead. Correct???

What I don't understand is:

1) If I have a xmlns:sns=... in my xml, why is it not considered wrong for
XmlNamespaceManager to override the sns mapping? That strikes me as changing
the meaning of my xml.

2) Why does XmlNamespaceManager not have a way to initialize itself with the
prefix:URIs in the root element of the xml file? Since that is what is
normally desired.

thanks - dave