Experience with XML DataBases

Don't know wether this is the right newsgroup,
anyways:

Has anyone some experience with Native XML Databases.
Im planning to use this:

http://exist.sourceforge.net/

It has many advantages over Object Relational Mapping,
(that is how i do it now).
Like XUpdate and XQuery support.

But the perfomance of these systems.. how are they?
When is an XML Database an option.

Anyone some usefull input ?

re: Experience with XML DataBases

Tjerk Wolterink wrote:
[color=blue]
> Don't know wether this is the right newsgroup,
> anyways:
>
> Has anyone some experience with Native XML Databases.
> Im planning to use this:
>
> http://exist.sourceforge.net/
>
> It has many advantages over Object Relational Mapping,
> (that is how i do it now).
> Like XUpdate and XQuery support.
>
> But the perfomance of these systems.. how are they?
> When is an XML Database an option.
>
> Anyone some usefull input ?[/color]

I'm also just about to start using eXist, so I don't have anything
useful to add at the moment...perhaps later :-)

The only thing that struck me on first consideration is that there is no
documentation on how you load your _own_ files into the database, but
maybe it's just well-hidden.

///Peter

re: Experience with XML DataBases

Peter Flynn wrote:[color=blue]
> Tjerk Wolterink wrote:
>
>[color=green]
>>Don't know wether this is the right newsgroup,
>>anyways:
>>
>>Has anyone some experience with Native XML Databases.
>>Im planning to use this:
>>
>>http://exist.sourceforge.net/
>>
>>It has many advantages over Object Relational Mapping,
>>(that is how i do it now).
>>Like XUpdate and XQuery support.
>>
>>But the perfomance of these systems.. how are they?
>>When is an XML Database an option.
>>
>>Anyone some usefull input ?[/color]
>
>
> I'm also just about to start using eXist, so I don't have anything
> useful to add at the moment...perhaps later :-)
>
> The only thing that struck me on first consideration is that there is no
> documentation on how you load your _own_ files into the database, but
> maybe it's just well-hidden.
>[/color]

Doesnt it support XUpdate?

I found another xml-database system,
they are working on that system at our university,
but also on other universiteis.

I got a message from a professor saying that
the system is already used in some production-stable applications.

The system i mean is: MonetDB/XQuery:

http://monetdb.cwi.nl/

re: Experience with XML DataBases

On Tue, 15 Nov 2005 20:39:45 +0100, Tjerk Wolterink
<tjerk@wolterinkwebdesign.com> wrote:
[color=blue]
>Has anyone some experience with Native XML Databases.[/color]

Yes, none of it good. I note that XML databases are far less common now
than they were about 5 years ago. IMHO, it's just not a useful
technology.

XML is not a good database model. It's hierarchical, and the real world
isn't. XML is OK for a "document", but it just doesn't scale well to
"database-scale" examples. Very few "XML databases" ever supported
ID/IDREF either.

The real-world, and the real-world of database applications, has
different shapes: relational, Kimball OLAP datacube or graph. These are
all more relevant than hierarchies.

Everyone has a phone bill and all of our phone bills have much the same
structure. It's a good fit to the relational model. In general, if you
really _need_ a huge database it's because you have a great many thigns
that are all alike - relational databases are commonly used for good
reason!

Many "XML databases" are also pure relational databases, with relational
data models, and with a minor output formatter to render their results
as XML (MS SQL Server is a prime example). These have little to offer
real XML work because their data modelling and implementation _process_
as seen by their developers is so RDBMS-like, no matter what the
executable application code looks like.

If you're trying to model the "real world", then you need something that
works with an arbitrary graph - either "nodes" and "connections" between
them, or "objects" and "properties" (pick your favourite description).
This has more in common with RDF than XML. There's also a reasonable
amount of work around in implementing such models as useful-performance
databases, either natively or (as my own work and patents led) by
hosting them on commercial SQL platforms. You can of course implement
hierarchies quite happily as a subset of this model, so there's no
problem in using it to store XML, if that's what you need.

--
Cats have nine lives, which is why they rarely post to Usenet.

re: Experience with XML DataBases

Andy Dingley wrote:[color=blue]
> On Tue, 15 Nov 2005 20:39:45 +0100, Tjerk Wolterink
> <tjerk@wolterinkwebdesign.com> wrote:
>
>[color=green]
>>Has anyone some experience with Native XML Databases.[/color]
>
>
> Yes, none of it good. I note that XML databases are far less common now
> than they were about 5 years ago. IMHO, it's just not a useful
> technology.
>[/color]

Ever tried MonetDB/XQuery?
http://monetdb.cwi.nl/

[color=blue]
> XML is not a good database model. It's hierarchical, and the real world
> isn't. XML is OK for a "document", but it just doesn't scale well to
> "database-scale" examples. Very few "XML databases" ever supported
> ID/IDREF either.
>
> The real-world, and the real-world of database applications, has
> different shapes: relational, Kimball OLAP datacube or graph. These are
> all more relevant than hierarchies.
>[/color]

There are some advanced method how to map xml to
relational type databases.
When you look at it a relational database system
often represents tree hierarchies.

But you are partly right,
but you have to understand that there is data-centric information (a
list of products)
and document-centric information (example: a website)
[color=blue]
>
> Everyone has a phone bill and all of our phone bills have much the same
> structure. It's a good fit to the relational model. In general, if you
> really _need_ a huge database it's because you have a great many thigns
> that are all alike - relational databases are commonly used for good
> reason!
>[/color]

Yes ok, but that is mostly the case for data-centric information.
[color=blue]
> Many "XML databases" are also pure relational databases, with relational
> data models, and with a minor output formatter to render their results
> as XML (MS SQL Server is a prime example). These have little to offer
> real XML work because their data modelling and implementation _process_
> as seen by their developers is so RDBMS-like, no matter what the
> executable application code looks like.
>[/color]

Well i am using the MonetDB system now, and it
performs extremely well. And it supports the advanced
XQuery language.

[color=blue]
>
> If you're trying to model the "real world", then you need something that
> works with an arbitrary graph - either "nodes" and "connections" between
> them, or "objects" and "properties" (pick your favourite description).
> This has more in common with RDF than XML.[/color]

I do not agree with you there.
XML is a very good document format for information.

I dont know what you mean with "real world" information.
[color=blue]
> There's also a reasonable
> amount of work around in implementing such models as useful-performance
> databases, either natively or (as my own work and patents led) by
> hosting them on commercial SQL platforms. You can of course implement
> hierarchies quite happily as a subset of this model, so there's no
> problem in using it to store XML, if that's what you need.
>[/color]


Don't follow you there... do you mean implementing xml databases
with exists database (relational) technology?

Why would you do that if there are better xml-database system that
perform well?

re: Experience with XML DataBases

There's a simple question to ask to help you determine whether your
application needs a native XML database: got XML? If the answer is yes, then
you need a native XML database. This argument is developed in more depth by
Ron Bourret at http://www.rpbourret.com/xml/XMLAndDatabases.htm. The basic
idea is that if you are dealing with XML documents in any form, and you need
to store and retrieve them, then you need a native XML database.

As an example, I worked on a web service project in which we received an XML
payload, manipulated it in various ways into more XML documents, before
eventually writing the data to a relational database. Along the way we used
a native XML database to archive the XML payload document and all the
interim documents we created. If any questions arose at a later date about
the contents of the payload or how it was manipulated, we could query the
XML database and retrieve the original document.

Trying to shoehorn hierarchal XML data into a relational database is hard
work but if you have no other option then there are some tools out there to
help you:

· Most relational database vendors now offer XML extenders. These are
largely SQL functions and data types that help you work with XML data in an
SQL-like way.

· I've used an open source product called XML-DBMS. Through a mapping file
it's possible to take an XML payload and write the contents to a relational
database. Overall performance and functionality are good and there's an
active user forum to help you out if things get sticky.

· Altova (http://www.altova.com/) have some tools to help with mapping
between relational databases and XML.

The key thing to remember is that relational and native XML databases are
not really in competition because they are working with fundamentally
different data structures. With a bit of work one can be made to perform the
duties of the other, for example by using some of the tools above, but it's
best to choose a database model that suits the structure of your data.

So, if you've decided that native XML databases are for you there are some
products worth evaluating:

· Berkeley DB XML(http://www.sleepycat.com/). The site also has lots of case
studies of companies using native XML databases in the real world.

· Apache's Xindice (http://xml.apache.org/xindice/). I evaluated this for
our web service project, but I think that at this stage of its development
there might be some limitations on the size of the XML documents it can
handle.

· The next major release of IBM's DB2 will include native XML support.

Personally, I find the Berkeley product good to use: it has a solid
programming APIs, lots of documentation, and the Stylus Studio editor
provides a GUI interface.

Also, check out the following book for some comparisons and evaluations of
native and hybrid XMl databases: Chaudhri, A. B., Rashid, A. and Zicari, R.,
(eds.). (2003). XML Data Management: Native XML and XML-Enabled Database
Systems. Boston: Addison Wesley.

re: Experience with XML DataBases

Tjerk Wolterink wrote:[color=blue]
> Don't know wether this is the right newsgroup,
> anyways:
>
> Has anyone some experience with Native XML Databases.
> Im planning to use this:
>
> http://exist.sourceforge.net/
>
> It has many advantages over Object Relational Mapping,
> (that is how i do it now).
> Like XUpdate and XQuery support.
>
> But the perfomance of these systems.. how are they?
> When is an XML Database an option.
>
> Anyone some usefull input ?[/color]

A while ago we published a story on a guy at NASA that was using the
eXist native XML database's XQuery features in a real application at
the Johnson Space Center. The write-up was probably not technical
enough to be of help but the Engineer we interviewed had nothing but
positive things to say about the results of his project. As far as I
know, it's the only published customer use-case involving eXist and
XQuery. Here's the link fyi:
http://www.stylusstudio.com/xquery/cust_nasa.html

Sincerely,
The Stylus Studio Team
http://www.stylusstudio.com