XQuery on big files

[Pascal Sartoretti]

Hello,

I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
processors that work in a "SAX-like" way (and not DOM-like), i.e. which
don't assume that they have everything in memory ?

Thanks for any help

Pascal

[Philippe Poulard]

Pascal Sartoretti wrote:

Hello,

I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
processors that work in a "SAX-like" way (and not DOM-like), i.e. which
don't assume that they have everything in memory ?

Thanks for any help

Pascal

hi,

with RefleX, you can process big files with XPath ; unlike many
streaming-XPath processors, RefleX doesn't restrict the XPath syntax,
but you must be aware that using XPath queries such as //* on the root
would cause caching the complete document causing an OutOfMemoryErro r ;
however, if you intend to use backwards axis, you'd rather "cast" small
chunks of your input in DOM subtrees, and use the full capabilities of XPath

an example here :
http://reflex.gforge.inria.fr/tutorial.html#N801C30
(the input file is not as big as yours, but big enough to make a simple
DOM parsing fail ; it was the case at the time I wrote this example)

in this example, you'll see how an inline-filter can split a big file in
about 13000 small files with 2 flavours : SAX to SAX and SAX to DOM

enjoy !

--
Cordialement,

///
(. .)
--------ooO--(_)--Ooo--------
| Philippe Poulard |
-----------------------------
http://reflex.gforge.inria.fr/
Have the RefleX !

[Martin Honnen]

Pascal Sartoretti wrote:

I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
processors that work in a "SAX-like" way (and not DOM-like), i.e. which
don't assume that they have everything in memory ?

SAXON-SA for XSLT (not for XQuery it seems) has a streaming processing
mode, see
<http://www.saxonica.co m/documentation/sourcedocs/serial.html>
--

Martin Honnen
http://JavaScript.FAQTs.com/

[Tony Lavinio]

DataDirect XQuery is optimized for very large files. It supports both
SAX and StAX modes. We've tested it with multi-gigabyte files.

See http://www.xquery.com

There is also an embedded version inside of Stylus Studio that you
can try out. See http://www.stylusstudio.com

Pascal Sartoretti wrote:

Hello,

I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
processors that work in a "SAX-like" way (and not DOM-like), i.e. which
don't assume that they have everything in memory ?

Thanks for any help

Pascal

--
Tony Lavinio <DataDirect <Stylus Studio XML <al******@progre ss.com
XQuery, XSLT, XML Schema and EDI Toolset <http://www.stylusstudio.com/
<There is no problem that brute force and ignorance cannot overcome <>

[Philippe Poulard]

....and it's free of charge :)

--
Cordialement,

///
(. .)
--------ooO--(_)--Ooo--------
| Philippe Poulard |
-----------------------------
http://reflex.gforge.inria.fr/
Have the RefleX !