473,320 Members | 1,817 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

XQuery on big files

Hello,

I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
processors that work in a "SAX-like" way (and not DOM-like), i.e. which
don't assume that they have everything in memory ?

Thanks for any help

Pascal
Apr 10 '07 #1
4 2457
Pascal Sartoretti wrote:
Hello,

I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
processors that work in a "SAX-like" way (and not DOM-like), i.e. which
don't assume that they have everything in memory ?

Thanks for any help

Pascal
hi,

with RefleX, you can process big files with XPath ; unlike many
streaming-XPath processors, RefleX doesn't restrict the XPath syntax,
but you must be aware that using XPath queries such as //* on the root
would cause caching the complete document causing an OutOfMemoryError ;
however, if you intend to use backwards axis, you'd rather "cast" small
chunks of your input in DOM subtrees, and use the full capabilities of XPath

an example here :
http://reflex.gforge.inria.fr/tutorial.html#N801C30
(the input file is not as big as yours, but big enough to make a simple
DOM parsing fail ; it was the case at the time I wrote this example)

in this example, you'll see how an inline-filter can split a big file in
about 13000 small files with 2 flavours : SAX to SAX and SAX to DOM

enjoy !

--
Cordialement,

///
(. .)
--------ooO--(_)--Ooo--------
| Philippe Poulard |
-----------------------------
http://reflex.gforge.inria.fr/
Have the RefleX !
Apr 10 '07 #2
Pascal Sartoretti wrote:
I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
processors that work in a "SAX-like" way (and not DOM-like), i.e. which
don't assume that they have everything in memory ?
SAXON-SA for XSLT (not for XQuery it seems) has a streaming processing
mode, see
<http://www.saxonica.com/documentation/sourcedocs/serial.html>
--

Martin Honnen
http://JavaScript.FAQTs.com/
Apr 10 '07 #3
DataDirect XQuery is optimized for very large files. It supports both
SAX and StAX modes. We've tested it with multi-gigabyte files.

See http://www.xquery.com

There is also an embedded version inside of Stylus Studio that you
can try out. See http://www.stylusstudio.com

Pascal Sartoretti wrote:
Hello,

I want to process big files (e.g. 100 MB) with XQuery. Are there XQuery
processors that work in a "SAX-like" way (and not DOM-like), i.e. which
don't assume that they have everything in memory ?

Thanks for any help

Pascal
--
Tony Lavinio <DataDirect <Stylus Studio XML <al******@progress.com
XQuery, XSLT, XML Schema and EDI Toolset <http://www.stylusstudio.com/
<There is no problem that brute force and ignorance cannot overcome <>
Apr 10 '07 #4
....and it's free of charge :)

--
Cordialement,

///
(. .)
--------ooO--(_)--Ooo--------
| Philippe Poulard |
-----------------------------
http://reflex.gforge.inria.fr/
Have the RefleX !
Apr 10 '07 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Tom Corcoran | last post by:
I am working to ease updating of a html page by transforming 2 xml files. I was going to use xslt for this and had bought 2 unopened books, wrox xslt and o'reilly's xslt cookbook. But am now...
0
by: Benjamin G. Jones | last post by:
I am having a very basic problem with XQuery. I want to use an XQuery API in Java (either Saxon or Qexo), and I have an XQuery expression that works as expeced from the command line if I specify...
0
by: Tony Lavinio | last post by:
Dear Stylus Studio Friends, The new year is scarcely one month old, but we already have lots to report! For starters, there's Stylus Studio 6 Release 2. The latest release of Stylus Studio...
1
by: FBS | last post by:
Hi everybody, I wonder if somebody could help me with this issue. I would like to rum some XQuery codes aginst my XML files and since I'm quite new in this area I'm not sure what to do. It is...
0
by: Stylus Studio | last post by:
DataDirect XQuery(TM) is the First Embeddable Component for XQuery That is Modeled after the XQuery API for Java(TM) (XQJ) BEDFORD, Mass.--Sept. 20, 2005--DataDirect Technologies...
2
by: Ale | last post by:
Hi all! I need to build a middleware for XQuery: I mean a software receiving a std xquery and outputting the xquery result on "database.xml". This middleware must use 7 or 8 different choosable...
2
by: tammo.mueller | last post by:
Hi, I'm trying to execute a xquery statement and I get always errors. It seems that only the prolog section is giving me errors. If I want to declare a namespace the xquery processor is coming...
0
by: Manish | last post by:
I am new to XML. Initially I was saving all the settings in either PHP or text files or database tables and use to parse the config variables from files or query from the database. Now I want to...
0
by: Cameron.MacNeil.024 | last post by:
Hi folks, hopefully you'll be able to help me with this problem I'm having. I have a bunch of xml files with format similar to the following: <?xml-stylesheet href='syntax.xsl'...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
0
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.