473,320 Members | 2,189 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Read past large XML nodes (800 MB)

Hello there,

Anybody out there who can show me how to read a large xml-file without
getting Out of Memory exception?

I am currently using xmlTextReader but when the reader gets to an element
which contains about 200-800 MB of data the program exits with an exception.

I would like to read the elment tag but not the element's data - if the
element is not the elements I'm looking for I would like to jump to the next
startElement. That sounds a lot like the functionality of
reader.startElement - but the program still exits with Out of Memory
exception.
Thanks in advance,
Kim
Nov 15 '05 #1
2 2770
"Kim Rasmussen" <ki***@NOSPAMPLEASEmobilixnet.dk> wrote in message
news:c0***********@news.cybercity.dk...
Anybody out there who can show me how to read a large xml-file without
getting Out of Memory exception?

I am currently using xmlTextReader but when the reader gets to an element
which contains about 200-800 MB of data the program exits with an

exception.

Wow... It may not be much help telling you now (if the XML files are just
handed to you) but XML files (implemented as a file system file) were
obviously never meant to be that large. So my answer would be to store the
lump somewhere else and include references to the lump in your XML document.

Martin.
Nov 15 '05 #2
Kim,
Try using XPathDocument xpd=new XPathDocument("Bigfile.xml"); XPathNavigator
xpn=xpd.CreateNavigator();
Then use XPathNodeIterator to step through the large file using MoveNext()
method.

Thanks,
Fakher Halim

"Kim Rasmussen" <ki***@NOSPAMPLEASEmobilixnet.dk> wrote in message
news:c0***********@news.cybercity.dk...
Hello there,

Anybody out there who can show me how to read a large xml-file without
getting Out of Memory exception?

I am currently using xmlTextReader but when the reader gets to an element
which contains about 200-800 MB of data the program exits with an exception.
I would like to read the elment tag but not the element's data - if the
element is not the elements I'm looking for I would like to jump to the next startElement. That sounds a lot like the functionality of
reader.startElement - but the program still exits with Out of Memory
exception.
Thanks in advance,
Kim

Nov 15 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Paul Arnold | last post by:
We are currently in the process of developing a client/server based generic tree application with which we want to be able to model and render any size, shape and depth of tree. Our primary...
29
by: pb648174 | last post by:
I have a very long transaction that runs on the same database that other users need to use for existing data. I don't care if they see data from the transaction before it is done and am only using...
75
by: Greg McIntyre | last post by:
I have a Python snippet: f = open("blah.txt", "r") while True: c = f.read(1) if c == '': break # EOF # ... work on c Is some way to make this code more compact and simple? It's a bit...
4
by: Gerrit | last post by:
It must be simple, but I don't find how I can read a XmlFile in an ArrayList. Sample of my XmlFile: <?xml version="1.0" encoding="utf-8" ?> <Relations> <Person> <FirstName>John</FirstName>...
0
by: Ntuthuko | last post by:
I'm designing an online form for a client using javascript and vbscript. After a user has filled in, the form must be converted to pdf (I'm using ABCpdf5). The problem I have is that the form can be...
1
sujathaeeshan
by: sujathaeeshan | last post by:
<POLICY> <POLINPUTS> <FLD> <ISCPM>FF</ISCPM> <ISVPM>UU</ISVPM> 800</FLD> <FLD> <ISCPM>FF1</ISCPM> <ISVPM>UU1</ISVPM> 901</FLD>
5
by: chrispoliquin | last post by:
I need to represent the hyperlinks between a large number of HTML files as a graph. My non-directed graph will have about 63,000 nodes and and probably close to 500,000 edges. I have looked...
6
by: =?Utf-8?B?VGFtbXkgTmVqYWRpYW4=?= | last post by:
Hi, can someone please let me know how I can read xml elements using object oriented program. I created a class to use the get and set properties however I dont know how I can pass the values from...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.