473,987 Members | 54,588 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Writing large set of data to xml files using DOM

Hello,

Need your valuable inputs for this scenario. Any pointers would be
helpful.

I have large data set (few 100s of MB or sometimes may even go up to
GB). I populate a DOM tree using this data set and then write this DOM
tree to xml file.
How should one handle such large data set in xerces, as we cannot have
such a large DOM tree in memory. I have a following questions:

1Can xerces handle such large data in DOM tree? If yes then how and
also does it do some caching to some temporary file on disk??

2Can I populate the DOM tree and flush it partially and in small
chunks?? If yes how?

I think there could be other alternatives than using DOM, but I want
to know whether xerces DOM can handle this.

Thanks in advance,
Kaustubh

Feb 9 '07 #1
2 2251
ka**********@gm ail.com wrote:
Hello,

Need your valuable inputs for this scenario. Any pointers would be
helpful.

I have large data set (few 100s of MB or sometimes may even go up to
GB). I populate a DOM tree using this data set and then write this DOM
tree to xml file.
How should one handle such large data set in xerces, as we cannot have
such a large DOM tree in memory. I have a following questions:
You'd be better off asking on a Apache forum or mail list.

--
Ian Collins.
Feb 9 '07 #2
ka**********@gm ail.com wrote:
Hello,

Need your valuable inputs for this scenario. Any pointers would be
helpful.

I have large data set (few 100s of MB or sometimes may even go up to
GB). I populate a DOM tree using this data set and then write this DOM
tree to xml file.
How should one handle such large data set in xerces, as we cannot have
such a large DOM tree in memory. I have a following questions:

1Can xerces handle such large data in DOM tree? If yes then how and
also does it do some caching to some temporary file on disk??

2Can I populate the DOM tree and flush it partially and in small
chunks?? If yes how?

I think there could be other alternatives than using DOM, but I want
to know whether xerces DOM can handle this.
You will need to consult a group for Xerxes if you need help using it.
This newsgroup is for questions about the Standard C++ language only.

Consult the C++ FAQ for more newsgroup information
http://www.parashift.com/c++-faq-lite/

--John Ratliff
Feb 9 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
23635
by: Sebastian Kemi | last post by:
How should a write a class to a file? Would this example work: object *myobject = 0; tfile.write(reinterpret_cast<char *>(myobject), sizeof(*object)); / sebek
6
2666
by: Greg | last post by:
I am working on a project that will have about 500,000 records in an XML document. This document will need to be queried with XPath, and records will need to be updated. I was thinking about splitting up the XML into several XML documents (perhaps 50,000 per document) to be more efficient but this will make things a lot more complex because the searching needs to go accross all 500,000 records. Can anyone point me to some best practices...
12
3790
by: Chris Springer | last post by:
I'd like to get some feedback on the issue of storing data out to disk and where to store it. I've never been in a production environment in programming so you'll have to bear with me... My question is about storing data in a database. Yes I understand that you can link to a database in your program and read and write to the database etc etc. Well, that's all find and dandy but what if the person you're writing the application for...
16
7228
by: Claudio Grondi | last post by:
I have a 250 Gbyte file (occupies the whole hard drive space) and want to change only eight bytes in this file at a given offset of appr. 200 Gbyte (all other data in that file should remain unchanged). How can I do that in Python? Claudio Grondi
6
5294
by: arne.muller | last post by:
Hello, I've come across some problems reading strucutres from binary files. Basically I've some strutures typedef struct { int i; double x; int n; double *mz;
5
5510
by: montyphyton | last post by:
as i understand there are two ways to write data to a file: using f.write("foo") and print >>f, "foo". what i want to know is which one is faster (if there is any difference in speed) since i'm working with very large files. of course, if there is any other way to write data to a file, i'd love to hear about it
12
5125
by: hemant.gaur | last post by:
I have an application which writes huge number of bytes into the binary files which is just some marshalled data. int len = Data.size(); //arrary size for (int i = 0; i < len; ++i) fwrite(&Data, 1, 1, f); now after running this for long time and pushing millions of bytes, It once misses writing the last byte of fData. Then the further push of bytes is again correct. As i am not using the return value for the
2
2270
by: mauricesmith42 | last post by:
Sorry i know this is rather large to be posting, but in order to understand the question you have to see all the code //#include <windows.h> //needed for opening folders #include <stdlib.h> //needed for converting integers to strings #include <iostream> //needed for in/out commands ie. cin/cout #include <fstream> //needed to open readable/writeable files ie. ifstream/ofstream #include <string> //needed to create strings from...
25
20625
by: tekctrl | last post by:
Anyone: I have a simple MSAccess DB which was created from an old ASCII flatfile. It works fine except for something that just started happening. I'll enter info in a record, save the record, and try to move to another record and get an Access error "Record is too large". The record is only half filled, with many empty fields. If I remove the added data or delete some older data, then it saves ok and works fine again. Whenever I'm...
0
10204
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
11899
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10979
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
8529
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7685
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
6480
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
6649
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
5234
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
3817
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.