469,579 Members | 1,214 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,579 developers. It's quick & easy.

Best way to read a 1GB XML file with C#?

Hello all,

I have a 1GB XML file that I need to read once a day and I would like to get
feedback to find out what is the most efficient way to go about reading this
file. The application reading this is in C# and I am using .NET 2.0.

How can I read it without loading it all at once into memory? Will a simple
XmlDocument and a XmlNodeIterator work?

All feedback is extremely appreciated. Thanks!

Johnny
Nov 22 '05 #1
3 18147
Johnny wrote:
Hello all,

I have a 1GB XML file that I need to read once a day and I would like to get
feedback to find out what is the most efficient way to go about reading this
file. The application reading this is in C# and I am using .NET 2.0.

How can I read it without loading it all at once into memory? Will a simple
XmlDocument and a XmlNodeIterator work?

All feedback is extremely appreciated. Thanks!

Johnny


You probably want to use the XmlReader and not load it all into a
document all at once ( although these days, you may have enough memory
to handle a file that large ). Still, I wouldn't do it.

Then you'd basically read it in node by node ( looking for the end node
) for the data your interested in, and store that as XmlNode, then use
it for whatever you want to do with it.

--
Texeme
http://texeme.com

Nov 22 '05 #2
Using XmlDocument or XmlNodeDocument will parse the entire document upon
loading. For large documents you will need a non cached XmlReader, but you
still have to parse the document yourself.

You can consider generating and using an XmlNameTable to increase
performance:
Efficient Techniques for Modifying Large XML Files
By Dare Obasanjo
http://msdn.microsoft.com/library/de...l/largexml.asp

Object Comparison Using XmlNameTable with XmlReader
http://msdn.microsoft.com/library/de...hXmlReader.asp

ok,
aq

"Elementary Penguin" <si*****@hari.krishna> wrote in message
news:BG***************@newsread3.news.pas.earthlin k.net...
Johnny wrote:
Hello all,

I have a 1GB XML file that I need to read once a day and I would like to get feedback to find out what is the most efficient way to go about reading this file. The application reading this is in C# and I am using .NET 2.0.

How can I read it without loading it all at once into memory? Will a simple XmlDocument and a XmlNodeIterator work?

All feedback is extremely appreciated. Thanks!

Johnny


You probably want to use the XmlReader and not load it all into a
document all at once ( although these days, you may have enough memory
to handle a file that large ). Still, I wouldn't do it.

Then you'd basically read it in node by node ( looking for the end node
) for the data your interested in, and store that as XmlNode, then use
it for whatever you want to do with it.

--
Texeme
http://texeme.com

Nov 22 '05 #3
Johnny,
In addition to the other comments, using an XmlReader might be the safest
way, as you only need to read in the section you need.

The following article identifies some new features in .NET 2.0 with Xml that
might help you.

http://msdn.microsoft.com/library/de...sysxmlvs05.asp

Hope this helps
Jay

"Johnny" <Jo****@discussions.microsoft.com> wrote in message
news:72**********************************@microsof t.com...
Hello all,

I have a 1GB XML file that I need to read once a day and I would like to
get
feedback to find out what is the most efficient way to go about reading
this
file. The application reading this is in C# and I am using .NET 2.0.

How can I read it without loading it all at once into memory? Will a
simple
XmlDocument and a XmlNodeIterator work?

All feedback is extremely appreciated. Thanks!

Johnny

Nov 22 '05 #4

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

4 posts views Thread by Frank Millman | last post: by
136 posts views Thread by Matt Kruse | last post: by
4 posts views Thread by ESPN Lover | last post: by
10 posts views Thread by jojobar | last post: by
3 posts views Thread by gordon | last post: by
4 posts views Thread by TC | last post: by
4 posts views Thread by Brian | last post: by
4 posts views Thread by guiromero | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.