quickest way to read xml from the web

[Emmanuel]

Hi there,

My client would like to process an xml file. the structure of which is as
below.
<xml>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<xml>
Now, the latest stock item will always be on top, thus if today i've process
all of the above xml stockitems, then tomorrow, all i need
to do is to process the first couple of xml stockitem elements (tags) which
are greater than yesterdays <releaseddate>.
This xml file can and does grow pretty fast (couple of megs) and is cleaned
up by the vendor only every couple of months (if not weeks).
I need to process the xml file such that as soon as i find an xml stockitem
which has the releaseddate as something that i've processed, then to finish
processing the xml file.
Which means i do not want to download the entire xml file for processing,
rather bring down only chunks (bits) of the xml file and read it line by
line (or stockitem by stockitem).

Can this be done?

(Currently i load the entire xml into an xmldocument class and then process
it).

[Karl Seguin]

Well, the quickest way to process an XML file is with the XML Reader. The
simplest is with the XMLDocument. You could use the XMLDocument with little
effort by retrieving all new items based on an xPath. However, this would
load the entire document into memory. If it's just a couple megs, I think
I'd start off with the XMLDocument and only move to the XMLReader if
necessary.

You mention that you want to "download" only the necessary part of the file.
I'm not sure what your question is anymore. Is it "what's the quickest way
to process part of an XML file" or "How to download only part of the file"?
Is the file not local to your code and must be downloaded whenever you want
to process it? If it's the latter, then the solution is more complicated.
I believe that even if you use the System.Net classes to get a stream to
your resource, once you call GetResponse() the entire response is on your
local computer. You can give it a try, but you'll really be limited by
whatever protocol you use to access the file.

Karl

--
MY ASP.Net tutorials
http://www.openmymind.net/
http://openmymind.net/redirector.aspx?documentId=51 - Learn about AJAX!

"Emmanuel" <[e-mail]> wrote in message
news:eM**************@TK2MSFTNGP14.phx.gbl...

Hi there,

My client would like to process an xml file. the structure of which is as
below.
<xml>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<xml>
Now, the latest stock item will always be on top, thus if today i've
process
all of the above xml stockitems, then tomorrow, all i need
to do is to process the first couple of xml stockitem elements (tags)
which
are greater than yesterdays <releaseddate>.
This xml file can and does grow pretty fast (couple of megs) and is
cleaned
up by the vendor only every couple of months (if not weeks).
I need to process the xml file such that as soon as i find an xml
stockitem
which has the releaseddate as something that i've processed, then to
finish
processing the xml file.
Which means i do not want to download the entire xml file for processing,
rather bring down only chunks (bits) of the xml file and read it line by
line (or stockitem by stockitem).

Can this be done?

(Currently i load the entire xml into an xmldocument class and then
process
it).

[Emmanuel]

Thank you for your help on this matter.
I need this entire process to be quick,
- processing the file
- downloading the xml file.

As you've mentioned the XMLDocument will load the entire document into the
memory (which probably means that the XML file will first have to be
downloaded from the site), which could take some time.

But i'll give this a go,
Once again, thanks.

"Karl Seguin" <karl REMOVE @ REMOVE openmymind REMOVEMETOO . ANDME net>
wrote in message news:ee****************@TK2MSFTNGP09.phx.gbl...

Well, the quickest way to process an XML file is with the XML Reader. The
simplest is with the XMLDocument. You could use the XMLDocument with
little effort by retrieving all new items based on an xPath. However,
this would load the entire document into memory. If it's just a couple
megs, I think I'd start off with the XMLDocument and only move to the
XMLReader if necessary.

You mention that you want to "download" only the necessary part of the
file. I'm not sure what your question is anymore. Is it "what's the
quickest way to process part of an XML file" or "How to download only part
of the file"? Is the file not local to your code and must be downloaded
whenever you want to process it? If it's the latter, then the solution is
more complicated. I believe that even if you use the System.Net classes to
get a stream to your resource, once you call GetResponse() the entire
response is on your local computer. You can give it a try, but you'll
really be limited by whatever protocol you use to access the file.

Karl

--
MY ASP.Net tutorials
http://www.openmymind.net/
http://openmymind.net/redirector.aspx?documentId=51 - Learn about AJAX!

"Emmanuel" <[e-mail]> wrote in message
news:eM**************@TK2MSFTNGP14.phx.gbl...

Hi there,

My client would like to process an xml file. the structure of which is as
below.
<xml>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<stockitem>
<releaseddate>.....date value...</releaseddate>
<...aditional tags for additional info>
</stockitem>
<xml>
Now, the latest stock item will always be on top, thus if today i've
process
all of the above xml stockitems, then tomorrow, all i need
to do is to process the first couple of xml stockitem elements (tags)
which
are greater than yesterdays <releaseddate>.
This xml file can and does grow pretty fast (couple of megs) and is
cleaned
up by the vendor only every couple of months (if not weeks).
I need to process the xml file such that as soon as i find an xml
stockitem
which has the releaseddate as something that i've processed, then to
finish
processing the xml file.
Which means i do not want to download the entire xml file for processing,
rather bring down only chunks (bits) of the xml file and read it line by
line (or stockitem by stockitem).

Can this be done?

(Currently i load the entire xml into an xmldocument class and then
process
it).