archival of newsfeeds?

[cody]

I want to program a feedreader which is able to archive all messages so that
I can view messages from weeks or months ago.

The question is now *how* to archive them. Since the feeds can have
different formats do I have to convert them in my own format? Is it better
to store them in a database or is it better to use a large xml file?
Will I still have satisfiing performance if I search a XmlDocument for a
newsfeed containing specific words in the title or feed having a specific
category?

[Pascal Schmitt]

Hello!

The question is now *how* to archive them. Since the feeds can have
different formats do I have to convert them in my own format?
Atom & RSS have equal features, you could for example use Atom
internally with own additions to support RSS features (or swapped).

Is it better
to store them in a database or is it better to use a large xml file?
Will I still have satisfiing performance if I search a XmlDocument for a
newsfeed containing specific words in the title or feed having a specific
category?

For optimal Performance, use a Database. But this not as flexible as
xml-files wich you could search using XPath.

Always try to avoid using the XmlDocument, especially for large files.
They will be read into memory completely wich is a total waste of
Resources (why parse & build a DOM-Tree of 10MB XML when you just want
to read the first Elements text value?).
For best comfort, use the XPathDocument wich allows you to use XPath on
streamed xml (eg. it is not loaded into memory) for even more
Performance but more specific and schema-centric code use Xml(Text)Reader.
--
Pascal Schmitt

[cody]

> > The question is now *how* to archive them. Since the feeds can have

different formats do I have to convert them in my own format?

Atom & RSS have equal features, you could for example use Atom
internally with own additions to support RSS features (or swapped).

> Is it better
to store them in a database or is it better to use a large xml file?
Will I still have satisfiing performance if I search a XmlDocument for a
newsfeed containing specific words in the title or feed having a specific category?

For optimal Performance, use a Database. But this not as flexible as
xml-files wich you could search using XPath.

Always try to avoid using the XmlDocument, especially for large files.
They will be read into memory completely wich is a total waste of
Resources (why parse & build a DOM-Tree of 10MB XML when you just want
to read the first Elements text value?).
For best comfort, use the XPathDocument wich allows you to use XPath on
streamed xml (eg. it is not loaded into memory) for even more
Performance but more specific and schema-centric code use Xml(Text)Reader.

But if I want to search within my feed is XmlDocument the right solution or
is there a better way?
How fast is XPath? Does it simply walk through all nodes or are there
optimized algorithms used, for example hashing?

[Pascal Schmitt]

Hello!

For best comfort, use the XPathDocument wich allows you to use XPath on
streamed xml (eg. it is not loaded into memory) for even more
Performance but more specific and schema-centric code use Xml(Text)Reader.
But if I want to search within my feed is XmlDocument the right solution or
is there a better way?

XPathDocument. If there is no need to modify anything, use it!
(and IF you need to modify a big XML file consider using XmlTextReader &
XmlTextWriter simultaneously: read data, modify it, write it at once -
not as nice too look at as DOM operations but really fast.)

XPathDocument x = new XPathDocument("file.xml");
int f = (int)(double)x.CreateNavigator().Evaluate("count(//foo)");

How fast is XPath? Does it simply walk through all nodes or are there
optimized algorithms used, for example hashing?

Afaik there is no need for optimisation because XPath just walks the
Document using an XPathNavigator (wich both XPathDocument and
XmlDocument implement but XPathDocument is faster but does not allow
editing data until .NET 2.0).
--
Pascal Schmitt

[cody]

But the problem when I use xml files is that if I want to modifiy them, I
have to rewrite the entire file, right?
"Pascal Schmitt" <ne*******@cebra.nu> schrieb im Newsbeitrag
news:uF*************@tk2msftngp13.phx.gbl...

Hello!

For best comfort, use the XPathDocument wich allows you to use XPath on
streamed xml (eg. it is not loaded into memory) for even more
Performance but more specific and schema-centric code use
Xml(Text)Reader.

But if I want to search within my feed is XmlDocument the right solution
or
is there a better way?

XPathDocument. If there is no need to modify anything, use it!
(and IF you need to modify a big XML file consider using XmlTextReader &
XmlTextWriter simultaneously: read data, modify it, write it at once - not
as nice too look at as DOM operations but really fast.)

XPathDocument x = new XPathDocument("file.xml");
int f = (int)(double)x.CreateNavigator().Evaluate("count(//foo)");

How fast is XPath? Does it simply walk through all nodes or are there
optimized algorithms used, for example hashing?

Afaik there is no need for optimisation because XPath just walks the
Document using an XPathNavigator (wich both XPathDocument and XmlDocument
implement but XPathDocument is faster but does not allow editing data
until .NET 2.0).
--
Pascal Schmitt