473,480 Members | 1,750 Online
Bytes | Software Development & Data Engineering Community
Create Post

Home Posts Topics Members FAQ

RSS Search

Hi! I'm looking for ideas on what would the best approach to design a
search system for a RSS feeds. I will have some 50 RSS feeds (all RSS
2.0 compliant) stored locally on the web server. Now I'm wondering
what would the best method to allow searching of these RSS files.
Since the search will cater to multiple users the search system has to
be robust and efficient. Some ideas that I have for the RSS search
system are:

1. Store all RSS files locally on the web server file system and
perform file system queries. But I guess this might get slow when a
number of users try to search. Moreover, the queries may not be
extensible (for example to allow boolean operations etc).

2. Move the RSS data to the database and then search perform search
using LIKE (or the more advanced indexing service features).

3. Use a 3rd party full-text search engine like Lucene.

4. Use something like XQuery or XPath to query the RSS files directly
but this again *might* (not sure since I haven't worked with either)
get slow when a number of users try to search.

Also, the RSS files I have on the web server will be updated every
hour or so.

So, I have the ideas but I'm not quite sure which one would the most
suitable and efficient. If anyone has ideas on implementing such a
search system for RSS feeds then please share your insight. Thank you
guys!
Nov 16 '05 #1
2 1159
You could use SQL Server 2005's XML datatype and XQuery support (it is
currently in Beta2).

Best regards
Michael

"clintonG" <cs*********@REMOVETHISTEXTmetromilwaukee.com> wrote in message
news:O$**************@TK2MSFTNGP11.phx.gbl...
I'm not surprised to see nobody has responded to your questions.
I'm working on the same type of issues and all I have learned so
far is that SQL Server 2000 would require 'shredding' the data
and putting it into the database where the server could be used
to return results all other current options not being performance
friendly.

--
<%= Clinton Gallagher, "Twice the Results -- Half the Cost"
Architectural & e-Business Consulting -- Software Development
NET cs*********@REMOVETHISTEXTmetromilwaukee.com
URL http://www.metromilwaukee.com/clintongallagher/

"RiceGuy" <9i*************@jetable.org> wrote in message
news:d7**************************@posting.google.c om...
Hi! I'm looking for ideas on what would the best approach to design a
search system for a RSS feeds. I will have some 50 RSS feeds (all RSS
2.0 compliant) stored locally on the web server. Now I'm wondering
what would the best method to allow searching of these RSS files.
Since the search will cater to multiple users the search system has to
be robust and efficient. Some ideas that I have for the RSS search
system are:

1. Store all RSS files locally on the web server file system and
perform file system queries. But I guess this might get slow when a
number of users try to search. Moreover, the queries may not be
extensible (for example to allow boolean operations etc).

2. Move the RSS data to the database and then search perform search
using LIKE (or the more advanced indexing service features).

3. Use a 3rd party full-text search engine like Lucene.

4. Use something like XQuery or XPath to query the RSS files directly
but this again *might* (not sure since I haven't worked with either)
get slow when a number of users try to search.

Also, the RSS files I have on the web server will be updated every
hour or so.

So, I have the ideas but I'm not quite sure which one would the most
suitable and efficient. If anyone has ideas on implementing such a
search system for RSS feeds then please share your insight. Thank you
guys!


Nov 16 '05 #2
I've got it loaded on another disk and have to re-cable to get
to it so it happens infrequently but its good to know this task
is going to become easier. Thanks for bringing it to my attention.

I'll certainly have to study 2005 features as I've asked for
comments at news://microsoft.public.sqlserver.programming to
discuss how others are 'shredding' XML data into SQL Server.

--
<%= Clinton Gallagher, "Twice the Results -- Half the Cost"
Architectural & e-Business Consulting -- Software Development
NET cs*********@REMOVETHISTEXTmetromilwaukee.com
URL http://www.metromilwaukee.com/clintongallagher/

"Michael Rys [MSFT]" <mr**@online.microsoft.com> wrote in message
news:OR*************@TK2MSFTNGP10.phx.gbl...
You could use SQL Server 2005's XML datatype and XQuery support (it is
currently in Beta2).

Best regards
Michael

"clintonG" <cs*********@REMOVETHISTEXTmetromilwaukee.com> wrote in message
news:O$**************@TK2MSFTNGP11.phx.gbl...
I'm not surprised to see nobody has responded to your questions.
I'm working on the same type of issues and all I have learned so
far is that SQL Server 2000 would require 'shredding' the data
and putting it into the database where the server could be used
to return results all other current options not being performance
friendly.

--
<%= Clinton Gallagher, "Twice the Results -- Half the Cost"
Architectural & e-Business Consulting -- Software Development
NET cs*********@REMOVETHISTEXTmetromilwaukee.com
URL http://www.metromilwaukee.com/clintongallagher/

"RiceGuy" <9i*************@jetable.org> wrote in message
news:d7**************************@posting.google.c om...
Hi! I'm looking for ideas on what would the best approach to design a
search system for a RSS feeds. I will have some 50 RSS feeds (all RSS
2.0 compliant) stored locally on the web server. Now I'm wondering
what would the best method to allow searching of these RSS files.
Since the search will cater to multiple users the search system has to
be robust and efficient. Some ideas that I have for the RSS search
system are:

1. Store all RSS files locally on the web server file system and
perform file system queries. But I guess this might get slow when a
number of users try to search. Moreover, the queries may not be
extensible (for example to allow boolean operations etc).

2. Move the RSS data to the database and then search perform search
using LIKE (or the more advanced indexing service features).

3. Use a 3rd party full-text search engine like Lucene.

4. Use something like XQuery or XPath to query the RSS files directly
but this again *might* (not sure since I haven't worked with either)
get slow when a number of users try to search.

Also, the RSS files I have on the web server will be updated every
hour or so.

So, I have the ideas but I'm not quite sure which one would the most
suitable and efficient. If anyone has ideas on implementing such a
search system for RSS feeds then please share your insight. Thank you
guys!



Nov 16 '05 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
4112
by: R. Rajesh Jeba Anbiah | last post by:
Q: Is PHP search engine friendly? Q: Will search engine spiders crawl my PHP pages? A: Spiders should crawl anything provided they're accessible. Since, nowadays most of the websites are been...
1
8697
by: Les Juby | last post by:
A year or two back I needed a search script to scan thru HTML files on a client site. Usual sorta thing. A quick search turned up a neat script that provided great search results. It was fast,...
5
2948
by: George | last post by:
Hi, Anyone has the background for explaining? I have made a search on my name and I have got a link to another search engine. The link's title was the search phrase for the other search engine...
3
4182
by: Alastair | last post by:
Hello guys, I've been building a search facility for an intranet site I'm part of developing and we've been building a search engine using Index Server. It mostly works, however there have been...
39
4349
by: Noticedtrends | last post by:
Can inference search-engines narrow-down the number of often irrelevant results, by using specific keywords; for the purpose of discerning emerging social & business trends? For example, if...
22
11077
by: Phlip | last post by:
C++ers: Here's an open ended STL question. What's the smarmiest most templated way to use <string>, <algorithms> etc. to turn this: " able search baker search charlie " into this: " able...
28
3124
by: joshc | last post by:
If I have an array of data that I know to be sorted in increasing order, and the array is less than 50 elements, and I want to find the first element greater than a certain value, is a simple...
4
2157
by: BenCoo | last post by:
Hello, In a Binary Search Tree I get the error : Object must be of type String if I run the form only with the "Dim bstLidnummer As New BinarySearchTree" it works fine. Thanks for any...
1
7508
Merlin1857
by: Merlin1857 | last post by:
How to search multiple fields using ASP A major issue for me when I first started writing in VB Script was constructing the ability to search a table using multiple field input from a form and...
0
2427
by: passion | last post by:
"Specialized Search Engines" along with Google Search Capability (2 in 1): http://specialized-search-engines.blogspot.com/ Billions of websites are available on the web and plenty of extremely...
0
7046
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
7048
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
7088
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
1
6741
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
6956
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
5342
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...
0
2997
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The...
0
1300
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated ...
0
183
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.