473,322 Members | 1,610 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

Retrieving many RSS feeds fast?

I have a simple scripts using magpie to retrieve upto 200 RSS feeds.
Unfortunately it takes forever as the script has to sequentially get
each feeds, wait for the response, then move on. Is it possible to
retrieve them all at once or do them in parrallel?
Mar 30 '08 #1
1 1320
OK, me again, sorry for not being more specific earlier.

You should create a script (lets say parse_rss_ext.php) that parses
the RSS and outputs the result.
Then create another script (lets say parse_rss_core) that makes calls
to the parse_rss_ext.php script providing the RSS to be parsed and
receiving the response, then doing whatever needs to be done to the
response.

The code that emulates multi-threading in PHP is the following:

$rss_list is an array containing all the RSS feeds;

[code start]
$threads=array();
foreach ($rss_list as $rss)
$threads[$rss]=fopen('http://localhost/path/to/file/
parse_rss_ext.php?rss={$rss}','r');

foreach ($threads as $thread_rss=>thread)
if ($thread)
{
while (!feof($thread))
$responses[$thread_rss].=fgets($thread);
fclose($thread);
}
[code end]

Hope this helps you. Cheers!
Mar 30 '08 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

6
by: plazma_41 | last post by:
Are there any uses for rss feeds on commercial websites selling products/services? Yes! I've just realized that eMaximarket Auctions ( http://www.emaximarket.com ) provides RSS feeds. For...
1
by: Ajay Bhonsule via .NET 247 | last post by:
Hi, I am writing an application to retrieve RSS feeds from websites. While doing so I get the following error, Unhandled Exception: System.Net.WebException: The remote server returned an error:...
0
by: jmwatte | last post by:
OK... 2 RSS feeds ... "http://feeds.feedburner.com/brainyquote/QUOTENA" "http://feeds.feedburner.com/qotd" Now I build a RSS reader in C#Express(did several of them including the starterkit)...
5
by: Shawn | last post by:
Hi, I want to understand and follow RSS feed because many web pages provide RSS feed for updated new data(e.g. http://www.weather.gov/rss/). But I never get how to use it. The link below suppose...
1
by: _spitFIRE | last post by:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 I'm using feedparser library to parser ATOM/RSS feeds. However, I don't get the entire post! but only summaries! How do I retrieve the entire feed?...
4
by: mojde | last post by:
Hi all, I am using : 1. Windows XP-Visual Studio 2005 2. FastReport Studio Trial (www.fast-report.com) I have used FastReport Studio Trial (FastReport3.dll) in my ASP.NET web Site...
3
by: mojde | last post by:
Hi all, I am using : 1. Windows XP-Visual Studio 2005 2. FastReport Studio Trial (www.fast-report.com) I have used FastReport Studio Trial (FastReport3.dll) in my ASP.NET web Site...
1
by: =?Utf-8?B?cm9kY2hhcg==?= | last post by:
hey all, in fx 1.1 how did you know when you retrived the connection string from the web.config what kind of database it was? thanks rodchar
1
by: fidgen | last post by:
Hiya, I'm trying to get a AJAX driven update to my list of news articles, so when users click the title of the news article, it pops up the article content in a thickbox overlay. Retrieving...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.