473,406 Members | 2,707 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,406 software developers and data experts.

How to save all search engine results urls in a text file

16
From this search engine:

https://siteexplorer.search.yahoo.com/mysites

when I make all subdomains search just by typing a domain url in a search box like
blogspot.com -and after I verify my yahoo email password I get 28 millions webpages listed on thousands of search result pages.I want to save the all urls and again only the one that have flv files in a text file so some flash downloader preferably freeware could preview or download them selectively or all.
Dec 26 '06 #1
1 909
I don't know if there is a *smarter* solution, but to my knowledge, you can use a *stupid* way to do so.


Basically your program takes the content of the result page. Then you put it as a string, and try to parse the string. I know that sounds stupid, but since those search engines would do paging for you, that's the only idea i come up with so far.


If anyone has a better idea, i would love to hear it as well.


Hope this helps...


Ben
Dec 27 '06 #2

Sign in to post your reply or Sign up for a free account.

Similar topics

1
by: phpkid | last post by:
Howdy I've been given conflicting answers about search engines picking up urls like: http://mysite.com/index.php?var1=1&var2=2&var3=3 Do search engines pick up these urls? I've been considering...
11
by: Petre Huile | last post by:
I have designed a site for a client, but they have hired an internet marketing person to incrase their search engine ranking and traffic. He wants to put extra-large fonts on every page which will...
5
by: George | last post by:
Hi, Anyone has the background for explaining? I have made a search on my name and I have got a link to another search engine. The link's title was the search phrase for the other search engine...
5
by: Sam | last post by:
Does anyone know of a way to create a search page under ASP.NET 2.0? I have started out by configuring a catalog in Index Server, registering the aspx, ascx extensions in the registry to allow...
8
by: david.lindsay.green | last post by:
Hello all, I am quite new a web scripting and making web pages in general and I have stumbled across a problem I have as yet been unable to solve. I am trying to take the contents of a textarea box...
4
by: zakhirn | last post by:
Hello, I have an extremely new user to XML and XSL, and I would like to know how to search XML data via a input form in HTML, and have the results displayed in HTML. If anyone has code that...
5
by: dbojan | last post by:
preferably only of the one that contain flash -flv files. All search results not only one displayed page.
2
by: dbojan | last post by:
From this search engine: https://siteexplorer.search.yahoo.com/mysites when I make all subdomains search just by typing a domain url in a search box like blogspot.com -and after I verify my...
2
by: anbaxter | last post by:
I have a small challenge and you'll have to excuse me because I haven’t touched JS for some time and have gotten a bit rusty. I have an intranet site at work that has roughly 500,000 htm pages...
8
by: Bruno Rafael Moreira de Barros | last post by:
I have this framework I'm building in PHP, and it has Search Engine Friendly URLs, with site.com/controller/page/args... And on my View files, I have <?=$this->baseURL;?to print the base URL on the...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.