473,662 Members | 2,593 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

How to save all search engine results urls in a text file

16 New Member
From this search engine:

https://siteexplorer.se arch.yahoo.com/mysites

when I make all subdomains search just by typing a domain url in a search box like
blogspot.com -and after I verify my yahoo email password I get 28 millions webpages listed on thousands of search result pages.I want to save the all urls and again only the one that have flv files in a text file so some flash downloader preferably freeware could preview or download them selectively or all.
Dec 26 '06 #1
2 1698
AricC
1,892 Recognized Expert Top Contributor
I'm not really sure what you are asking.
Dec 27 '06 #2
dbojan
16 New Member
I'm not really sure what you are asking.
On this search engine:

https://siteexplorer.se arch.yahoo.com/mysites

go enter the domain url in the search box (the one where you would enter keywords normaly before the invention of this search engine) .Enter

blogspot.com

you will get on the top left corner of the search page notification that there were

28 million websites found.Each web search page can list 50 of that number.
I want to copy as much as I want these urls (that are under the title of each found webpage on a search page)to have them copied till the end if I don't interrupt them and can acces them even unfinished and continue their saving from the place they have stopped -save them into a text file.Note that yahoo site explorer lists only the subdomains of the domains we enter and lists nothing else.

so for example we got only two websites of one search engine quest:

1. title A
(www.1domain.co m/1subdomain.html )

2.title B
(www.1domain.co m/2subdomain.html )

all I would like is to have them copied in a text file:

www.1domain.com/1subdomain.html
www.1domain.com/2subdomain.html

and if there were 28 million of such than 28 millions such lines were be written.Name it

1a.txt

and again to copy but only the urls of the websites that contain flash -flv,swf movies
inside them.Save them as 1b.txt

and again,if possible some seo data of each webpage.E.g number of views,rank.As

1c.txt
Dec 27 '06 #3

Sign in to post your reply or Sign up for a free account.

Similar topics

1
3419
by: phpkid | last post by:
Howdy I've been given conflicting answers about search engines picking up urls like: http://mysite.com/index.php?var1=1&var2=2&var3=3 Do search engines pick up these urls? I've been considering converting a site of mine to PHP-Nuke, but if the individual modules aren't picked up in search engines I'm not going to do it. Thanks phpKid
11
4210
by: Petre Huile | last post by:
I have designed a site for a client, but they have hired an internet marketing person to incrase their search engine ranking and traffic. He wants to put extra-large fonts on every page which will make the design looks a bit rediculous. He also said that the big text cannot be hidden. I am just trying to find a compromise. Here are the questions: (1) Is it true that a page with an <H1> tag and very big font size will make a search...
5
2975
by: George | last post by:
Hi, Anyone has the background for explaining? I have made a search on my name and I have got a link to another search engine. The link's title was the search phrase for the other search engine (a wicked phrase). Some but not all of the words of that phrase exist in my website. I reported it and they removed the link but did not give explanations.
5
2060
by: Sam | last post by:
Does anyone know of a way to create a search page under ASP.NET 2.0? I have started out by configuring a catalog in Index Server, registering the aspx, ascx extensions in the registry to allow them to be indexed and built the catalog as per KB article, but I've run into an interesting problem. When you publish a website from Whidbey, it precompiles everything and strips out the searchable details of the page (metadata, html, etc)...
8
3659
by: david.lindsay.green | last post by:
Hello all, I am quite new a web scripting and making web pages in general and I have stumbled across a problem I have as yet been unable to solve. I am trying to take the contents of a textarea box and save it to a file. This step is not to hard however the contents of the textarea is mostly latex source so it contains just about every special character you can imagine. My question is this, how do I save an exact copy of the textarea...
5
2385
by: dbojan | last post by:
preferably only of the one that contain flash -flv files. All search results not only one displayed page.
0
1017
by: dbojan | last post by:
From this search engine: https://siteexplorer.search.yahoo.com/mysites when I make all subdomains search just by typing a domain url in a search box like blogspot.com -and after I verify my yahoo email password I get 28 millions webpages listed on thousands of search result pages.I want to save the all urls and again only the one that have flv files in a text file so some flash downloader preferably freeware could preview or download...
2
1636
by: anbaxter | last post by:
I have a small challenge and you'll have to excuse me because I haven’t touched JS for some time and have gotten a bit rusty. I have an intranet site at work that has roughly 500,000 htm pages (no joke). I have a search engine (zoom) that returns results of the various htm files. The site that I indexed is setup with about 41,600 folders with each folder containing 12 to 13 htm files. One of the htm files is a default.htm file that is...
8
2119
by: Bruno Rafael Moreira de Barros | last post by:
I have this framework I'm building in PHP, and it has Search Engine Friendly URLs, with site.com/controller/page/args... And on my View files, I have <?=$this->baseURL;?to print the base URL on the links (eg. <a href='<?=$this->baseURL;?>/controller/page/args'>Go somewhere</ a>. But on the CSS / JS files, how will I do it? I wonder, because on the View files, I can do <?=$this->baseURL;?>/css/site.css, and it will work. But images on the...
0
8344
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
8857
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
8764
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8546
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8633
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7367
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6186
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
4347
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
2
1752
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.