473,383 Members | 1,846 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,383 software developers and data experts.

search an entire website given the homepage URL

I know I can use urllib2 to get at a website given urllib2.urlopen(url)
but I'm unsure how to then go through all pages that are linked to it,
but still in the domain. If I want to search through the entire python
website give the homepage, how would I go about it? I don't reinvent
the wheel if someones already written a module for parsing out the links
and drilling through them!

TIA
KEV
Apr 25 '06 #1
0 991

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

67
by: Sandy.Pittendrigh | last post by:
Here's a question I don't know the answer to: I have a friend who makes very expensive, hand-made bamboo flyrods. He's widely recognized (in the fishing industry) as one of the 3-5 'best' rod...
83
by: D. Dante Lorenso | last post by:
Trying to use the 'search' in the docs section of PostgreSQL.org is extremely SLOW. Considering this is a website for a database and databases are supposed to be good for indexing content, I'd...
1
by: Jack-of-all-traits | last post by:
This is a big problem and I really need help, no one seems to know how to solve this problem. I want to take the data from a record in a particular a field that contains the names of generic...
60
by: Julie | last post by:
What is the *fastest* way in .NET to search large on-disk text files (100+ MB) for a given string. The files are unindexed and unsorted, and for the purposes of my immediate requirements, can't...
8
by: Stimp | last post by:
I need to take my website offline for an hour or so while I perform some maintenance... is it possible to create a 'catch-all' redirect to a temporary page (i.e. something in Web.config or...
1
by: newspost2000 | last post by:
If anyone could help me out here. I am the webmaster of the following corporate website: http://www.otpp.com/. When I load this page in IE or Netscape for PC the postion of all of the elements on...
1
by: HYRY | last post by:
I want to add some simple search function for my homepage. It need to search through all the html files of my homepage (about 300 pages), and highlight the search words. I made some test with...
11
by: emailus | last post by:
I am webmaster for the domain <www.alpha1.org.au>. Not being an expert in html, I take advantage of my domain Registrant's web building tool, 'Instant Website'. This tool is provided as part of...
4
by: Twisted01 | last post by:
Hello, I am new here and I hope I have the correct forum to answer my question. I am creating a website for a friend and they need a "Search Form" so that visitors can search for items on the...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.