By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
435,165 Members | 894 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 435,165 IT Pros & Developers. It's quick & easy.

Finding Broken Link in WebSite

P: 3
Hi everyone,
i want .net(VB or C#) code for finding broken links in a website.
The requirement is that the user will be able to type the
url in a text box so once the button is clicked , it has to show
whether there are any broken links in that particular page.
Please help me out in this.


Thanks
Sridhar.S
Feb 11 '08 #1
Share this Question
Share on Google+
8 Replies


P: 19
You can download the website entered into the textbox.
try using the webclient class
Now you have the source html code of the webpage.
Than find al the links in the webpages <a href="**"></a> (maybe a method in webclient, otherwise use a regex)
extract the ** and try to download this website as you did before. If it succeeds the link is alive, otherwise it is dead.

This is a guidline you can follow. try it, if you run into trouble don't hesitate to get more detailed info.

regards
W.
Feb 11 '08 #2

kenobewan
Expert 2.5K+
P: 4,871
I find this question interesting because a link isn't really broken until you click and get the 404 error. Also an ok local link may be broken on the web. Having said that there are third party tools that may help, but my assumption is that it may be a case of 'physician heal thyself".

Having said that a site map may help, but you would need to customize to cover all dynamic links. Here is a great resource on TDD, applied mainly to software but if web programming was done this way...
My Uber-Test-Driven Development ("TDD") Links Listing
Feb 11 '08 #3

Plater
Expert 5K+
P: 7,872
W3.org offers testing for broken links.
All they do is check to see if a good status is returned when they attempt to navigate to the address inside an anchor tag.
Feb 11 '08 #4

P: 3
Hi all,

I'm storing a set of "<a href" tags in a list. Now i want to check all these links are valid or not? i.e, the list contains any broken links or not.

This is the code.....

the code will allow user to enter the url and it will render all the html code back
and display only the href tags. Now i want to check the validity of all these links
and display the result.... i.e all the links are not dead.....

Expand|Select|Wrap|Line Numbers
  1. // make an object of the WebClient class 
  2.             WebClient objWebClient = new WebClient();
  3.          // gets the HTML from the url written in the textbox
  4.             aRequestHTML = objWebClient.DownloadData(TextBox1.Text); 
  5.          // creates UTf8 encoding object
  6.             UTF8Encoding utf8 = new UTF8Encoding();  
  7.         // gets the UTF8 encoding of all the html we got in aRequestHTML
  8.             myString = utf8.GetString(aRequestHTML); 
  9.  
  10.  
  11.             ArrayList list = new ArrayList();
  12.             int curindex = 0;
  13.             int index = 0;
  14.             do
  15.             {
  16.                 index = myString.IndexOf("<a href=", curindex);
  17.                 if (index==-1) {break;}
  18.                 curindex = myString.IndexOf(">", index);
  19.                 string ancordata = myString.Substring(index, curindex - index);
  20.                 if (ancordata.ToLower().IndexOf("javascript") < 0)
  21.                 {                    
  22.                     list.Add(ancordata);
  23.                 }
  24.             } 
  25.             while (index != -1);
  26.               GridView1.DataSource = list; 
  27.         //// binds the databind
  28.             GridView1.DataBind();
  29.  
Feb 13 '08 #5

P: 3
You can download the website entered into the textbox.
try using the webclient class
Now you have the source html code of the webpage.
Than find al the links in the webpages <a href="**"></a> (maybe a method in webclient, otherwise use a regex)
extract the ** and try to download this website as you did before. If it succeeds the link is alive, otherwise it is dead.

This is a guidline you can follow. try it, if you run into trouble don't hesitate to get more detailed info.

regards
W.
Hi thanks for ur reply,

i have passed half the ocean.
Now i have all the list of "<a href" tags in a array list.
I'm totally stuck here. Please help me.
Feb 13 '08 #6

Plater
Expert 5K+
P: 7,872
Try using an HttpWebRequest to see if they return a good status or not?
Feb 14 '08 #7

P: 6
Try using an HttpWebRequest to see if they return a good status or not?
Spot on. For each url, use the code at http://msdn2.microsoft.com/en-us/library/system.net.httpwebrequest.getresponse.aspx, check the status code of the response. 200 is OK, while 404 is the typical broken link. Other status codes will require some judgement.

John
http://blogs.msdn.com/usisvde/
Feb 18 '08 #8

P: 19
Tip:
It might be not that important but I still recommend using a regular expression to search for the <a href="" >

It's cleaner, it 's faster, more reliable

regards
Feb 20 '08 #9

Post your reply

Sign in to post your reply or Sign up for a free account.