473,385 Members | 1,888 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,385 software developers and data experts.

Finding Broken Link in WebSite

Hi everyone,
i want .net(VB or C#) code for finding broken links in a website.
The requirement is that the user will be able to type the
url in a text box so once the button is clicked , it has to show
whether there are any broken links in that particular page.
Please help me out in this.


Thanks
Sridhar.S
Feb 11 '08 #1
8 5044
wimpos
19
You can download the website entered into the textbox.
try using the webclient class
Now you have the source html code of the webpage.
Than find al the links in the webpages <a href="**"></a> (maybe a method in webclient, otherwise use a regex)
extract the ** and try to download this website as you did before. If it succeeds the link is alive, otherwise it is dead.

This is a guidline you can follow. try it, if you run into trouble don't hesitate to get more detailed info.

regards
W.
Feb 11 '08 #2
kenobewan
4,871 Expert 4TB
I find this question interesting because a link isn't really broken until you click and get the 404 error. Also an ok local link may be broken on the web. Having said that there are third party tools that may help, but my assumption is that it may be a case of 'physician heal thyself".

Having said that a site map may help, but you would need to customize to cover all dynamic links. Here is a great resource on TDD, applied mainly to software but if web programming was done this way...
My Uber-Test-Driven Development ("TDD") Links Listing
Feb 11 '08 #3
Plater
7,872 Expert 4TB
W3.org offers testing for broken links.
All they do is check to see if a good status is returned when they attempt to navigate to the address inside an anchor tag.
Feb 11 '08 #4
Hi all,

I'm storing a set of "<a href" tags in a list. Now i want to check all these links are valid or not? i.e, the list contains any broken links or not.

This is the code.....

the code will allow user to enter the url and it will render all the html code back
and display only the href tags. Now i want to check the validity of all these links
and display the result.... i.e all the links are not dead.....

Expand|Select|Wrap|Line Numbers
  1. // make an object of the WebClient class 
  2.             WebClient objWebClient = new WebClient();
  3.          // gets the HTML from the url written in the textbox
  4.             aRequestHTML = objWebClient.DownloadData(TextBox1.Text); 
  5.          // creates UTf8 encoding object
  6.             UTF8Encoding utf8 = new UTF8Encoding();  
  7.         // gets the UTF8 encoding of all the html we got in aRequestHTML
  8.             myString = utf8.GetString(aRequestHTML); 
  9.  
  10.  
  11.             ArrayList list = new ArrayList();
  12.             int curindex = 0;
  13.             int index = 0;
  14.             do
  15.             {
  16.                 index = myString.IndexOf("<a href=", curindex);
  17.                 if (index==-1) {break;}
  18.                 curindex = myString.IndexOf(">", index);
  19.                 string ancordata = myString.Substring(index, curindex - index);
  20.                 if (ancordata.ToLower().IndexOf("javascript") < 0)
  21.                 {                    
  22.                     list.Add(ancordata);
  23.                 }
  24.             } 
  25.             while (index != -1);
  26.               GridView1.DataSource = list; 
  27.         //// binds the databind
  28.             GridView1.DataBind();
  29.  
Feb 13 '08 #5
You can download the website entered into the textbox.
try using the webclient class
Now you have the source html code of the webpage.
Than find al the links in the webpages <a href="**"></a> (maybe a method in webclient, otherwise use a regex)
extract the ** and try to download this website as you did before. If it succeeds the link is alive, otherwise it is dead.

This is a guidline you can follow. try it, if you run into trouble don't hesitate to get more detailed info.

regards
W.
Hi thanks for ur reply,

i have passed half the ocean.
Now i have all the list of "<a href" tags in a array list.
I'm totally stuck here. Please help me.
Feb 13 '08 #6
Plater
7,872 Expert 4TB
Try using an HttpWebRequest to see if they return a good status or not?
Feb 14 '08 #7
Try using an HttpWebRequest to see if they return a good status or not?
Spot on. For each url, use the code at http://msdn2.microsoft.com/en-us/library/system.net.httpwebrequest.getresponse.aspx, check the status code of the response. 200 is OK, while 404 is the typical broken link. Other status codes will require some judgement.

John
http://blogs.msdn.com/usisvde/
Feb 18 '08 #8
wimpos
19
Tip:
It might be not that important but I still recommend using a regular expression to search for the <a href="" >

It's cleaner, it 's faster, more reliable

regards
Feb 20 '08 #9

Sign in to post your reply or Sign up for a free account.

Similar topics

2
by: Danny | last post by:
Anyone knows how to detect broken link using ASP? Just say the link is got from query string, ie: http://www.mydomain.com/check_validity_link.asp?www.otherdomain.com/1.htm TIA.
1
by: talyabn | last post by:
Hi, I'm trying to invoke the 'Broken Hyperlinks' option in the FrontPage application. The problem is that I get all the links in a given HTML page instead of getting only the broken links. ...
1
by: | last post by:
I am planning to develop a directory website (ASP.NET) which will contain links to hundreds of external web pages. In an effort to keep the directory up to date, I would like to trap (perhaps as an...
0
by: Martin Atukunda | last post by:
On this page: http://techdocs.postgresql.org/techdocs/ this section: Development the link: http://techdocs.postgresql.org/redir.php?link=http://www.postgresql.org/docs/momjian/writing_apps...
28
by: Craig Cockburn | last post by:
I have a tool which tells me the number of times that visitors attempt to access a link from my site to an external site and what the response code received was. In the event of the remote site...
11
by: sweetbox | last post by:
Hi Guys! I am new on this forum and I am hoping that I could acquire help to improve my PERL skill. My goal is to be able to create a report of all broken links from a website. Based from my...
4
by: Mel | last post by:
My link does not work. I want to create a link to a file that exists on a different server. The path to the file is this "\\Junebug\Groups \90000003.pdf". My web server is "10.10.1.111". How do...
8
by: punk86 | last post by:
Hi, i been working on this codes but i keep getting broken links for the pictures. Im using apache. Need help for this please. I think its just the codes in my index.php is wrong and i do not know...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.