472,958 Members | 2,338 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 472,958 software developers and data experts.

Read source code of a web page

Hi All,

I use VC .Net 2003 and I want to create a small program which is able to
read the source code of a web site. I have a textbox with the URL, a button
and a multiline textbox for the source code. If the web page is secure, how
can I post my login details?

Any help appreciated.

Regards,
Nicolae
Jul 21 '05 #1
5 1935
"Nicolae Fieraru" <no****@please.com> wrote:
Hi All,

I use VC .Net 2003 and I want to create a small program which is able to
read the source code of a web site. I have a textbox with the URL, a button
and a multiline textbox for the source code. If the web page is secure, how
can I post my login details?

Any help appreciated.

Regards,
Nicolae


Depends on how the server is secured. In some cases

http://username:pa******@www.domain.com/path/filename

may work but in the majority of cases you'll have to use the
WebClient class and set up its Credentials property.

WebClient Class
http://msdn.microsoft.com/library/de...classtopic.asp

WebClient.Credentials
http://msdn.microsoft.com/library/de...tialstopic.asp

NetworkCredential Class
http://msdn.microsoft.com/library/de...classtopic.asp

NetworkCredential Constructor
http://msdn.microsoft.com/library/de...sctortopic.asp
'The real crux of the "software crisis" is that software IS hard.'
Robert C. Martin,
'Designing Object-Oriented C++ Applications Using the Booch Method', p. ix
Jul 21 '05 #2
The advice from UAError is only appropriate if the web site is on YOUR
computer. If you are trying to read the html contents of a page that is
running on someone else's computer, you will need to be able to log in to
that site. I assume you are talking about a situation where you can log in,
but you want to download the HTML to an application.

Most sites do what .Net does, and that is to make a login page available
that uses cookies. In this model, you log in to the server. The server
checks its database or directory for your credentials. If found, the server
issues a token. This is a string of characters to you, but means something
to the server. It provides that token to your site as a cookie.

All subsequent pages on the site require the cookie. Web browsers already
do this. If a cookie comes from a web site, with instructions for how long
the cookie should remain in the browser, the browser will automatically pass
the cookie back to every subsequent request from that web site.

So, your app has to pretend to be a browser. You have to use the GET and
POST methods of the HTTPWebRequest class to ask for the login page and to
provide the credentials. You will get back a collection of cookies. You
need to attach that cookie collection to all subsequent Web Requests against
that site.

So, look to the HTTPWebRequest HTTPWebResponse and CookieCollection classes
for further information on how to do what you want.

Note: you said you wanted to read the "source code" of the site. You will
be able to get the HTML, not the source code per se. If you want the C# or
ASP.NET code for a site, you will need direct access to the filesystem it is
on. That will normally require the cooperation of the system administrator.
In that case, my advice is meaningless. (I'm guessing here).

I hope this helps,

--
--- Nick Malik [Microsoft]
MCSD, CFPS, Certified Scrummaster
http://blogs.msdn.com/nickmalik

Disclaimer: Opinions expressed in this forum are my own, and not
representative of my employer.
I do not answer questions on behalf of my employer. I'm just a
programmer helping programmers.
--
"Nicolae Fieraru" <no****@please.com> wrote in message
news:O8**************@TK2MSFTNGP09.phx.gbl...
Hi All,

I use VC .Net 2003 and I want to create a small program which is able to
read the source code of a web site. I have a textbox with the URL, a button and a multiline textbox for the source code. If the web page is secure, how can I post my login details?

Any help appreciated.

Regards,
Nicolae

Jul 21 '05 #3
"Nick Malik [Microsoft]" <ni*******@hotmail.nospam.com>
wrote:
The advice from UAError is only appropriate if the web site is on YOUR
computer.
Surely you mean a computer within your domain, not your own
local computer? When I'm wrong, I'm wrong but the latter
would seem so restrictive that it is useless. The sample

How To Use WebClient Class To Make HTTP Requests
http://support.microsoft.com/default...b;en-us;328820

states that WebClient supports
- Basic authentication
- Integrated Windows authentication

<snip>All subsequent pages on the site require the cookie. Web browsers already
do this. If a cookie comes from a web site, with instructions for how long
the cookie should remain in the browser, the browser will automatically pass
the cookie back to every subsequent request from that web site.
I was under the (now obviously mistaken) impression that
WebClient was like a stripped down browser. I guess I should
have looked for a Cookies or CookieContainer property and
drawn my own conclusions when it wasn't there.
You have to use the GET and POST methods of the HTTPWebRequest
class to ask for the login page and to provide the credentials.


You had me confused here until I found the
HttpWebRequest.Method property, which was impossible to find
under "Public Methods" :)
Thanks for straightening this out.
Jul 21 '05 #4
sorry for not being clear and I certainly didn't mean to offend.

Thanks for taking my remarks in a good spirit.

--
--- Nick Malik [Microsoft]
MCSD, CFPS, Certified Scrummaster
http://blogs.msdn.com/nickmalik

Disclaimer: Opinions expressed in this forum are my own, and not
representative of my employer.
I do not answer questions on behalf of my employer. I'm just a
programmer helping programmers.
--
Jul 21 '05 #5
The advice from UAError is only appropriate if the web site is on YOUR
computer. If you are trying to read the html contents of a page that is
running on someone else's computer, you will need to be able to log in to
that site. I assume you are talking about a situation where you can log
in,
but you want to download the HTML to an application.


That is not true Nick, if you use Internet Explorer as browser than you see
an icon "Edit" which will allow you to change the page content you are on
even with notepad. (The only point is you can only save it on your own
computers)

Cor
Jul 21 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
by: Noozer | last post by:
I am able to place values on forms that load into my webbrowser control, but I'd like to be able to scan the page for data. I know to use the HTMLDocument object. Basically I'm hoping to find an...
6
by: Dung Ping | last post by:
A while ago I posted a message saying that saving Javascript code as external file might hide the code better. All replies to it said it would still be very easy for people to look at the source...
4
by: Kevin Vogler | last post by:
I need to read the value of a hidden field on a remote HTML page and then use that value in a form submit on my local page. How can I read the hidden field programatically? Thanks in advance....
5
by: Nicolae Fieraru | last post by:
Hi All, I use VC .Net 2003 and I want to create a small program which is able to read the source code of a web site. I have a textbox with the URL, a button and a multiline textbox for the...
17
by: chris.schwalm | last post by:
I would like to first state that I have searched through the archives and found a lot of related material, but am still new enough to javascript that I can't fit all the pieces together yet. So...
6
by: Karthik | last post by:
I am trying to read Http request body values NOT form values The content type is text/xml and its POST I am posting data from Infopath to an asp.net page. In ASP it works by just writing...
2
by: keithb | last post by:
This is on a remote server. Everything works in the development envirmonment. Any ideas? Thanks, Keith Failed to update database "C:\BTM_ROOT\APP_DATA\ASPNETDB.MDF" because the database is...
4
by: Kuldeep | last post by:
Hi All, I am trying to read the contents of a page through its URL. My code snippet is as follows: public void mtdGetPageDataHWR() { HttpWebRequest objRequ =...
9
by: =?Utf-8?B?TWlrZQ==?= | last post by:
Hi. Is it programatically possible in VB.NET to read the contents of web.config's <customErrorselement? I looked at using ConfigurationSettings.AppSettings, but that doesn't work. I need to...
2
by: DJRhino | last post by:
Was curious if anyone else was having this same issue or not.... I was just Up/Down graded to windows 11 and now my access combo boxes are not acting right. With win 10 I could start typing...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 4 Oct 2023 starting at 18:00 UK time (6PM UTC+1) and finishing at about 19:15 (7.15PM) The start time is equivalent to 19:00 (7PM) in Central...
0
by: Aliciasmith | last post by:
In an age dominated by smartphones, having a mobile app for your business is no longer an option; it's a necessity. Whether you're a startup or an established enterprise, finding the right mobile app...
0
tracyyun
by: tracyyun | last post by:
Hello everyone, I have a question and would like some advice on network connectivity. I have one computer connected to my router via WiFi, but I have two other computers that I want to be able to...
3
NeoPa
by: NeoPa | last post by:
Introduction For this article I'll be using a very simple database which has Form (clsForm) & Report (clsReport) classes that simply handle making the calling Form invisible until the Form, or all...
1
by: Teri B | last post by:
Hi, I have created a sub-form Roles. In my course form the user selects the roles assigned to the course. 0ne-to-many. One course many roles. Then I created a report based on the Course form and...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 1 Nov 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM) Please note that the UK and Europe revert to winter time on...
0
NeoPa
by: NeoPa | last post by:
Introduction For this article I'll be focusing on the Report (clsReport) class. This simply handles making the calling Form invisible until all of the Reports opened by it have been closed, when it...
0
isladogs
by: isladogs | last post by:
The next online meeting of the Access Europe User Group will be on Wednesday 6 Dec 2023 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, Mike...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.