Logon internetsite and download file

[Henrik]

Hi,,

Can anyone tell me how to log on to a internet website with username and
password, navigate to other subpages on that site, and then start downloading
a textfile to disk??

Thanks

[Mike Labosh]

Here's a snip from a web spider I wrote last year:
<snip>
Private Function getHtml(ByVal i As Integer) As String

Dim url As String = "http://www.wtng.info/wtng-" & Chr(i) & Chr(i) &
".html"
Dim reader As StreamReader
Dim html As String

Try
reader = New StreamReader( _
CType( _
CType( _
WebRequest.Create(url), _
HttpWebRequest).GetResponse(), _
HttpWebResponse).GetResponseStream())

html = reader.ReadToEnd()
reader.Close()

Catch ex As Exception
Try: reader.Close(): Catch exc As Exception: End Try
html = ""
Console.WriteLine("Could not connect to " & url)
End Try

Return html.Trim()

End Function
</snip>

--
Peace & happy computing,

Mike Labosh, MCSD
"Musha ring dum a doo dum a da!" -- James Hetfield
"Henrik" <He****@discussions.microsoft.com> wrote in message
news:70**********************************@microsof t.com...

Hi,,

Can anyone tell me how to log on to a internet website with username and
password, navigate to other subpages on that site, and then start
downloading
a textfile to disk??

Thanks

[Mike Labosh]

Here's a snip from a web spider I wrote last year:
<snip>
Private Function getHtml(ByVal i As Integer) As String

Dim url As String = "http://www.wtng.info/wtng-" & Chr(i) & Chr(i) &
".html"
Dim reader As StreamReader
Dim html As String

Try
reader = New StreamReader( _
CType( _
CType( _
WebRequest.Create(url), _
HttpWebRequest).GetResponse(), _
HttpWebResponse).GetResponseStream())

html = reader.ReadToEnd()
reader.Close()

Catch ex As Exception
Try: reader.Close(): Catch exc As Exception: End Try
html = ""
Console.WriteLine("Could not connect to " & url)
End Try

Return html.Trim()

End Function
</snip>

--
Peace & happy computing,

Mike Labosh, MCSD
"Musha ring dum a doo dum a da!" -- James Hetfield
"Henrik" <He****@discussions.microsoft.com> wrote in message
news:70**********************************@microsof t.com...

Hi,,

Can anyone tell me how to log on to a internet website with username and
password, navigate to other subpages on that site, and then start
downloading
a textfile to disk??

Thanks

[Herfried K. Wagner [MVP]]

"Henrik" <He****@discussions.microsoft.com> schrieb:

Can anyone tell me how to log on to a internet website with username and
password, navigate to other subpages on that site, and then start
downloading
a textfile to disk??

\\\
Dim w As New WebClient()
w.Credentials = New NetworkCredential("username", "password")
w.DownloadFile(...)
///

--
M S Herfried K. Wagner
M V P <URL:http://dotnet.mvps.org/>
V B <URL:http://classicvb.org/petition/>

[Herfried K. Wagner [MVP]]

"Henrik" <He****@discussions.microsoft.com> schrieb:

Can anyone tell me how to log on to a internet website with username and
password, navigate to other subpages on that site, and then start
downloading
a textfile to disk??

\\\
Dim w As New WebClient()
w.Credentials = New NetworkCredential("username", "password")
w.DownloadFile(...)
///

--
M S Herfried K. Wagner
M V P <URL:http://dotnet.mvps.org/>
V B <URL:http://classicvb.org/petition/>

[Henrik]

Hi Mike,,

Thanks for you reply,, But it's not quite that im looking for?

I want to connect to a site, where in need to type in my username and
password on a form and submit this form. After valid check i get a new page,
where i can download a ".txt"-file. And it's this file im interesed in..

I've got the download procedure to work just fine by using a
"WebClient1"-component.. But my main problem is the logon-procedure to the
site. I've tried this:

Dim w As New WebClient
w.Credentials = New NetworkCredential("username", "password")
w.DownloadFile("https://Reports/xx/xxx.TXT", "c:\xxx.TXT")

It's saves a file,, but its the logonpage??

"Mike Labosh" wrote:

Here's a snip from a web spider I wrote last year:
<snip>
Private Function getHtml(ByVal i As Integer) As String

Dim url As String = "http://www.wtng.info/wtng-" & Chr(i) & Chr(i) &
".html"
Dim reader As StreamReader
Dim html As String

Try
reader = New StreamReader( _
CType( _
CType( _
WebRequest.Create(url), _
HttpWebRequest).GetResponse(), _
HttpWebResponse).GetResponseStream())

html = reader.ReadToEnd()
reader.Close()

Catch ex As Exception
Try: reader.Close(): Catch exc As Exception: End Try
html = ""
Console.WriteLine("Could not connect to " & url)
End Try

Return html.Trim()

End Function
</snip>

--
Peace & happy computing,

Mike Labosh, MCSD
"Musha ring dum a doo dum a da!" -- James Hetfield
"Henrik" <He****@discussions.microsoft.com> wrote in message
news:70**********************************@microsof t.com...

Hi,,

Can anyone tell me how to log on to a internet website with username and
password, navigate to other subpages on that site, and then start
downloading
a textfile to disk??

Thanks

[Henrik]

Hi Mike,,

Thanks for you reply,, But it's not quite that im looking for?

I want to connect to a site, where in need to type in my username and
password on a form and submit this form. After valid check i get a new page,
where i can download a ".txt"-file. And it's this file im interesed in..

I've got the download procedure to work just fine by using a
"WebClient1"-component.. But my main problem is the logon-procedure to the
site. I've tried this:

Dim w As New WebClient
w.Credentials = New NetworkCredential("username", "password")
w.DownloadFile("https://Reports/xx/xxx.TXT", "c:\xxx.TXT")

It's saves a file,, but its the logonpage??

"Mike Labosh" wrote:

Here's a snip from a web spider I wrote last year:
<snip>
Private Function getHtml(ByVal i As Integer) As String

Dim url As String = "http://www.wtng.info/wtng-" & Chr(i) & Chr(i) &
".html"
Dim reader As StreamReader
Dim html As String

Try
reader = New StreamReader( _
CType( _
CType( _
WebRequest.Create(url), _
HttpWebRequest).GetResponse(), _
HttpWebResponse).GetResponseStream())

html = reader.ReadToEnd()
reader.Close()

Catch ex As Exception
Try: reader.Close(): Catch exc As Exception: End Try
html = ""
Console.WriteLine("Could not connect to " & url)
End Try

Return html.Trim()

End Function
</snip>

--
Peace & happy computing,

Mike Labosh, MCSD
"Musha ring dum a doo dum a da!" -- James Hetfield
"Henrik" <He****@discussions.microsoft.com> wrote in message
news:70**********************************@microsof t.com...

Hi,,

Can anyone tell me how to log on to a internet website with username and
password, navigate to other subpages on that site, and then start
downloading
a textfile to disk??

Thanks

[Henrik]

Hi Herfried,,

Thanks for replying :o)

I have tried that?,, it only takes me to the loginform with the username,
password and submitbutton :o(.. And i have even tried to write in the whole
url to the file, but still no luck??

Do i really have to "fill" in username and password and make the
submit-action programly instead?? and the call the url with the file to
download afterwards?

Thanks

"Herfried K. Wagner [MVP]" wrote:

"Henrik" <He****@discussions.microsoft.com> schrieb:

Can anyone tell me how to log on to a internet website with username and
password, navigate to other subpages on that site, and then start
downloading
a textfile to disk??

\\\
Dim w As New WebClient()
w.Credentials = New NetworkCredential("username", "password")
w.DownloadFile(...)
///

--
M S Herfried K. Wagner
M V P <URL:http://dotnet.mvps.org/>
V B <URL:http://classicvb.org/petition/>

[Henrik]

Hi Herfried,,

Thanks for replying :o)

I have tried that?,, it only takes me to the loginform with the username,
password and submitbutton :o(.. And i have even tried to write in the whole
url to the file, but still no luck??

Do i really have to "fill" in username and password and make the
submit-action programly instead?? and the call the url with the file to
download afterwards?

Thanks

"Herfried K. Wagner [MVP]" wrote:

"Henrik" <He****@discussions.microsoft.com> schrieb:

Can anyone tell me how to log on to a internet website with username and
password, navigate to other subpages on that site, and then start
downloading
a textfile to disk??

\\\
Dim w As New WebClient()
w.Credentials = New NetworkCredential("username", "password")
w.DownloadFile(...)
///

--
M S Herfried K. Wagner
M V P <URL:http://dotnet.mvps.org/>
V B <URL:http://classicvb.org/petition/>

[MARTIN LANNY]

I have the exact same problem.
I need to download the csv file from Google AdSense.
Google requires the name and password to be send using "POST" method.

This code (below) will only download the html content of the page, but
if url points to file it doesn't do anything. Can someone amend this
code to work for files as well?

Public Function GetPageHTML(ByVal URL As String) As String
Dim myWebClient As New WebClient
Dim myNameValueCollection As New NameValueCollection
myNameValueCollection.Add("username", "joedoe")
myNameValueCollection.Add("password", "123456")
Dim responseArray As Byte() = myWebClient.UploadValues(URL,
"POST", myNameValueCollection)
GetPageHTML = Encoding.UTF8.GetString(responseArray)
End Function

[Henrik]

Fixed my problem by doing this:

Dim Request As HttpWebRequest
Dim Response As HttpWebResponse
Dim URL As String = "https://login.form"
Dim Params As String = "action=login&username=" & User & "&password=" & Pwd
& "&submit=login"
Dim CC As New CookieContainer
Dim myWriter As StreamWriter
Dim sr As StreamReader

Request = WebRequest.Create(url)
Request.Timeout = 120 * 1000
Request.CookieContainer = CC
Request.Method = "POST"
Request.ContentLength = params.Length
Request.ContentType = "application/x-www-form-urlencoded"
Request.KeepAlive = True
myWriter = New StreamWriter(Request.GetRequestStream())
myWriter.Write(params)

myWriter.Close()
Response = Request.GetResponse()
Response.Cookies = Request.CookieContainer.GetCookies(Request.Request Uri)

For i = 0 To Response.Cookies.Count - 1
CC.Add(Response.Cookies(i))
Next

sr = New StreamReader(Response.GetResponseStream(), Encoding.GetEncoding(850))
ResultHTML = sr.ReadToEnd()
-----------------------------------------------------------------------------

Then i could create a new request and navigate to the page with my file..

Thanks for your help


"MARTIN LANNY" wrote:

I have the exact same problem.
I need to download the csv file from Google AdSense.
Google requires the name and password to be send using "POST" method.

This code (below) will only download the html content of the page, but
if url points to file it doesn't do anything. Can someone amend this
code to work for files as well?

Public Function GetPageHTML(ByVal URL As String) As String
Dim myWebClient As New WebClient
Dim myNameValueCollection As New NameValueCollection
myNameValueCollection.Add("username", "joedoe")
myNameValueCollection.Add("password", "123456")
Dim responseArray As Byte() = myWebClient.UploadValues(URL,
"POST", myNameValueCollection)
GetPageHTML = Encoding.UTF8.GetString(responseArray)
End Function