473,396 Members | 2,039 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

Issue with httpwebrequest and redirect

I am having an issue with using the httpwebrequest to
perform a screenscrape. The site I am retrieving data
from has forms based authentication. When I pass the
correct credentials it authenticates properly, but then I
get an error saying too many redirections attempted. Here
is the code I am using:
--------------------------------------------------------
Private Sub Button1_Click(ByVal sender As
System.Object, ByVal e As System.EventArgs) Handles
Button1.Click
Dim x As String

x = readHtmlPage("http://awebsite.com")

End Sub

Function readHtmlPage(ByVal url As String) As String
Dim objResponse As WebResponse

Dim result As String

Dim myCred As New NetworkCredential("uname", "pw")
Dim myCache As New CredentialCache
myCache.Add(New Uri(url), "Basic", myCred)
Dim wr As HttpWebRequest = HttpWebRequest.Create
(url)
wr.Credentials = myCache
wr.AllowAutoRedirect = True
wr.MaximumAutomaticRedirections = 5

Dim myWebResponse As HttpWebResponse =
wr.GetResponse()

Dim sr As New StreamReader(myWebResponse.GetResponseStream
())
result = sr.ReadToEnd()

'clean up StreamReader
sr.Close()

Return result
End Function

----------------------------------------------

Can anyone assist?

Thanks,

Scott

Jul 19 '05 #1
1 3538
"scott rosa" <sc********@analog.com> wrote in message news:<0a****************************@phx.gbl>...
I am having an issue with using the httpwebrequest to
perform a screenscrape. The site I am retrieving data
from has forms based authentication. When I pass the
correct credentials it authenticates properly, but then I
get an error saying too many redirections attempted.

Scott -

Is there a reason you set the MaximumAutomaticRedirections
property to 5? The default is 50. Also, with forms based
authentication, the login credential is stored as a cookie. I don't
see where you capture cookies returned from the web site. Hope this
helps out some.

Rich Blum
Author of "C# Network Programming" (Sybex)
http://www.sybex.com/sybexbooks.nsf/Booklist/4176
Jul 19 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Nick Jacobsen | last post by:
I am using HTTPWebRequest to connect to a server which requires authentication, then responds with a 302 redirect message, and redirects to DIFFERANT server, which also requires authentication (the...
4
by: Vivek | last post by:
how to add the headers to httpwebrequest/httpwebresponse from .net code ?
1
by: scott rosa | last post by:
I am having an issue with using the httpwebrequest to perform a screenscrape. The site I am retrieving data from has forms based authentication. When I pass the correct credentials it...
2
by: TK | last post by:
I have a trouble to get web resopnse from an aspx page which is secured by Forms Authentication with custom user account database. My client application is a console application but not a browser....
3
by: Hawksey | last post by:
I would like to be able to programatically(Using HTTPWebRequest and HTTPWebResponse) FORM post a user's credentials to a HTTPS login page (.ascx)control. The problem I think I have after attempting...
6
by: Oliver | last post by:
I have a very wired problem requesting one specific url from within my application. I have struggeled with this for 5 hours now, and searched google withour any luck, so i hope that someone are...
0
by: Alex Papadimoulis | last post by:
Hey Group, I'm in the process of converting an ASP-based site to an ASP.NET site and built a control that wraps around an ASP page. The control simply does a GET to the same server to render the...
0
by: Alex Papadimoulis | last post by:
Hey Group, I'm in the process of converting an ASP-based site to an ASP.NET site and built a control that wraps around an ASP page. The control simply does a GET to the same server to render the...
0
by: barrybevel | last post by:
Hi, I'm trying to login to the www.vodafone.ie website using HttpWebRequest. It works fine with IE/Firefox and the .NET Web Control too, just not with my code. I think it's a redirect 302...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.