I have the following URL:
http://patft.uspto.gov/netacgi/nph-P...er)+OR+provide)
Accessing it with the following code snippet (which works for other sites):
HttpWebRequest req = (HttpWebRequest)WebRequest.Create(url);
HttpWebResponse resp = (HttpWebResponse)req.GetResponse();
throws a WebException (timeout). Accessing the same URL with IE works fine.
I compared the request header sent by the code and the one sent by IE and
they are identical. The user agent has been set to "Mozilla/4.0 (compatible;
MSIE 6.0; Windows NT 5.1; SV1; .NET CLR 1.1.4322; .NET CLR 2.0.50215)"
I even increased the request timeout to a massively huge value and it didn't
make any difference.