Long files: HttpWebRequest & StreamRead

[Brent]

I'm hoping to grab some 70,000 text files over http using the code
below. For the most part, the files are a few KB in size, and are
downloaded quickly. The code meets its Waterloo, however, with a rare
behemoth of 3 MB. I bumped up the Timeout property to 10 minutes (or, at
least I think I did) in hopes it wouldn't fail, but it fails still.

I'd appreciate any tips to fix the problem!

--Brent
==================================

public string get13f (string strURL)
{
try
{
HttpWebRequest oRequest = (HttpWebRequest)WebRequest.Create(strURL);
oRequest.Timeout = 10*60000; // 10 minutes; for long files (10000 =
10 seconds)
oRequest.UserAgent = "Web Client";
HttpWebResponse oResponse = (HttpWebResponse)oRequest.GetResponse();

Stream myStream = oResponse.GetResponseStream();
StreamReader sr = new StreamReader(myStream);
string strResponse = sr.ReadToEnd();
return strResponse;
myStream.Close();
}
catch
{
return "0";
}
}

==================================

[Juan T. Llibre]

Check your maxRequestLength setting.

<httpRuntime executionTimeout="90" maxRequestLength="#ofKBtodownload"

maxRequestLength="8192" would allow 8MB.

Juan T. Llibre
ASP.NET MVP
http://asp.net.do/foros/
Foros de ASP.NET en Español
Ven, y hablemos de ASP.NET...
======================

"Brent" <""b b i g l e r \"@ y a h o o . c o m"> wrote in message
news:11*************@corp.supernews.com...

I'm hoping to grab some 70,000 text files over http using the code below. For the most
part, the files are a few KB in size, and are downloaded quickly. The code meets its
Waterloo, however, with a rare behemoth of 3 MB. I bumped up the Timeout property to 10
minutes (or, at least I think I did) in hopes it wouldn't fail, but it fails still.

I'd appreciate any tips to fix the problem!

--Brent
==================================

public string get13f (string strURL)
{
try
{
HttpWebRequest oRequest = (HttpWebRequest)WebRequest.Create(strURL);
oRequest.Timeout = 10*60000; // 10 minutes; for long files (10000 = 10 seconds)
oRequest.UserAgent = "Web Client";
HttpWebResponse oResponse = (HttpWebResponse)oRequest.GetResponse();

Stream myStream = oResponse.GetResponseStream();
StreamReader sr = new StreamReader(myStream);
string strResponse = sr.ReadToEnd();
return strResponse;
myStream.Close();
}
catch
{
return "0";
}
}

==================================

[Juan T. Llibre]

Check your maxRequestLength setting.

<httpRuntime executionTimeout="90" maxRequestLength="#ofKBtodownload"

maxRequestLength="8192" would allow 8MB.

Juan T. Llibre
ASP.NET MVP
http://asp.net.do/foros/
Foros de ASP.NET en Español
Ven, y hablemos de ASP.NET...
======================

"Brent" <""b b i g l e r \"@ y a h o o . c o m"> wrote in message
news:11*************@corp.supernews.com...

I'm hoping to grab some 70,000 text files over http using the code below. For the most
part, the files are a few KB in size, and are downloaded quickly. The code meets its
Waterloo, however, with a rare behemoth of 3 MB. I bumped up the Timeout property to 10
minutes (or, at least I think I did) in hopes it wouldn't fail, but it fails still.

I'd appreciate any tips to fix the problem!

--Brent
==================================

public string get13f (string strURL)
{
try
{
HttpWebRequest oRequest = (HttpWebRequest)WebRequest.Create(strURL);
oRequest.Timeout = 10*60000; // 10 minutes; for long files (10000 = 10 seconds)
oRequest.UserAgent = "Web Client";
HttpWebResponse oResponse = (HttpWebResponse)oRequest.GetResponse();

Stream myStream = oResponse.GetResponseStream();
StreamReader sr = new StreamReader(myStream);
string strResponse = sr.ReadToEnd();
return strResponse;
myStream.Close();
}
catch
{
return "0";
}
}

==================================

[Brent]

Juan:

Thanks for your help. This is a good suggestion, but I've discovered
the problem is a memory error. I'm going to post another thread on that
error specifically.

Thanks again.

--Brent

Juan T. Llibre wrote:

Check your maxRequestLength setting.

<httpRuntime executionTimeout="90" maxRequestLength="#ofKBtodownload"

maxRequestLength="8192" would allow 8MB.

[Brent]

Juan:

Thanks for your help. This is a good suggestion, but I've discovered
the problem is a memory error. I'm going to post another thread on that
error specifically.

Thanks again.

--Brent

Juan T. Llibre wrote:

Check your maxRequestLength setting.

<httpRuntime executionTimeout="90" maxRequestLength="#ofKBtodownload"

maxRequestLength="8192" would allow 8MB.