473,324 Members | 2,473 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,324 software developers and data experts.

Long files: HttpWebRequest & StreamRead

I'm hoping to grab some 70,000 text files over http using the code
below. For the most part, the files are a few KB in size, and are
downloaded quickly. The code meets its Waterloo, however, with a rare
behemoth of 3 MB. I bumped up the Timeout property to 10 minutes (or, at
least I think I did) in hopes it wouldn't fail, but it fails still.

I'd appreciate any tips to fix the problem!

--Brent
==================================

public string get13f (string strURL)
{
try
{
HttpWebRequest oRequest = (HttpWebRequest)WebRequest.Create(strURL);
oRequest.Timeout = 10*60000; // 10 minutes; for long files (10000 =
10 seconds)
oRequest.UserAgent = "Web Client";
HttpWebResponse oResponse = (HttpWebResponse)oRequest.GetResponse();

Stream myStream = oResponse.GetResponseStream();
StreamReader sr = new StreamReader(myStream);
string strResponse = sr.ReadToEnd();
return strResponse;
myStream.Close();
}
catch
{
return "0";
}
}

==================================
Nov 19 '05 #1
4 1713
Check your maxRequestLength setting.

<httpRuntime executionTimeout="90" maxRequestLength="#ofKBtodownload"

maxRequestLength="8192" would allow 8MB.

Juan T. Llibre
ASP.NET MVP
http://asp.net.do/foros/
Foros de ASP.NET en Español
Ven, y hablemos de ASP.NET...
======================

"Brent" <""b b i g l e r \"@ y a h o o . c o m"> wrote in message
news:11*************@corp.supernews.com...
I'm hoping to grab some 70,000 text files over http using the code below. For the most
part, the files are a few KB in size, and are downloaded quickly. The code meets its
Waterloo, however, with a rare behemoth of 3 MB. I bumped up the Timeout property to 10
minutes (or, at least I think I did) in hopes it wouldn't fail, but it fails still.

I'd appreciate any tips to fix the problem!

--Brent
==================================

public string get13f (string strURL)
{
try
{
HttpWebRequest oRequest = (HttpWebRequest)WebRequest.Create(strURL);
oRequest.Timeout = 10*60000; // 10 minutes; for long files (10000 = 10 seconds)
oRequest.UserAgent = "Web Client";
HttpWebResponse oResponse = (HttpWebResponse)oRequest.GetResponse();

Stream myStream = oResponse.GetResponseStream();
StreamReader sr = new StreamReader(myStream);
string strResponse = sr.ReadToEnd();
return strResponse;
myStream.Close();
}
catch
{
return "0";
}
}

==================================

Nov 19 '05 #2
Check your maxRequestLength setting.

<httpRuntime executionTimeout="90" maxRequestLength="#ofKBtodownload"

maxRequestLength="8192" would allow 8MB.

Juan T. Llibre
ASP.NET MVP
http://asp.net.do/foros/
Foros de ASP.NET en Español
Ven, y hablemos de ASP.NET...
======================

"Brent" <""b b i g l e r \"@ y a h o o . c o m"> wrote in message
news:11*************@corp.supernews.com...
I'm hoping to grab some 70,000 text files over http using the code below. For the most
part, the files are a few KB in size, and are downloaded quickly. The code meets its
Waterloo, however, with a rare behemoth of 3 MB. I bumped up the Timeout property to 10
minutes (or, at least I think I did) in hopes it wouldn't fail, but it fails still.

I'd appreciate any tips to fix the problem!

--Brent
==================================

public string get13f (string strURL)
{
try
{
HttpWebRequest oRequest = (HttpWebRequest)WebRequest.Create(strURL);
oRequest.Timeout = 10*60000; // 10 minutes; for long files (10000 = 10 seconds)
oRequest.UserAgent = "Web Client";
HttpWebResponse oResponse = (HttpWebResponse)oRequest.GetResponse();

Stream myStream = oResponse.GetResponseStream();
StreamReader sr = new StreamReader(myStream);
string strResponse = sr.ReadToEnd();
return strResponse;
myStream.Close();
}
catch
{
return "0";
}
}

==================================

Nov 19 '05 #3
Juan:

Thanks for your help. This is a good suggestion, but I've discovered
the problem is a memory error. I'm going to post another thread on that
error specifically.

Thanks again.

--Brent

Juan T. Llibre wrote:
Check your maxRequestLength setting.

<httpRuntime executionTimeout="90" maxRequestLength="#ofKBtodownload"

maxRequestLength="8192" would allow 8MB.

Nov 19 '05 #4
Juan:

Thanks for your help. This is a good suggestion, but I've discovered
the problem is a memory error. I'm going to post another thread on that
error specifically.

Thanks again.

--Brent

Juan T. Llibre wrote:
Check your maxRequestLength setting.

<httpRuntime executionTimeout="90" maxRequestLength="#ofKBtodownload"

maxRequestLength="8192" would allow 8MB.

Nov 19 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Mark Rae | last post by:
Hi, Can anyone please tell me if it's possible to use HttpWebRequest and HttpWebResponse in a class in a Windows application? I've tried referencing System and System.Web but there's still...
3
by: Stig-Arne Basberg | last post by:
I am working on a project where I want to send at file from a Win CE device to a receiving PC using the HTTP protocol. For testing purpose I made a simple test app on my PC (code below). The...
2
by: Paul J. Lay | last post by:
BlankThere are a number of good examples illustrating how an http client can send http mulipart/form-data files to a server using the dotnet.framework.aspnet support. I haven't been able to find a...
0
by: Pmcg | last post by:
I would appreciate any help with the following, havn't found any answers for this in my research to date. I am trying to retreive a web page (a htm file intially) from a vdir on an intranet site...
0
by: Brent | last post by:
I'm hoping to grab some 70,000 text files over http using the code below. For the most part, the files are a few KB in size, and are downloaded quickly. The code meets its Waterloo, however, with a...
4
by: Brent | last post by:
I'd like to think that my code* is pretty simple, but I'm running into memory errors when loading larger documents. The URL you see below in the first line of the Page_Load function is about 3...
16
by: Cheung, Jeffrey Jing-Yen | last post by:
I have a windows form application that generates a request, downloads an image, and waits the user to enter in login info. Unfortunately, this image is dynamic and based on session data. I have...
15
by: Snedker | last post by:
I'm using (HttpWebRequest and HttpWebResponse to check for updates. But how do I determine the size of the file before download? What I have in mind is a status text like "You have downloaded...
4
by: JVNewbie | last post by:
I'm attempting to test an aspx module that will receive XML data from a web service (the receiver module). I want to be able to test this portion before attempting to create the Web Service that will...
1
by: tregewitz | last post by:
I am uploading a zip file using an HttpWebRequest and a PUT operation to Sharepoint (2003) from a Windows Form application. When I upload the same file using the Sharepoint Portal Web UI itself,...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.