473,322 Members | 1,347 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

Configuring More Than Two HttpWebRequests To The Same Host?

I'm writing a C# application for .NET 2.0 in which I intend a number of
background worker threads to be conveying information from our SQL Server
database to a legacy database via HTTP GET requests. Each worker thread
runs the following code in a loop:

-----
HttpWebRequest request = (HttpWebRequest)WebRequest.Create([snip]);

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
string responseText = String.Empty;
string line;
while ((line = responseReader.ReadLine()) != null)
{
if (line != String.Empty)
responseText += line;
}

responseReader.Close();
response.Close();
-----

The total number of requests will end up being approximately 30,000 for each
run of the transfer, so I'm trying to run as many threads as I can so the
total run time will be reduced. But it appears that .NET by default
restricts the number of simultaneous network connections to a single host to
2 (why?!). I did find that if I create an application configuration file
that looks like this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.net>
<connectionManagement>
<add address = "10.134.0.2" maxconnection = "24" />
<add address = "*" maxconnection = "2" />
</connectionManagement>
</system.net>
</configuration>

I should be able to have up to 24 connections to that 10.134.0.2 host (see
http://support.microsoft.com/kb/821268/en-us).

However, this isn't working--I still seem to be constrained to two
connections. Any suggestions as to what I might be doing wrong? The
application name is RightsTransferToADAMS.exe and I have a
RightsTransferToADAMS.exe.config file in the same directory with the content
above. Is there a way to programmatically check that the config file has
been loaded correctly and that I should be able to make 24 connections? Do
I need to do something else?

Thanks very much for any help you might be able to provide.
Jun 13 '06 #1
3 1282
Take a look at the ServicePoint class
http://msdn2.microsoft.com/en-us/sys...vicepoint.aspx

http://msdn2.microsoft.com/en-us/library/7af54za5.aspx includes a bit more
on it.

Cheers,

Greg Young
MVP - C#
http://codebetter.com/blogs/gregyoung

"Alexander J. Oss" <al**@alexoss.net> wrote in message
news:%2****************@TK2MSFTNGP03.phx.gbl...
I'm writing a C# application for .NET 2.0 in which I intend a number of
background worker threads to be conveying information from our SQL Server
database to a legacy database via HTTP GET requests. Each worker thread
runs the following code in a loop:

-----
HttpWebRequest request = (HttpWebRequest)WebRequest.Create([snip]);

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
string responseText = String.Empty;
string line;
while ((line = responseReader.ReadLine()) != null)
{
if (line != String.Empty)
responseText += line;
}

responseReader.Close();
response.Close();
-----

The total number of requests will end up being approximately 30,000 for
each run of the transfer, so I'm trying to run as many threads as I can so
the total run time will be reduced. But it appears that .NET by default
restricts the number of simultaneous network connections to a single host
to 2 (why?!). I did find that if I create an application configuration
file that looks like this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.net>
<connectionManagement>
<add address = "10.134.0.2" maxconnection = "24" />
<add address = "*" maxconnection = "2" />
</connectionManagement>
</system.net>
</configuration>

I should be able to have up to 24 connections to that 10.134.0.2 host (see
http://support.microsoft.com/kb/821268/en-us).

However, this isn't working--I still seem to be constrained to two
connections. Any suggestions as to what I might be doing wrong? The
application name is RightsTransferToADAMS.exe and I have a
RightsTransferToADAMS.exe.config file in the same directory with the
content above. Is there a way to programmatically check that the config
file has been loaded correctly and that I should be able to make 24
connections? Do I need to do something else?

Thanks very much for any help you might be able to provide.

Jun 13 '06 #2
Alexander,
If you use a ThreadPool and the asynchronous pattern with async callback
methods, you should be able to make as many connections as you need (within
reason).
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com


"Alexander J. Oss" wrote:
I'm writing a C# application for .NET 2.0 in which I intend a number of
background worker threads to be conveying information from our SQL Server
database to a legacy database via HTTP GET requests. Each worker thread
runs the following code in a loop:

-----
HttpWebRequest request = (HttpWebRequest)WebRequest.Create([snip]);

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
string responseText = String.Empty;
string line;
while ((line = responseReader.ReadLine()) != null)
{
if (line != String.Empty)
responseText += line;
}

responseReader.Close();
response.Close();
-----

The total number of requests will end up being approximately 30,000 for each
run of the transfer, so I'm trying to run as many threads as I can so the
total run time will be reduced. But it appears that .NET by default
restricts the number of simultaneous network connections to a single host to
2 (why?!). I did find that if I create an application configuration file
that looks like this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.net>
<connectionManagement>
<add address = "10.134.0.2" maxconnection = "24" />
<add address = "*" maxconnection = "2" />
</connectionManagement>
</system.net>
</configuration>

I should be able to have up to 24 connections to that 10.134.0.2 host (see
http://support.microsoft.com/kb/821268/en-us).

However, this isn't working--I still seem to be constrained to two
connections. Any suggestions as to what I might be doing wrong? The
application name is RightsTransferToADAMS.exe and I have a
RightsTransferToADAMS.exe.config file in the same directory with the content
above. Is there a way to programmatically check that the config file has
been loaded correctly and that I should be able to make 24 connections? Do
I need to do something else?

Thanks very much for any help you might be able to provide.

Jun 13 '06 #3
Okay, here's what I ended up doing. I'm sure there's redundancy in there,
but I'm not taking the time to experiment with which things are necessary or
not at the moment. Besides... it turns out that with only a few threads
(5-10) I can max out our old HP 9000 D270's two CPUs (it's a CGI app that
kicks off a Progress database client process with each call... horribly
inefficient).

1. Created the application configuration file I had in the earlier post.

2. Added this line of code before creating any HttpWebRequest objects:

ServicePointManager.DefaultConnectionLimit = 24;

3. Added these lines of code for each HttpWebRequest object created:

request.KeepAlive = false;
request.ServicePoint.ConnectionLimit = 24;
request.ProtocolVersion = HttpVersion.Version10;

Thanks again for everyone's help!

"Vadym Stetsyak" <va*****@ukr.net> wrote in message
news:ej**************@TK2MSFTNGP03.phx.gbl...
Hello, Alexander!

<skip>

AJO> The total number of requests will end up being approximately 30,000
AJO> for each run of the transfer, so I'm trying to run as many threads as
AJO> I can so the total run time will be reduced. But it appears that
.NET
AJO> by default restricts the number of simultaneous network connections
to
AJO> a single host to 2 (why?!). I did find that if I create an
AJO> application configuration file that looks like this:

AFAIR 2 connections is the number recommended connection stated in the
HTTP RFC

The question how to do more connections was discussed earlier in this or
related group
look at (
http://groups.google.com.ua/group/mi...221216c41f0a5b
)

(
http://groups.google.com.ua/group/mi...cbd5801911ae53 )

--
Regards, Vadym Stetsyak
www: http://vadmyst.blogspot.com

Jun 14 '06 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Pat Allan | last post by:
I've spent half a day on this with no success, so I'm hoping someone can point out what I'm doing wrong. I have an ASP.NET Server-side component that I've created, and as part of its Render...
3
by: Alexander J. Oss | last post by:
I'm writing a C# application for .NET 2.0 in which I intend a number of background worker threads to be conveying information from our SQL Server database to a legacy database via HTTP GET...
7
by: djc | last post by:
I see many ways to accomplish setting up proxy settings for a web request in the documentation but many of them are marked as obsolete. 1) What is the current and correct way to tell all my...
2
by: daokfella | last post by:
I'm having trouble using localhost to test the e-mail functions in my application. I keep getting the following error when sending mail: "Unable to read data from the transport connection: An...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
1
by: Shællîpôpï 09 | last post by:
If u are using a keypad phone, how do u turn on JavaScript, to access features like WhatsApp, Facebook, Instagram....
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.