473,387 Members | 1,528 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,387 software developers and data experts.

Configuring More Than Two HttpWebRequests To The Same Host?

I'm writing a C# application for .NET 2.0 in which I intend a number of
background worker threads to be conveying information from our SQL Server
database to a legacy database via HTTP GET requests. Each worker thread
runs the following code in a loop:

-----
HttpWebRequest request = (HttpWebRequest)WebRequest.Create([snip]);

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
string responseText = String.Empty;
string line;
while ((line = responseReader.ReadLine()) != null)
{
if (line != String.Empty)
responseText += line;
}

responseReader.Close();
response.Close();
-----

The total number of requests will end up being approximately 30,000 for each
run of the transfer, so I'm trying to run as many threads as I can so the
total run time will be reduced. But it appears that .NET by default
restricts the number of simultaneous network connections to a single host to
2 (why?!). I did find that if I create an application configuration file
that looks like this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.net>
<connectionManagement>
<add address = "10.134.0.2" maxconnection = "24" />
<add address = "*" maxconnection = "2" />
</connectionManagement>
</system.net>
</configuration>

I should be able to have up to 24 connections to that 10.134.0.2 host (see
http://support.microsoft.com/kb/821268/en-us).

However, this isn't working--I still seem to be constrained to two
connections. Any suggestions as to what I might be doing wrong? The
application name is RightsTransferToADAMS.exe and I have a
RightsTransferToADAMS.exe.config file in the same directory with the content
above. Is there a way to programmatically check that the config file has
been loaded correctly and that I should be able to make 24 connections? Do
I need to do something else?

Thanks very much for any help you might be able to provide.
Jun 13 '06 #1
3 3597
Take a look at the ServicePoint class
http://msdn2.microsoft.com/en-us/sys...vicepoint.aspx

http://msdn2.microsoft.com/en-us/library/7af54za5.aspx includes a bit more
on it.

Cheers,

Greg Young
MVP - C#
http://codebetter.com/blogs/gregyoung

"Alexander J. Oss" <al**@alexoss.net> wrote in message
news:%2****************@TK2MSFTNGP03.phx.gbl...
I'm writing a C# application for .NET 2.0 in which I intend a number of
background worker threads to be conveying information from our SQL Server
database to a legacy database via HTTP GET requests. Each worker thread
runs the following code in a loop:

-----
HttpWebRequest request = (HttpWebRequest)WebRequest.Create([snip]);

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
string responseText = String.Empty;
string line;
while ((line = responseReader.ReadLine()) != null)
{
if (line != String.Empty)
responseText += line;
}

responseReader.Close();
response.Close();
-----

The total number of requests will end up being approximately 30,000 for
each run of the transfer, so I'm trying to run as many threads as I can so
the total run time will be reduced. But it appears that .NET by default
restricts the number of simultaneous network connections to a single host
to 2 (why?!). I did find that if I create an application configuration
file that looks like this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.net>
<connectionManagement>
<add address = "10.134.0.2" maxconnection = "24" />
<add address = "*" maxconnection = "2" />
</connectionManagement>
</system.net>
</configuration>

I should be able to have up to 24 connections to that 10.134.0.2 host (see
http://support.microsoft.com/kb/821268/en-us).

However, this isn't working--I still seem to be constrained to two
connections. Any suggestions as to what I might be doing wrong? The
application name is RightsTransferToADAMS.exe and I have a
RightsTransferToADAMS.exe.config file in the same directory with the
content above. Is there a way to programmatically check that the config
file has been loaded correctly and that I should be able to make 24
connections? Do I need to do something else?

Thanks very much for any help you might be able to provide.

Jun 13 '06 #2
Alexander,
If you use a ThreadPool and the asynchronous pattern with async callback
methods, you should be able to make as many connections as you need (within
reason).
Peter

--
Co-founder, Eggheadcafe.com developer portal:
http://www.eggheadcafe.com
UnBlog:
http://petesbloggerama.blogspot.com


"Alexander J. Oss" wrote:
I'm writing a C# application for .NET 2.0 in which I intend a number of
background worker threads to be conveying information from our SQL Server
database to a legacy database via HTTP GET requests. Each worker thread
runs the following code in a loop:

-----
HttpWebRequest request = (HttpWebRequest)WebRequest.Create([snip]);

HttpWebResponse response = (HttpWebResponse)request.GetResponse();

Stream responseStream = response.GetResponseStream();
StreamReader responseReader = new StreamReader(responseStream);
string responseText = String.Empty;
string line;
while ((line = responseReader.ReadLine()) != null)
{
if (line != String.Empty)
responseText += line;
}

responseReader.Close();
response.Close();
-----

The total number of requests will end up being approximately 30,000 for each
run of the transfer, so I'm trying to run as many threads as I can so the
total run time will be reduced. But it appears that .NET by default
restricts the number of simultaneous network connections to a single host to
2 (why?!). I did find that if I create an application configuration file
that looks like this:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
<system.net>
<connectionManagement>
<add address = "10.134.0.2" maxconnection = "24" />
<add address = "*" maxconnection = "2" />
</connectionManagement>
</system.net>
</configuration>

I should be able to have up to 24 connections to that 10.134.0.2 host (see
http://support.microsoft.com/kb/821268/en-us).

However, this isn't working--I still seem to be constrained to two
connections. Any suggestions as to what I might be doing wrong? The
application name is RightsTransferToADAMS.exe and I have a
RightsTransferToADAMS.exe.config file in the same directory with the content
above. Is there a way to programmatically check that the config file has
been loaded correctly and that I should be able to make 24 connections? Do
I need to do something else?

Thanks very much for any help you might be able to provide.

Jun 13 '06 #3
Okay, here's what I ended up doing. I'm sure there's redundancy in there,
but I'm not taking the time to experiment with which things are necessary or
not at the moment. Besides... it turns out that with only a few threads
(5-10) I can max out our old HP 9000 D270's two CPUs (it's a CGI app that
kicks off a Progress database client process with each call... horribly
inefficient).

1. Created the application configuration file I had in the earlier post.

2. Added this line of code before creating any HttpWebRequest objects:

ServicePointManager.DefaultConnectionLimit = 24;

3. Added these lines of code for each HttpWebRequest object created:

request.KeepAlive = false;
request.ServicePoint.ConnectionLimit = 24;
request.ProtocolVersion = HttpVersion.Version10;

Thanks again for everyone's help!

"Vadym Stetsyak" <va*****@ukr.net> wrote in message
news:ej**************@TK2MSFTNGP03.phx.gbl...
Hello, Alexander!

<skip>

AJO> The total number of requests will end up being approximately 30,000
AJO> for each run of the transfer, so I'm trying to run as many threads as
AJO> I can so the total run time will be reduced. But it appears that
.NET
AJO> by default restricts the number of simultaneous network connections
to
AJO> a single host to 2 (why?!). I did find that if I create an
AJO> application configuration file that looks like this:

AFAIR 2 connections is the number recommended connection stated in the
HTTP RFC

The question how to do more connections was discussed earlier in this or
related group
look at (
http://groups.google.com.ua/group/mi...221216c41f0a5b
)

(
http://groups.google.com.ua/group/mi...cbd5801911ae53 )

--
Regards, Vadym Stetsyak
www: http://vadmyst.blogspot.com

Jun 14 '06 #4

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

1
by: Toni | last post by:
I've tried to reinstall the zend environment and debugger on my new computer. I almost succeeded but when I try to debug or profile a webpage by clicking on the buttons on the toolbars I get the...
5
by: Todd | last post by:
Could someone help me find a tutorial (or perhaps instuct me) in the proper way to configure IIS 5.1 so that I can design and test (using localhost) multiple web sites. My problem arises when I...
10
by: Ed Stevens | last post by:
A tale of woe, and a question . . . Last week my boss said to me "we've installed DB2 Connect on this Solaris box. Make it work." Now, I've barely seen DB2 Connect on Windows, having fumbled...
0
by: Pat Allan | last post by:
I've spent half a day on this with no success, so I'm hoping someone can point out what I'm doing wrong. I have an ASP.NET Server-side component that I've created, and as part of its Render...
6
by: Mike | last post by:
Hi, I've been using httpwebrequest / xmlserverhttp to log into a website via POST. I've received all the response headers back, some of which contain the "set-cookie" headers. When...
3
by: Alexander J. Oss | last post by:
I'm writing a C# application for .NET 2.0 in which I intend a number of background worker threads to be conveying information from our SQL Server database to a legacy database via HTTP GET...
7
by: djc | last post by:
I see many ways to accomplish setting up proxy settings for a web request in the documentation but many of them are marked as obsolete. 1) What is the current and correct way to tell all my...
5
by: NonNB | last post by:
Apols for the post here, but there is no public.inetserver.cassini Is there a way to configure the cassini / ASP NET Development web server: 1) So that the port can be pegged (seems to be...
2
by: daokfella | last post by:
I'm having trouble using localhost to test the e-mail functions in my application. I keep getting the following error when sending mail: "Unable to read data from the transport connection: An...
1
by: rada.lambretha | last post by:
Configuring Linux as a Firewall * Making installation choices * Introducing iptables * Using iptables commands * Simplifying things with firewall GUIs * Adding proxy functionality As...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.