473,728 Members | 1,623 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Client-side TCP socket receiving "Address already in use" upon connect

Hi,

The reason is that my application does about 16 connects and data
transfers per second, to the same 16 remote hosts. After approx 200 secs
there are 4000 sockets waiting to be garbage collected by the OS. At
this point is seems that connect loops and starts using the same local
addresses it used 4000 connections ago, resulting in an "Address already
in use" exception.

A possible solution to this would be to keep the connection to each
remote host open, but that would require parsing of the received data
into the data items (which are files by the way) sent by the application.

My question is if there are any other methods of solving this? Maybe a
socket option of some sort...

regards
Sep 2 '06 #1
11 6861
Sybren Stuvel wrote:
Tor Erik enlightened us with:
>The reason is that my application does about 16 connects and data
transfers per second, to the same 16 remote hosts. After approx 200
secs there are 4000 sockets waiting to be garbage collected by the
OS.

Which OS are we talking about?
Windows XP
>
>At this point is seems that connect loops and starts using the same
local addresses it used 4000 connections ago, resulting in an
"Address already in use" exception.

After how many seconds does this happen?
200 seconds approx
>
>My question is if there are any other methods of solving this? Maybe
a socket option of some sort...

If I'm correct (please correct me if I'm not), on Linux you can use
'sysctl -w net.ipv4.tcp_fi n_timeout=X' to set the time between closing
the socket and releasing it to be reused. You can also check the
SO_REUSEADDR argument to the setsockopt function. Read 'man 7 socket'
for more info.
I've read about SO_REUSEADDR. As far as I understand, this is what
SO_REUSEADDR is for:

1. Allow a listening socket to bind itself to its well-known port even
if previously established connections use it as their local port.
Setting this option should be done between calls to socket and bind, and
hence is only usable for listening sockets, not client sockets like mine.

2. Allow multiple servers on the same host with different ip-adresses to
listen to the same port.

I've tried setting this option, but could not see any notable changes...
>
Sybren
Sep 3 '06 #2
I've read about SO_REUSEADDR. As far as I understand, this is what
SO_REUSEADDR is for:
....
I've tried setting this option, but could not see any notable changes...
I was having a similiar problem as you, where as soon as my program
exited, it would get started up again, but could not bind to the same
address.
So i added the follow straight after I create my server object:
server.setsocko pt(socket.SOL_S OCKET, socket.SO_REUSE ADDR, 1)

And it worked. Note that my program was running on Linux, so this might
be a windows issue.

Sep 3 '06 #3
Tor Erik wrote:
The reason is that my application does about 16 connects and data
transfers per second, to the same 16 remote hosts. After approx 200 secs
there are 4000 sockets waiting to be garbage collected by the OS.
what does "netstat" say about these sockets ?

</F>

Sep 3 '06 #4
Fredrik Lundh wrote:
Tor Erik wrote:
>The reason is that my application does about 16 connects and data
transfers per second, to the same 16 remote hosts. After approx 200
secs there are 4000 sockets waiting to be garbage collected by the OS.

what does "netstat" say about these sockets ?

</F>
They are in the TIME_WAIT state... The msn library has an article on how
to solve this:

http://msdn.microsoft.com/library/de...1fc8ba06a4.asp

Summing up one could either:

1. Increase the upper range of ephemeral ports that are dynamically
allocated to client TCP/IP socket connections:

set registry key:
KEY_LOCAL_MACHI NE\SYSTEM\Curre ntControlSet\Se rvices\Tcpip\Pa rameters\MaxUse rPort
to a new DWORD value... (5000 - 65534)
The default in XP is 3976 -http://support.microsoft.com/kb/Q149532

or

2. Reduce the client TCP/IP socket connection timeout value from the
default value of 240 seconds

set registry key:
HKEY_LOCAL_MACH INE\SYSTEM\Curr entControlSet\S ervices\Tcpip\P arameters\TcpTi medWaitDelay
to a new DWORD value (30 - 300)

The TCP RFC (RFC 793) recommends a value of 2*msl(Maximum Segment
Lifetime). The general consensus about the value of msn seems to be 1-2
minutes, depending on the underlying network... (2*2 min = 2*120 sec =
240 sec)
I do not want to alter my registry, so I'm currently testing an idea
where I let the client connect and send its content, appended with my
own "magic" EOF byte-sequence. When the server receives this EOF, it
takes care to close the connection. This should eliminate the problem as
it is the peer closing the connection that enters the TIME_WAIT state...

I will report my experiences...
Sep 3 '06 #5
ke***********@g mail.com wrote:
>>I've read about SO_REUSEADDR. As far as I understand, this is what
SO_REUSEADD R is for:

....
>>I've tried setting this option, but could not see any notable changes...


I was having a similiar problem as you, where as soon as my program
exited, it would get started up again, but could not bind to the same
address.
So i added the follow straight after I create my server object:
server.setsocko pt(socket.SOL_S OCKET, socket.SO_REUSE ADDR, 1)

And it worked. Note that my program was running on Linux, so this might
be a windows issue.
.... and note also that your program was apprently a server, while the OP
was reporting an error on a client program that presumably asks for an
ephemeral port rather than a specifically-numbered one.

Since there are roughly 64,000 ports, the real question seems to be why
his client runs out after about 4,000.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

Sep 3 '06 #6
Tor Erik wrote:
Fredrik Lundh wrote:
>Tor Erik wrote:
>>The reason is that my application does about 16 connects and data
transfers per second, to the same 16 remote hosts. After approx 200
secs there are 4000 sockets waiting to be garbage collected by the OS.

what does "netstat" say about these sockets ?

</F>

They are in the TIME_WAIT state... The msn library has an article on how
to solve this:

http://msdn.microsoft.com/library/de...1fc8ba06a4.asp
Summing up one could either:

1. Increase the upper range of ephemeral ports that are dynamically
allocated to client TCP/IP socket connections:

set registry key:
KEY_LOCAL_MACHI NE\SYSTEM\Curre ntControlSet\Se rvices\Tcpip\Pa rameters\MaxUse rPort

to a new DWORD value... (5000 - 65534)
The default in XP is 3976 -http://support.microsoft.com/kb/Q149532

or

2. Reduce the client TCP/IP socket connection timeout value from the
default value of 240 seconds

set registry key:
HKEY_LOCAL_MACH INE\SYSTEM\Curr entControlSet\S ervices\Tcpip\P arameters\TcpTi medWaitDelay

to a new DWORD value (30 - 300)

The TCP RFC (RFC 793) recommends a value of 2*msl(Maximum Segment
Lifetime). The general consensus about the value of msn seems to be 1-2
minutes, depending on the underlying network... (2*2 min = 2*120 sec =
240 sec)
I do not want to alter my registry, so I'm currently testing an idea
where I let the client connect and send its content, appended with my
own "magic" EOF byte-sequence. When the server receives this EOF, it
takes care to close the connection. This should eliminate the problem as
it is the peer closing the connection that enters the TIME_WAIT state...

I will report my experiences...
Well... my idea does not work as expected. Even though the server
(remote host) calls socket.close(), it is the client that executes
TIME_WAIT. My guess is that the subtrates below socket closes the
connection at the peer calling connect regardless of where socket.close
is called.

Thoughts anyone?
Sep 3 '06 #7
Tor Erik wrote:
Tor Erik wrote:
>>Fredrik Lundh wrote:
>>>Tor Erik wrote:
The reason is that my application does about 16 connects and data
transfers per second, to the same 16 remote hosts. After approx 200
secs there are 4000 sockets waiting to be garbage collected by the OS.

what does "netstat" say about these sockets ?

</F>
They are in the TIME_WAIT state... The msn library has an article on how
to solve this:

http://msdn.microsoft.com/library/de...1fc8ba06a4.asp
Summing up one could either:

1. Increase the upper range of ephemeral ports that are dynamically
allocated to client TCP/IP socket connections:

set registry key:
KEY_LOCAL_MAC HINE\SYSTEM\Cur rentControlSet\ Services\Tcpip\ Parameters\MaxU serPort

to a new DWORD value... (5000 - 65534)
The default in XP is 3976 -http://support.microsoft.com/kb/Q149532

or

2. Reduce the client TCP/IP socket connection timeout value from the
default value of 240 seconds

set registry key:
HKEY_LOCAL_MA CHINE\SYSTEM\Cu rrentControlSet \Services\Tcpip \Parameters\Tcp TimedWaitDelay

to a new DWORD value (30 - 300)

The TCP RFC (RFC 793) recommends a value of 2*msl(Maximum Segment
Lifetime). The general consensus about the value of msn seems to be 1-2
minutes, depending on the underlying network... (2*2 min = 2*120 sec =
240 sec)
I do not want to alter my registry, so I'm currently testing an idea
where I let the client connect and send its content, appended with my
own "magic" EOF byte-sequence. When the server receives this EOF, it
takes care to close the connection. This should eliminate the problem as
it is the peer closing the connection that enters the TIME_WAIT state...

I will report my experiences...


Well... my idea does not work as expected. Even though the server
(remote host) calls socket.close(), it is the client that executes
TIME_WAIT. My guess is that the subtrates below socket closes the
connection at the peer calling connect regardless of where socket.close
is called.

Thoughts anyone?
Yes, it's the transport layer that puts the socket into the TIME_WAIT state.

regards
Steve
--
Steve Holden +44 150 684 7255 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://holdenweb.blogspot.com
Recent Ramblings http://del.icio.us/steve.holden

Sep 3 '06 #8
Steve Holden <st***@holdenwe b.comwrote:
...
>set registry key:
>>KEY_LOCAL_MAC HINE\SYSTEM\Cur rentControlSet\ Services\Tcpip\ Parameters\M
axUserPort
>
to a new DWORD value... (5000 - 65534)
The default in XP is 3976 -http://support.microsoft.com/kb/Q149532
I wonder why (performance under RAM-costrained conditions? but then why
not have this vary depending on available RAM -- complications?)
Yes, it's the transport layer that puts the socket into the TIME_WAIT state.
Yes, there's a good explanation at
<http://www.developerwe b.net/forum/showthread.php? t=2941(though one
should really study Stevens' "TCP-IP Illustrated" for better
understanding in depth). Playing with SO_LINGER and/or the MSL is not
recommended unless you're operating only on a network that you entirely
control (particularly in terms of round-trip times and router behavior).

As debated at
<http://www.sunmanagers.org/pipermail...ry/007068.html
>, you may be able to have your clients go into CLOSE_WAIT (instead of
TIME_WAIT) by playing around with "who closes the socket first", and
CLOSE_WAIT might be more transient than the 2*MSL (240 seconds...)
needed for TIME_WAIT on possibly-unreliable networks. But it's far from
sure that this would gain you much.

Reflecting on the OP's use case, since all connections are forever being
made to the same 16 servers, why not tweak thinks a bit to hold those
connections open for longer periods of time, using a connection for many
send/receive "transactio ns" instead of opening and closing such
connections all of the time? That might well work better...
Alex
Sep 3 '06 #9
2006/9/3, Alex Martelli <al***@mac.com> :
Reflecting on the OP's use case, since all connections are forever being
made to the same 16 servers, why not tweak thinks a bit to hold those
connections open for longer periods of time, using a connection for many
send/receive "transactio ns" instead of opening and closing such
connections all of the time? That might well work better...
Connecting to 16 differente servers per second gives a very poor
performance, right? There's some overhead in creating TCP connections,
even on fast networks and computers. Am I right?

--
Felipe.
Sep 3 '06 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

4
16846
by: Tig | last post by:
Hi all. I have a need to connect to an Oracle 7.3.3.5 database. I have a user who successfully connects to it with her Oracle 7.3 client. I have an Oracle 9.2 client installed on my machine. I had her send me her tnsnames.ora from her PC. The entry I wanted was as follows (some letters replaced with xxx for security reasons):
7
21310
by: Privacy Advocate | last post by:
//crossposted to: comp.lang.javascript, alt.comp.lang.javascript in an effort to get factual answers from JavaScript experts// Simply put; Is it possible to obtain the real (actual) IP address of someone (client) that visits a web site through an anonymous proxy if this person ONLY has JavaScript enabled in their browser? This is NOT a question about PHP, perl, VBScript, Java(.class), or ActiveX. Let us _only_ deal with JavaScript for...
30
7430
by: aka | last post by:
Hi I have a DB2 v8.1 on AIX and DB2 Connect EE on Solaris wich is connected to OS/390 DB2 subsystems via APPC / SNA. I have cataloged the DB2 Connect instance as tcpip node and then the Host DB cataloged on that node...this works from v7.2 Fixpack 11 clients and servers, but with v8.1 server I get SQL1334N. Does anyone has an idea? Thanks
1
1845
by: luciano | last post by:
Hi everyone, I want to create a application and a webservice, application connect to web service to activate, web sevice will create a certificate to authenticate this client, for each transaction between client anh server, server will check cerfiticate of client connect to it. How do i do that? Thanks for your reply.
7
1547
by: rob | last post by:
I want to create an application that can be used both for a server-client model as well as a standalong application. The idea is to write a general server-client application. The standalong then just has both components on the same system. Now the question is if this is a good approach or if this is even doable. For instance do web services have requirements that would prevent them from running on a non-server OS? Any input? Rob
3
2298
by: D. Yates | last post by:
Hi, I'm about to embark on a project that will both send and receive information to/from our client computers in the field. The area that I still need to finalize is the method of communication and
3
45468
by: Laurence | last post by:
Hi there, Does anyone know what's difference among "connect reset", "disconnect", and "terminate"? Thanks in advance,
3
8204
by: RFD | last post by:
I've been slaving at this problem for over a week, and would appreciate some help from you kind folks. Basic Problem: I have made a server program and a client program. When I try to use the client to connect to 127.0.0.1 or my LAN IP, the program connects and transfers information successfully. However, when I get somebody else to host the server, or when I try to connect to a server on my own machine using my WAN IP, I currently get a...
2
9792
by: jeffhan | last post by:
we have os/400 db2 database at the backend. i installed db2 v9 connect server on one windows server which is locating the same network with database server. now how to i configure the connect server and client, so client could access the database through the connect server? like that: db2client <--db2 connect server <--db2 mainframe database
0
8757
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
9418
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9272
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
9196
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
9129
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
8124
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6712
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
6013
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
2
2651
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.