"jack" <ja*******@gmail.com> wrote in message
news:11**********************@f14g2000cwb.googlegr oups.com...
At this link I have two c# projects, one is a client, the other is a
server. Just point the ip address of the client at the server
http://www.slip-angle.com/hosted/bug/
The server does little more than the examples in the c# documentation
for how to set up an asynchronous server. It just accepts connections,
reads data into a buffer continously, doing nothing with it.
The client connects 200 sockets, sends some data with each continously,
and occasionally disconnects a socket and reconnects a new one.
Things of note:
1. in this scenario memory increases forever. Not just the private
bytes but the gen2 heap increases forever, even after a forced gen2
garbage collection
2. memory profilers indicate no left behind object references
3.the memory profiler indicates "holes" in the gen2 heap, which can be
caused by "pinned objects". I think perhaps that the data in the
receive buffers is being held onto indefinitely, despite all sockets
being closed on the server.
4. If you change the client to not send any data, the memory will not
increase. If you change the client to not disconnect and reconnect any
sockets, memory will not increase.
If someone can verify if this is a .net bug, or point out my idiocy I
would greatly appreciate it.
No, this is not a bug in the socket library, this relates to the way you
handle your receive buffers and the way your testclient is implemented.
Let me try to explain what happens(I hope I succeed).
You noticed two important things:
1. The Gen2 heap keeps growing, while the number of objects don't.
2. There are holes in the Gen2 heap.
Now if you look at the #Pinned objects perf. counter, you will see there are
a considerable number of them.
These objects are long lived so they end in the Gen2 heap, worse, they end
near the top of the heap, as such preventing a compactation of the heap.
Note that collection is not prevented by this, which explains the ( nearly)
constant # of objects.
Now, why are objects aging into gen2, which objects are these and which are
the pinned objects?
These objects are (1)the MsgState instances, they get instantiated when the
Client connects and they live until the client closes the outgoing socket.
The MsgState contains a reference (2) dataBuffer to a byte[] used as socket
buffer, you know by now that these are pinned for the duration of the
BeginReceive/EndReceive and ProcessMessage sequence. But the problem (lets
call it 'issue1') is that their lifetime is tied to the lifetime of the
MsgState object.
Great, but why are they aging into gen2? This is related to the large number
of objects (sockets) and the way your client is implemented.
Your client creates 200 sockets, sends a (too) small buffer to each of them
in sequence, when done it closes and re-opens one single socket in the
sequence they were created. At that time the Socket (and related objects)
it's MsgState and dataBuffer become eligible for collection. Lets say the GC
kicks in, he won't have a problem to collect the garbage, but heck, he can't
compact the heap (Gen0 to start with) because there are pinned objects at
the top of it preventing reallocation, the result is a (small) hole. After
each GC run, the hole becomes bigger as the same pinned object (your lastly
created socket buffer) prevents compactation.
At some point in time, the Gen0 pointer gets adjusted and becomes the Gen1
pointer and after another number of GC runs the Gen1 pointer becomes the
Gen2 pointer, but at/near the top of it you will always find a (number of)
pinned object.
Ok, how to solve this?
1. In ProcessMessage, you should move the contents of the (pinned) receive
buffer to another buffer and release the buffer (re-create a new instance).
This will shorten the period the buffer is pinned, so it won't end as an
aged object.
2. You should change your test method (or at least your message sending
algorithm), I understand that you are trying to simulate a large number of
clients, but this is not the right way of doing. You need separate client
machines, and each one must run a number of processes and each process must
consist of a number of threads each accessing a single socket.
Another remark is that you are touching the UI from thread pool threads,
this is a definitive no no in windows, you have to marshal the call sing
Control.Invoke.
Willy.
