SSLStream broken due to heap fragmentation

[Chris Mullins]

We've been using the SSLStream class found in System.Net.Secu rity to build a
giant Sockets server that provides TLS encryption at the channel leve.
Before .Net 2.0, we used an open-source encryption channel from Mentalis,
and have even looked at the Mono implementation for doing this.

The problem comes from the SSLStream not doing any buffer management. None.
Zero. In the "no buffer management" case, each SSLStream allocates bufferes
whenevers it needs them, copies data into those buffers, and then calls
Socket.BeingRea d / Socket.BeginWri te. This causes pinning. Massive,
long-lived, horrible pins. This leads to horrendus non-recoverable
fragmentation.

The killer here is that both of these socket methods immediatly pin the
buffers, and pass the data off to unmanaged code. The BeginRead method may
not return for 10 seconds, 10 minutes, or 10 hours, leaving a hole in the
Managed Heap the entire time. BeginWrite causes the same problem, although
for a smaller length of time.

To get around this in the past, we've created a block of buffers ahead of
time, and cycled through them. This keeps all the pinned memory together in
a single spot. This technique is described in detail here:
http://blogs.msdn.com/yunjin/archive.../27/63642.aspx
http://www.coversant.net/dotnetnuke/...d=88&EntryID=9

In our last implementation (.Net 1.1), we put alot of work into managing
these buffer pools and really trying hard to eliminate heap fragmentation.
This worked great.
With the .Net 2.0 SSLStream, the BeginRead method ends up allocating
bufferes here:
BeginRead->ProcessRead->EnsureInternal BufferSize
That method allocates a buffer by:
this._InternalB uffer = new byte[addSize + curOffset];

Notice there's no fregging pool here? No attempt to eliminate fragmentation.
No attempt to be efficient.

This buffer gets passed to the NetworkStream, and in turn to the Socket,
which promptly pins it.

The BeginWrite case is the same thing:
BeginWrite->ProcessWrite->StartWriting->EncryptBuffe rs->EncryptData->Encrypt
This method allocates it's buffer:
buffer1 = new byte[(size + this.m_HeaderSi ze) + this.m_TrailerS ize];

Again, no fregging attempt to use a buffer pool. No attempt to eliminate
heap fragmentation.

Because the SSLStream is retarded, it only works on NetworkStreams - and
Network Streams only work on Sockets. This means there's nowhere in the
whole chain I can insert code prior to the Pin that would use a pooled
buffer. Ugh.

Now, .Net 2.0 is improved at managing pinning in the heap:
http://blogs.msdn.com/maoni/archive/...lr-2-0-gc.aspx

.... but I have it on VERY good authority that their technique isn't nearly
as effective as the BufferPool method. In fact, it's not even close.

This has runied my whole day. Ugh.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise

[Chris Mullins]

We did figure out a way around this, in case anyone's interested.

By deriving a class from NetworkStream and overring the relevant methods, we
can swap out the buffers used by SSLStream with our own buffers. Because our
buffers come from a bufferpool that's allocated in a way to minimize heap
fragmentation, the heap stays unfragmented, and scalability is largely
unaffected.

We certainly end up doing number of buffer copies, but that's a much cheaper
price to pay than heap fragmentation.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise
http://www.coversant.net/blogs/cmullins

"Chris Mullins" <cm******@yahoo .comwrote:

We've been using the SSLStream class found in System.Net.Secu rity to build
a giant Sockets server that provides TLS encryption at the channel leve.
Before .Net 2.0, we used an open-source encryption channel from Mentalis,
and have even looked at the Mono implementation for doing this.

The problem comes from the SSLStream not doing any buffer management.
None. Zero. In the "no buffer management" case, each SSLStream allocates
bufferes whenevers it needs them, copies data into those buffers, and then
calls Socket.BeingRea d / Socket.BeginWri te. This causes pinning. Massive,
long-lived, horrible pins. This leads to horrendus non-recoverable
fragmentation.

The killer here is that both of these socket methods immediatly pin the
buffers, and pass the data off to unmanaged code. The BeginRead method may
not return for 10 seconds, 10 minutes, or 10 hours, leaving a hole in the
Managed Heap the entire time. BeginWrite causes the same problem, although
for a smaller length of time.

To get around this in the past, we've created a block of buffers ahead of
time, and cycled through them. This keeps all the pinned memory together
in a single spot. This technique is described in detail here:
http://blogs.msdn.com/yunjin/archive.../27/63642.aspx
http://www.coversant.net/dotnetnuke/...d=88&EntryID=9

In our last implementation (.Net 1.1), we put alot of work into managing
these buffer pools and really trying hard to eliminate heap fragmentation.
This worked great.
With the .Net 2.0 SSLStream, the BeginRead method ends up allocating
bufferes here:
BeginRead->ProcessRead->EnsureInternal BufferSize
That method allocates a buffer by:
this._InternalB uffer = new byte[addSize + curOffset];

Notice there's no fregging pool here? No attempt to eliminate
fragmentation. No attempt to be efficient.

This buffer gets passed to the NetworkStream, and in turn to the Socket,
which promptly pins it.

The BeginWrite case is the same thing:
BeginWrite->ProcessWrite->StartWriting->EncryptBuffe rs->EncryptData->Encrypt
This method allocates it's buffer:
buffer1 = new byte[(size + this.m_HeaderSi ze) + this.m_TrailerS ize];

Again, no fregging attempt to use a buffer pool. No attempt to eliminate
heap fragmentation.

Because the SSLStream is retarded, it only works on NetworkStreams - and
Network Streams only work on Sockets. This means there's nowhere in the
whole chain I can insert code prior to the Pin that would use a pooled
buffer. Ugh.

Now, .Net 2.0 is improved at managing pinning in the heap:
http://blogs.msdn.com/maoni/archive/...lr-2-0-gc.aspx

... but I have it on VERY good authority that their technique isn't nearly
as effective as the BufferPool method. In fact, it's not even close.

This has runied my whole day. Ugh.

--
Chris Mullins, MCSD.NET, MCPD:Enterprise