NativeOverlapped, does it work??

[Soren S. Jorgensen]

Hi,

I'm trying to read some messages (native structs) from a kernel mode
mini-filter driver. I'm using my own implementation of IAsyncResult to
pack/unpack the NativeOverlapped structure, and waiting for the result to
arrive.

To read the messages frm the kernek driver i'm using native function:
HRESULT WINAPI FilterGetMessage(
IN HANDLE hPort,
IN OUT PFILTER_MESSAGE_HEADER lpMessageBuffer,
IN DWORD dwMessageBufferSize,
IN LPOVERLAPPED lpOverlapped OPTIONAL);

- in C# it looks like:
[DllImport("FltLib", CallingConvention = CallingConvention.Winapi, CharSet =
CharSet.Auto)]
unsafe private static extern int FilterGetMessage(
SafeFileHandle hPort,
IntPtr lpMessageBuffer,
int dwMessageBufferSize,
NativeOverlapped* lpOverlapped);

The code at first glance seems to be working - no errors on either side (not
in the driver and not in the user mode app) - but the data that's being
transferred are invalid. The data is a simple structure of ULONG's with a
total size of 24 bytes.

I know what is written from kernel mode (i wrote the driver myself), but
when the data is received in user mode it's changed!!

One the that puzzles me is that the C# counter part of the data structure
has to be 16 bytes longer than the C++ structure, which perhaps can be
explained by the four int's reserved at the end of the NativeOverlapped??

I'm able to send data to the driver (synchronously, not using overlapped)
with no problems.

Has anyone gotten NativeOverlapped to work, and if so, please point me to
examples.
I build my code greatly inspired from the "Concurrent Affairs" article in
MSDN magazine june 2007 by Jeffrey Richter

Thanks SSJ

[Nicholas Paldino [.NET/C# MVP]]

Soren,

How are you passing the pointer to the NativeOverlapped structure? You
need to keep that fixed in memory, and if you are using a fixed statement
(you might have it on the stack, in which case, you have to store the
structure somewhere) the memory location of the structure is subject to
change.

My guess is that you aren't passing the structure correctly.

Instead, I would pin the structure (and then get the unsafe pointer to
the structure), or marshal it to unmanaged memory, where you can hold onto
the pointer for the life of the call (and dispose of it appropriately when
done).

--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com
"Soren S. Jorgensen" <[e-mail] in message
news:Oy**************@TK2MSFTNGP05.phx.gbl...

Hi,

I'm trying to read some messages (native structs) from a kernel mode
mini-filter driver. I'm using my own implementation of IAsyncResult to
pack/unpack the NativeOverlapped structure, and waiting for the result to
arrive.

To read the messages frm the kernek driver i'm using native function:
HRESULT WINAPI FilterGetMessage(
IN HANDLE hPort,
IN OUT PFILTER_MESSAGE_HEADER lpMessageBuffer,
IN DWORD dwMessageBufferSize,
IN LPOVERLAPPED lpOverlapped OPTIONAL);

- in C# it looks like:
[DllImport("FltLib", CallingConvention = CallingConvention.Winapi, CharSet
= CharSet.Auto)]
unsafe private static extern int FilterGetMessage(
SafeFileHandle hPort,
IntPtr lpMessageBuffer,
int dwMessageBufferSize,
NativeOverlapped* lpOverlapped);

The code at first glance seems to be working - no errors on either side
(not in the driver and not in the user mode app) - but the data that's
being transferred are invalid. The data is a simple structure of ULONG's
with a total size of 24 bytes.

I know what is written from kernel mode (i wrote the driver myself), but
when the data is received in user mode it's changed!!

One the that puzzles me is that the C# counter part of the data structure
has to be 16 bytes longer than the C++ structure, which perhaps can be
explained by the four int's reserved at the end of the NativeOverlapped??

I'm able to send data to the driver (synchronously, not using overlapped)
with no problems.

Has anyone gotten NativeOverlapped to work, and if so, please point me to
examples.
I build my code greatly inspired from the "Concurrent Affairs" article in
MSDN magazine june 2007 by Jeffrey Richter

Thanks SSJ

[Soren S. Jorgensen]

Hi Nicholas,

Well, I create the buffer object (struct) and (in my AsyncResult
implementation) binds it from being collected by GC using a GCHandle
(pinned) and keeps the handle stored there. Then my AsyncResult is passed
into a Overlapped object and the Overlapped is packed into a
NativeOverlapped pointer and passed to the native function, together with
the address of the pinned object (gotten from the GCHandle). When the
IOCompletionCallback (created when unpacking Overlapped) is called, I unpack
the NativeOverlapped into a Overlapped struct a gets my AsyncResult from it
(expecting GCHandle.Target inside it to hold the result of the operation)

The main parts of the code (simplified) is here:

public class FilterPort : Object, IDisposable
{
private void ProbeForMessage()
{
if(_messageEvent.WaitOne(10, false))
{
_messageEvent.Reset();

FILTER_MESSAGE message = new FILTER_MESSAGE();

FilterPortAsyncResult asyncResult = new
FilterPortAsyncResult(message,null,null);

Overlapped ol = new Overlapped(0,
0,_messageEvent.SafeWaitHandle.DangerousGetHandle( ),asyncResult);

NativeOverlapped* nol = ol.Pack(new
IOCompletionCallback(this.OnProbeForMessageComplet ed), message);

int res = FilterGetMessage(_hPort, // Bound to ThreadPool
asyncResult.BufferAddress, asyncResult.BufferSize,nol);

switch(res)
{
case STATUS_SUCCESS: // Message received instantly, never
happens
Overlapped.Free(nol);
asyncResult.Dispose();
break;
case STATUS_ERROR_IO_PENDING: // As it should be
break;
default:
throw new Win32Exception(res); // Need to do something
here
}
}
}

private unsafe void OnProbeForMessageCompleted(uint errorCode, uint
numBytes, NativeOverlapped* nol)
{
try
{
if(errorCode != 0)
{
throw new Win32Exception((int)errorCode);
}
else
{
Overlapped ol = Overlapped.Unpack(nol);

using(FilterPortAsyncResult asyncResult = ol.AsyncResult as
FilterPortAsyncResult)
{
if(asyncResult != null)
{
_messageResult = asyncResult.BufferObject; // This
should be outcome of operation
}
}
}
}
catch(Exception ex)
{
// Do some error handling here
}
finally
{
Overlapped.Free(nol);
}
}
}

public class FilterPortAsyncResult : Object, IAsyncResult, IDisposable
{
public FilterPortAsyncResult(object buffer, AsyncCallback callback,
object userState)
{
if(buffer == null)
{
throw new ArgumentNullException("buffer");
}

_gchBuffer = GCHandle.Alloc(buffer, GCHandleType.Pinned);
_callback = callback;
_userState = userState;
}

public object BufferObject
{
get { return _gchBuffer.Target; }
}

public IntPtr BufferAddress
{
get { return _gchBuffer.AddrOfPinnedObject(); }
}

public int BufferSize
{
get { return Marshal.SizeOf(_gchBuffer.Target); }
}

// IAsyncResult / IDisposable interface members here
}

"Nicholas Paldino [.NET/C# MVP]" <mv*@spam.guard.caspershouse.comskrev i
en meddelelse news:FA**********************************@microsof t.com...

Soren,

How are you passing the pointer to the NativeOverlapped structure? You
need to keep that fixed in memory, and if you are using a fixed statement
(you might have it on the stack, in which case, you have to store the
structure somewhere) the memory location of the structure is subject to
change.

My guess is that you aren't passing the structure correctly.

Instead, I would pin the structure (and then get the unsafe pointer to
the structure), or marshal it to unmanaged memory, where you can hold onto
the pointer for the life of the call (and dispose of it appropriately when
done).

--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com
"Soren S. Jorgensen" <[e-mail] in message
news:Oy**************@TK2MSFTNGP05.phx.gbl...

>Hi,

I'm trying to read some messages (native structs) from a kernel mode
mini-filter driver. I'm using my own implementation of IAsyncResult to
pack/unpack the NativeOverlapped structure, and waiting for the result to
arrive.

To read the messages frm the kernek driver i'm using native function:
HRESULT WINAPI FilterGetMessage(
IN HANDLE hPort,
IN OUT PFILTER_MESSAGE_HEADER lpMessageBuffer,
IN DWORD dwMessageBufferSize,
IN LPOVERLAPPED lpOverlapped OPTIONAL);

- in C# it looks like:
[DllImport("FltLib", CallingConvention = CallingConvention.Winapi,
CharSet = CharSet.Auto)]
unsafe private static extern int FilterGetMessage(
SafeFileHandle hPort,
IntPtr lpMessageBuffer,
int dwMessageBufferSize,
NativeOverlapped* lpOverlapped);

The code at first glance seems to be working - no errors on either side
(not in the driver and not in the user mode app) - but the data that's
being transferred are invalid. The data is a simple structure of ULONG's
with a total size of 24 bytes.

I know what is written from kernel mode (i wrote the driver myself), but
when the data is received in user mode it's changed!!

One the that puzzles me is that the C# counter part of the data structure
has to be 16 bytes longer than the C++ structure, which perhaps can be
explained by the four int's reserved at the end of the NativeOverlapped??

I'm able to send data to the driver (synchronously, not using overlapped)
with no problems.

Has anyone gotten NativeOverlapped to work, and if so, please point me to
examples.
I build my code greatly inspired from the "Concurrent Affairs" article in
MSDN magazine june 2007 by Jeffrey Richter

Thanks SSJ

[Nicholas Paldino [.NET/C# MVP]]

Soren,

Looking at your code, it seems like you are doing everything correctly.
The only thing that I am unsure about is why you are pinning the piece of
data in your IAsyncResult implementation. It could very well be because of
my advice before, which was targeted at the NativeOverlapped structure.

Can you boil this down to a complete example (perhaps on another IO
routine which uses the overlapped structure)? I ask, because if it is a
problem with the overlapped structure (or the way it is being called) it
should manifest itself on anyone's machine.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com

"Soren S. Jorgensen" <[e-mail] in message
news:OH**************@TK2MSFTNGP05.phx.gbl...

>
Hi Nicholas,

Well, I create the buffer object (struct) and (in my AsyncResult
implementation) binds it from being collected by GC using a GCHandle
(pinned) and keeps the handle stored there. Then my AsyncResult is passed
into a Overlapped object and the Overlapped is packed into a
NativeOverlapped pointer and passed to the native function, together with
the address of the pinned object (gotten from the GCHandle). When the
IOCompletionCallback (created when unpacking Overlapped) is called, I
unpack the NativeOverlapped into a Overlapped struct a gets my AsyncResult
from it (expecting GCHandle.Target inside it to hold the result of the
operation)

The main parts of the code (simplified) is here:

public class FilterPort : Object, IDisposable
{
private void ProbeForMessage()
{
if(_messageEvent.WaitOne(10, false))
{
_messageEvent.Reset();

FILTER_MESSAGE message = new FILTER_MESSAGE();

FilterPortAsyncResult asyncResult = new
FilterPortAsyncResult(message,null,null);

Overlapped ol = new Overlapped(0,
0,_messageEvent.SafeWaitHandle.DangerousGetHandle( ),asyncResult);

NativeOverlapped* nol = ol.Pack(new
IOCompletionCallback(this.OnProbeForMessageComplet ed), message);

int res = FilterGetMessage(_hPort, // Bound to ThreadPool
asyncResult.BufferAddress, asyncResult.BufferSize,nol);

switch(res)
{
case STATUS_SUCCESS: // Message received instantly, never
happens
Overlapped.Free(nol);
asyncResult.Dispose();
break;
case STATUS_ERROR_IO_PENDING: // As it should be
break;
default:
throw new Win32Exception(res); // Need to do something
here
}
}
}

private unsafe void OnProbeForMessageCompleted(uint errorCode, uint
numBytes, NativeOverlapped* nol)
{
try
{
if(errorCode != 0)
{
throw new Win32Exception((int)errorCode);
}
else
{
Overlapped ol = Overlapped.Unpack(nol);

using(FilterPortAsyncResult asyncResult = ol.AsyncResult as
FilterPortAsyncResult)
{
if(asyncResult != null)
{
_messageResult = asyncResult.BufferObject; // This
should be outcome of operation
}
}
}
}
catch(Exception ex)
{
// Do some error handling here
}
finally
{
Overlapped.Free(nol);
}
}
}

public class FilterPortAsyncResult : Object, IAsyncResult, IDisposable
{
public FilterPortAsyncResult(object buffer, AsyncCallback callback,
object userState)
{
if(buffer == null)
{
throw new ArgumentNullException("buffer");
}

_gchBuffer = GCHandle.Alloc(buffer, GCHandleType.Pinned);
_callback = callback;
_userState = userState;
}

public object BufferObject
{
get { return _gchBuffer.Target; }
}

public IntPtr BufferAddress
{
get { return _gchBuffer.AddrOfPinnedObject(); }
}

public int BufferSize
{
get { return Marshal.SizeOf(_gchBuffer.Target); }
}

// IAsyncResult / IDisposable interface members here
}

"Nicholas Paldino [.NET/C# MVP]" <mv*@spam.guard.caspershouse.comskrev i
en meddelelse news:FA**********************************@microsof t.com...

>Soren,

How are you passing the pointer to the NativeOverlapped structure?
You need to keep that fixed in memory, and if you are using a fixed
statement (you might have it on the stack, in which case, you have to
store the structure somewhere) the memory location of the structure is
subject to change.

My guess is that you aren't passing the structure correctly.

Instead, I would pin the structure (and then get the unsafe pointer to
the structure), or marshal it to unmanaged memory, where you can hold
onto the pointer for the life of the call (and dispose of it
appropriately when done).

--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com
"Soren S. Jorgensen" <[e-mail] in message
news:Oy**************@TK2MSFTNGP05.phx.gbl...

>>Hi,

I'm trying to read some messages (native structs) from a kernel mode
mini-filter driver. I'm using my own implementation of IAsyncResult to
pack/unpack the NativeOverlapped structure, and waiting for the result
to arrive.

To read the messages frm the kernek driver i'm using native function:
HRESULT WINAPI FilterGetMessage(
IN HANDLE hPort,
IN OUT PFILTER_MESSAGE_HEADER lpMessageBuffer,
IN DWORD dwMessageBufferSize,
IN LPOVERLAPPED lpOverlapped OPTIONAL);

- in C# it looks like:
[DllImport("FltLib", CallingConvention = CallingConvention.Winapi,
CharSet = CharSet.Auto)]
unsafe private static extern int FilterGetMessage(
SafeFileHandle hPort,
IntPtr lpMessageBuffer,
int dwMessageBufferSize,
NativeOverlapped* lpOverlapped);

The code at first glance seems to be working - no errors on either side
(not in the driver and not in the user mode app) - but the data that's
being transferred are invalid. The data is a simple structure of ULONG's
with a total size of 24 bytes.

I know what is written from kernel mode (i wrote the driver myself), but
when the data is received in user mode it's changed!!

One the that puzzles me is that the C# counter part of the data
structure has to be 16 bytes longer than the C++ structure, which
perhaps can be explained by the four int's reserved at the end of the
NativeOverlapped??

I'm able to send data to the driver (synchronously, not using
overlapped) with no problems.

Has anyone gotten NativeOverlapped to work, and if so, please point me
to examples.
I build my code greatly inspired from the "Concurrent Affairs" article
in MSDN magazine june 2007 by Jeffrey Richter

Thanks SSJ

[Soren S. Jorgensen]

Nicholas,

I thought the most natural place to put the handle of the data object was in
the IAsyncResult as this is the only "link" between the method actually
calling the native function and the method receiving the outcome. I really
havn't changed much since my first post.

The handle is pinned as you otherwise (with normal alloc) wont be able to
use the pointer to the data after completion (I needed this to marshal out
the actual bytes in the data object, debug debug...) - but really isn't
nessecary.

My true concern is that some odd thing might happen under the hood, as:
1. the memory is moved from kernel to user mode

2. the kernel function that sends the data has two void pointers, one for
the data being send and one for a possible reply - inside the data an
identifier will make it possible to reply to the message (using
FilterReplyMessage) instantly before the kernel call returns. This is not
used in my case, but perhaps something goes wrong here.

3. an additional 16 bytes is needed in the data buffer to be large enough
for data to be marshalled in user mode. 24 bytes is sent from kernel mode
and 40 bytes are received in the completion callback. This is a real
puzzle!!

I'll try to make a working sample with some OS distributed driver (perhaps
an IoControl call or something) and post it somewhere. This might even turn
out working :-)

Thanks again, SSJ

"Nicholas Paldino [.NET/C# MVP]" <mv*@spam.guard.caspershouse.comskrev i
en meddelelse news:0E**********************************@microsof t.com...

Soren,

Looking at your code, it seems like you are doing everything correctly.
The only thing that I am unsure about is why you are pinning the piece of
data in your IAsyncResult implementation. It could very well be because
of my advice before, which was targeted at the NativeOverlapped structure.

Can you boil this down to a complete example (perhaps on another IO
routine which uses the overlapped structure)? I ask, because if it is a
problem with the overlapped structure (or the way it is being called) it
should manifest itself on anyone's machine.
--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com

"Soren S. Jorgensen" <[e-mail] in message
news:OH**************@TK2MSFTNGP05.phx.gbl...

>>
Hi Nicholas,

Well, I create the buffer object (struct) and (in my AsyncResult
implementation) binds it from being collected by GC using a GCHandle
(pinned) and keeps the handle stored there. Then my AsyncResult is passed
into a Overlapped object and the Overlapped is packed into a
NativeOverlapped pointer and passed to the native function, together with
the address of the pinned object (gotten from the GCHandle). When the
IOCompletionCallback (created when unpacking Overlapped) is called, I
unpack the NativeOverlapped into a Overlapped struct a gets my
AsyncResult from it (expecting GCHandle.Target inside it to hold the
result of the operation)

The main parts of the code (simplified) is here:

public class FilterPort : Object, IDisposable
{
private void ProbeForMessage()
{
if(_messageEvent.WaitOne(10, false))
{
_messageEvent.Reset();

FILTER_MESSAGE message = new FILTER_MESSAGE();

FilterPortAsyncResult asyncResult = new
FilterPortAsyncResult(message,null,null);

Overlapped ol = new Overlapped(0,
0,_messageEvent.SafeWaitHandle.DangerousGetHandle (),asyncResult);

NativeOverlapped* nol = ol.Pack(new
IOCompletionCallback(this.OnProbeForMessageComple ted), message);

int res = FilterGetMessage(_hPort, // Bound to ThreadPool
asyncResult.BufferAddress, asyncResult.BufferSize,nol);

switch(res)
{
case STATUS_SUCCESS: // Message received instantly, never
happens
Overlapped.Free(nol);
asyncResult.Dispose();
break;
case STATUS_ERROR_IO_PENDING: // As it should be
break;
default:
throw new Win32Exception(res); // Need to do something
here
}
}
}

private unsafe void OnProbeForMessageCompleted(uint errorCode, uint
numBytes, NativeOverlapped* nol)
{
try
{
if(errorCode != 0)
{
throw new Win32Exception((int)errorCode);
}
else
{
Overlapped ol = Overlapped.Unpack(nol);

using(FilterPortAsyncResult asyncResult = ol.AsyncResult
as FilterPortAsyncResult)
{
if(asyncResult != null)
{
_messageResult = asyncResult.BufferObject; // This
should be outcome of operation
}
}
}
}
catch(Exception ex)
{
// Do some error handling here
}
finally
{
Overlapped.Free(nol);
}
}
}

public class FilterPortAsyncResult : Object, IAsyncResult, IDisposable
{
public FilterPortAsyncResult(object buffer, AsyncCallback callback,
object userState)
{
if(buffer == null)
{
throw new ArgumentNullException("buffer");
}

_gchBuffer = GCHandle.Alloc(buffer, GCHandleType.Pinned);
_callback = callback;
_userState = userState;
}

public object BufferObject
{
get { return _gchBuffer.Target; }
}

public IntPtr BufferAddress
{
get { return _gchBuffer.AddrOfPinnedObject(); }
}

public int BufferSize
{
get { return Marshal.SizeOf(_gchBuffer.Target); }
}

// IAsyncResult / IDisposable interface members here
}

"Nicholas Paldino [.NET/C# MVP]" <mv*@spam.guard.caspershouse.comskrev
i en meddelelse
news:FA**********************************@microso ft.com...

>>Soren,

How are you passing the pointer to the NativeOverlapped structure?
You need to keep that fixed in memory, and if you are using a fixed
statement (you might have it on the stack, in which case, you have to
store the structure somewhere) the memory location of the structure is
subject to change.

My guess is that you aren't passing the structure correctly.

Instead, I would pin the structure (and then get the unsafe pointer
to the structure), or marshal it to unmanaged memory, where you can hold
onto the pointer for the life of the call (and dispose of it
appropriately when done).

--
- Nicholas Paldino [.NET/C# MVP]
- mv*@spam.guard.caspershouse.com
"Soren S. Jorgensen" <[e-mail] in message
news:Oy**************@TK2MSFTNGP05.phx.gbl...
Hi,

I'm trying to read some messages (native structs) from a kernel mode
mini-filter driver. I'm using my own implementation of IAsyncResult to
pack/unpack the NativeOverlapped structure, and waiting for the result
to arrive.

To read the messages frm the kernek driver i'm using native function:
HRESULT WINAPI FilterGetMessage(
IN HANDLE hPort,
IN OUT PFILTER_MESSAGE_HEADER lpMessageBuffer,
IN DWORD dwMessageBufferSize,
IN LPOVERLAPPED lpOverlapped OPTIONAL);

- in C# it looks like:
[DllImport("FltLib", CallingConvention = CallingConvention.Winapi,
CharSet = CharSet.Auto)]
unsafe private static extern int FilterGetMessage(
SafeFileHandle hPort,
IntPtr lpMessageBuffer,
int dwMessageBufferSize,
NativeOverlapped* lpOverlapped);

The code at first glance seems to be working - no errors on either side
(not in the driver and not in the user mode app) - but the data that's
being transferred are invalid. The data is a simple structure of
ULONG's with a total size of 24 bytes.

I know what is written from kernel mode (i wrote the driver myself),
but when the data is received in user mode it's changed!!

One the that puzzles me is that the C# counter part of the data
structure has to be 16 bytes longer than the C++ structure, which
perhaps can be explained by the four int's reserved at the end of the
NativeOverlapped??

I'm able to send data to the driver (synchronously, not using
overlapped) with no problems.

Has anyone gotten NativeOverlapped to work, and if so, please point me
to examples.
I build my code greatly inspired from the "Concurrent Affairs" article
in MSDN magazine june 2007 by Jeffrey Richter

Thanks SSJ

[Ben Voigt [C++ MVP]]

"Soren S. Jorgensen" <[e-mail] in message
news:Oy**************@TK2MSFTNGP05.phx.gbl...

Hi,

I'm trying to read some messages (native structs) from a kernel mode
mini-filter driver. I'm using my own implementation of IAsyncResult to
pack/unpack the NativeOverlapped structure, and waiting for the result to
arrive.

To read the messages frm the kernek driver i'm using native function:
HRESULT WINAPI FilterGetMessage(
IN HANDLE hPort,
IN OUT PFILTER_MESSAGE_HEADER lpMessageBuffer,
IN DWORD dwMessageBufferSize,
IN LPOVERLAPPED lpOverlapped OPTIONAL);

- in C# it looks like:
[DllImport("FltLib", CallingConvention = CallingConvention.Winapi, CharSet
= CharSet.Auto)]
unsafe private static extern int FilterGetMessage(
SafeFileHandle hPort,
IntPtr lpMessageBuffer,
int dwMessageBufferSize,
NativeOverlapped* lpOverlapped);

The code at first glance seems to be working - no errors on either side
(not in the driver and not in the user mode app) - but the data that's
being transferred are invalid. The data is a simple structure of ULONG's
with a total size of 24 bytes.

I know what is written from kernel mode (i wrote the driver myself), but
when the data is received in user mode it's changed!!

If you are programming drivers, you will be a lot happier doing this in C++.
Use the C++/CLI compiler to make .NET classes in C++, then you can call them
from your main app in C#.