(apparent) Marshalling related problem in x64 but works in x86

[d-42]

Hi,

I'm pretty sure I've just got a Marshalling problem, but I'm
completely stumped. If there is a better newsgroup to post this in,
please point me towards it.

First I'm trying to use SendMessage() with WM_COPYDATA to send a
string data between two instances of a program. As part of the
debugging I've written two methods one where a given instance will
obtain its own hWnd and send itself the message, one where it accepts
obtains the hWnd of the other instance and sends it to the other one.
Both methods work fine when compiled as x86. And the send-to-itself
works on x64, but the send-to-another-instance breaks... the
WM_COPYDATA message is sent, and received, but I'm unable to read back
the string data.

I *think* its a marshalling problem related to changes with x64. But
I'm not sure.

Here are the relevant bits:

// The COPYDATA structure used for the WM_COPYDATA message:

[StructLayout(LayoutKind.Sequential)]
public struct COPYDATASTRUCT
{
[MarshalAs(UnmanagedType.U4)]
public int dwData; // not used
[MarshalAs(UnmanagedType.U4)]
public int cbData; // holds length of string
[MarshalAs(UnmanagedType.LPStr)]
public string lpData; //holds string
}

//DLL Import of SendMessage
[DllImport("user32.dll", CharSet=CharSet.Ansi)]
private static extern IntPtr SendMessage(IntPtr hWnd,
[MarshalAs(UnmanagedType.U4)] int Msg,
IntPtr wParam, ref COPYDATASTRUCT lParam);

//The function that sends the message:

public static IntPtr SendCopyDataMessage(IntPtr toHandle)
{
int messageid = 1234; // random message id

stringdata = "Testing\0"; // send the string "Testing"

// Create/populate the copydata structure.
COPYDATASTRUCT dataStruct = new COPYDATASTRUCT();

dataStruct.dwData = 0; //not used
dataStruct.cbData = stringdata.Length;
dataStruct.lpData = stringdata;

// Make the call
IntPtr result = SendMessage(toHandle, WM_COPYDATA,
messageid, ref dataStruct);

return result;
}

The WndProc that receives the messages is an override of Form.WndProc:

// Process Custom Messages!
protected override void WndProc(ref Message m)
{
if (m.Msg == WM_COPYDATA)
{
// unpack the data
string stringdata = ProcessWM_COPYDATA(m);
// show what it was
if (stringdata != null)
{
MessageBox.Show(stringdata);
}
}
base.WndProc(ref m);
}

The function it calls to unpack the COPYDATA structure is as follows:

public static string
ProcessWM_COPYDATA(System.Windows.Forms.Message m)
{
if (m.WParam.ToInt32() == 1234)
{
COPYDATASTRUCT datastruct =
(COPYDATASTRUCT)Marshal.PtrToStructure(
m.LParam, typeof(COPYDATASTRUCT));
return datastruct.lpData;
}
return null;
}
}

As I said: It works when targeted at x86. In x64 it also works when I
fire the send message at the same instance. (e.g. I have a button on
the form that calls the SendCopyData(...) with its own window handle.

The only time it doesn't work is when I have 2nd instance running and
it calls SendCopyData(...) with the hWnd of the OTHER instance. In
that case the message is received, and the MessageBox is displayed,
but instead of saying testing its either blank or contains garbage
data.

I'd be grateful if someone can point out where I've screwed it up. As
I said, I think its in marshalling/unmarshalling the lpData string,
but I'm at a loss to fix it. I've tried doing it a number of ways,
including setting lpData in COPYDATASTRUCTURE to an IntPtr, and then
manually calling Marshal.StringToHGlobal(stringdata); and then when
unpacking it, calling Marshal.PtrToString(lpData), but I end up with
the same behaviour... breaking in x64.

-best regards,
Dave

[Stephen Martin]

The dwData member of COPYDATASTRUCT is not an int. It is a pointer sized
type so you must use IntPtr in your struct definition.
"d-42" <db********@gmail.comwrote in message
news:e3**********************************@v26g2000 prm.googlegroups.com...

Hi,

I'm pretty sure I've just got a Marshalling problem, but I'm
completely stumped. If there is a better newsgroup to post this in,
please point me towards it.

First I'm trying to use SendMessage() with WM_COPYDATA to send a
string data between two instances of a program. As part of the
debugging I've written two methods one where a given instance will
obtain its own hWnd and send itself the message, one where it accepts
obtains the hWnd of the other instance and sends it to the other one.
Both methods work fine when compiled as x86. And the send-to-itself
works on x64, but the send-to-another-instance breaks... the
WM_COPYDATA message is sent, and received, but I'm unable to read back
the string data.

I *think* its a marshalling problem related to changes with x64. But
I'm not sure.

Here are the relevant bits:

// The COPYDATA structure used for the WM_COPYDATA message:

[StructLayout(LayoutKind.Sequential)]
public struct COPYDATASTRUCT
{
[MarshalAs(UnmanagedType.U4)]
public int dwData; // not used
[MarshalAs(UnmanagedType.U4)]
public int cbData; // holds length of string
[MarshalAs(UnmanagedType.LPStr)]
public string lpData; //holds string
}

//DLL Import of SendMessage
[DllImport("user32.dll", CharSet=CharSet.Ansi)]
private static extern IntPtr SendMessage(IntPtr hWnd,
[MarshalAs(UnmanagedType.U4)] int Msg,
IntPtr wParam, ref COPYDATASTRUCT lParam);

//The function that sends the message:

public static IntPtr SendCopyDataMessage(IntPtr toHandle)
{
int messageid = 1234; // random message id

stringdata = "Testing\0"; // send the string "Testing"

// Create/populate the copydata structure.
COPYDATASTRUCT dataStruct = new COPYDATASTRUCT();

dataStruct.dwData = 0; //not used
dataStruct.cbData = stringdata.Length;
dataStruct.lpData = stringdata;

// Make the call
IntPtr result = SendMessage(toHandle, WM_COPYDATA,
messageid, ref dataStruct);

return result;
}

The WndProc that receives the messages is an override of Form.WndProc:

// Process Custom Messages!
protected override void WndProc(ref Message m)
{
if (m.Msg == WM_COPYDATA)
{
// unpack the data
string stringdata = ProcessWM_COPYDATA(m);
// show what it was
if (stringdata != null)
{
MessageBox.Show(stringdata);
}
}
base.WndProc(ref m);
}

The function it calls to unpack the COPYDATA structure is as follows:

public static string
ProcessWM_COPYDATA(System.Windows.Forms.Message m)
{
if (m.WParam.ToInt32() == 1234)
{
COPYDATASTRUCT datastruct =
(COPYDATASTRUCT)Marshal.PtrToStructure(
m.LParam, typeof(COPYDATASTRUCT));
return datastruct.lpData;
}
return null;
}
}

As I said: It works when targeted at x86. In x64 it also works when I
fire the send message at the same instance. (e.g. I have a button on
the form that calls the SendCopyData(...) with its own window handle.

The only time it doesn't work is when I have 2nd instance running and
it calls SendCopyData(...) with the hWnd of the OTHER instance. In
that case the message is received, and the MessageBox is displayed,
but instead of saying testing its either blank or contains garbage
data.

I'd be grateful if someone can point out where I've screwed it up. As
I said, I think its in marshalling/unmarshalling the lpData string,
but I'm at a loss to fix it. I've tried doing it a number of ways,
including setting lpData in COPYDATASTRUCTURE to an IntPtr, and then
manually calling Marshal.StringToHGlobal(stringdata); and then when
unpacking it, calling Marshal.PtrToString(lpData), but I end up with
the same behaviour... breaking in x64.

-best regards,
Dave

[d-42]

Thank you very much, that resolved the problem.

-regards,
Dave
On May 5, 7:21 am, "Stephen Martin"
<[e-mail]:

The dwData member of COPYDATASTRUCT is not an int. It is a pointer sized
type so you must use IntPtr in your struct definition.