(apparent) Marshalling related problem in x64 but works in x86

[d-42]

Hi,

I'm pretty sure I've just got a Marshalling problem, but I'm
completely stumped. If there is a better newsgroup to post this in,
please point me towards it.

First I'm trying to use SendMessage() with WM_COPYDATA to send a
string data between two instances of a program. As part of the
debugging I've written two methods one where a given instance will
obtain its own hWnd and send itself the message, one where it accepts
obtains the hWnd of the other instance and sends it to the other one.
Both methods work fine when compiled as x86. And the send-to-itself
works on x64, but the send-to-another-instance breaks... the
WM_COPYDATA message is sent, and received, but I'm unable to read back
the string data.

I *think* its a marshalling problem related to changes with x64. But
I'm not sure.

Here are the relevant bits:

// The COPYDATA structure used for the WM_COPYDATA message:

[StructLayout(La youtKind.Sequen tial)]
public struct COPYDATASTRUCT
{
[MarshalAs(Unman agedType.U4)]
public int dwData; // not used
[MarshalAs(Unman agedType.U4)]
public int cbData; // holds length of string
[MarshalAs(Unman agedType.LPStr)]
public string lpData; //holds string
}

//DLL Import of SendMessage
[DllImport("user 32.dll", CharSet=CharSet .Ansi)]
private static extern IntPtr SendMessage(Int Ptr hWnd,
[MarshalAs(Unman agedType.U4)] int Msg,
IntPtr wParam, ref COPYDATASTRUCT lParam);

//The function that sends the message:

public static IntPtr SendCopyDataMes sage(IntPtr toHandle)
{
int messageid = 1234; // random message id

stringdata = "Testing\0" ; // send the string "Testing"

// Create/populate the copydata structure.
COPYDATASTRUCT dataStruct = new COPYDATASTRUCT( );

dataStruct.dwDa ta = 0; //not used
dataStruct.cbDa ta = stringdata.Leng th;
dataStruct.lpDa ta = stringdata;

// Make the call
IntPtr result = SendMessage(toH andle, WM_COPYDATA,
messageid, ref dataStruct);

return result;
}

The WndProc that receives the messages is an override of Form.WndProc:

// Process Custom Messages!
protected override void WndProc(ref Message m)
{
if (m.Msg == WM_COPYDATA)
{
// unpack the data
string stringdata = ProcessWM_COPYD ATA(m);
// show what it was
if (stringdata != null)
{
MessageBox.Show (stringdata);
}
}
base.WndProc(re f m);
}

The function it calls to unpack the COPYDATA structure is as follows:

public static string
ProcessWM_COPYD ATA(System.Wind ows.Forms.Messa ge m)
{
if (m.WParam.ToInt 32() == 1234)
{
COPYDATASTRUCT datastruct =
(COPYDATASTRUCT )Marshal.PtrToS tructure(
m.LParam, typeof(COPYDATA STRUCT));
return datastruct.lpDa ta;
}
return null;
}
}

As I said: It works when targeted at x86. In x64 it also works when I
fire the send message at the same instance. (e.g. I have a button on
the form that calls the SendCopyData(.. .) with its own window handle.

The only time it doesn't work is when I have 2nd instance running and
it calls SendCopyData(.. .) with the hWnd of the OTHER instance. In
that case the message is received, and the MessageBox is displayed,
but instead of saying testing its either blank or contains garbage
data.

I'd be grateful if someone can point out where I've screwed it up. As
I said, I think its in marshalling/unmarshalling the lpData string,
but I'm at a loss to fix it. I've tried doing it a number of ways,
including setting lpData in COPYDATASTRUCTU RE to an IntPtr, and then
manually calling Marshal.StringT oHGlobal(string data); and then when
unpacking it, calling Marshal.PtrToSt ring(lpData), but I end up with
the same behaviour... breaking in x64.

-best regards,
Dave

[Stephen Martin]

The dwData member of COPYDATASTRUCT is not an int. It is a pointer sized
type so you must use IntPtr in your struct definition.
"d-42" <db********@gma il.comwrote in message
news:e3******** *************** ***********@v26 g2000prm.google groups.com...

Hi,

I'm pretty sure I've just got a Marshalling problem, but I'm
completely stumped. If there is a better newsgroup to post this in,
please point me towards it.

First I'm trying to use SendMessage() with WM_COPYDATA to send a
string data between two instances of a program. As part of the
debugging I've written two methods one where a given instance will
obtain its own hWnd and send itself the message, one where it accepts
obtains the hWnd of the other instance and sends it to the other one.
Both methods work fine when compiled as x86. And the send-to-itself
works on x64, but the send-to-another-instance breaks... the
WM_COPYDATA message is sent, and received, but I'm unable to read back
the string data.

I *think* its a marshalling problem related to changes with x64. But
I'm not sure.

Here are the relevant bits:

// The COPYDATA structure used for the WM_COPYDATA message:

[StructLayout(La youtKind.Sequen tial)]
public struct COPYDATASTRUCT
{
[MarshalAs(Unman agedType.U4)]
public int dwData; // not used
[MarshalAs(Unman agedType.U4)]
public int cbData; // holds length of string
[MarshalAs(Unman agedType.LPStr)]
public string lpData; //holds string
}

//DLL Import of SendMessage
[DllImport("user 32.dll", CharSet=CharSet .Ansi)]
private static extern IntPtr SendMessage(Int Ptr hWnd,
[MarshalAs(Unman agedType.U4)] int Msg,
IntPtr wParam, ref COPYDATASTRUCT lParam);

//The function that sends the message:

public static IntPtr SendCopyDataMes sage(IntPtr toHandle)
{
int messageid = 1234; // random message id

stringdata = "Testing\0" ; // send the string "Testing"

// Create/populate the copydata structure.
COPYDATASTRUCT dataStruct = new COPYDATASTRUCT( );

dataStruct.dwDa ta = 0; //not used
dataStruct.cbDa ta = stringdata.Leng th;
dataStruct.lpDa ta = stringdata;

// Make the call
IntPtr result = SendMessage(toH andle, WM_COPYDATA,
messageid, ref dataStruct);

return result;
}

The WndProc that receives the messages is an override of Form.WndProc:

// Process Custom Messages!
protected override void WndProc(ref Message m)
{
if (m.Msg == WM_COPYDATA)
{
// unpack the data
string stringdata = ProcessWM_COPYD ATA(m);
// show what it was
if (stringdata != null)
{
MessageBox.Show (stringdata);
}
}
base.WndProc(re f m);
}

The function it calls to unpack the COPYDATA structure is as follows:

public static string
ProcessWM_COPYD ATA(System.Wind ows.Forms.Messa ge m)
{
if (m.WParam.ToInt 32() == 1234)
{
COPYDATASTRUCT datastruct =
(COPYDATASTRUCT )Marshal.PtrToS tructure(
m.LParam, typeof(COPYDATA STRUCT));
return datastruct.lpDa ta;
}
return null;
}
}

As I said: It works when targeted at x86. In x64 it also works when I
fire the send message at the same instance. (e.g. I have a button on
the form that calls the SendCopyData(.. .) with its own window handle.

The only time it doesn't work is when I have 2nd instance running and
it calls SendCopyData(.. .) with the hWnd of the OTHER instance. In
that case the message is received, and the MessageBox is displayed,
but instead of saying testing its either blank or contains garbage
data.

I'd be grateful if someone can point out where I've screwed it up. As
I said, I think its in marshalling/unmarshalling the lpData string,
but I'm at a loss to fix it. I've tried doing it a number of ways,
including setting lpData in COPYDATASTRUCTU RE to an IntPtr, and then
manually calling Marshal.StringT oHGlobal(string data); and then when
unpacking it, calling Marshal.PtrToSt ring(lpData), but I end up with
the same behaviour... breaking in x64.

-best regards,
Dave

[d-42]

Thank you very much, that resolved the problem.

-regards,
Dave
On May 5, 7:21 am, "Stephen Martin"
<smar...@remove this.emsoft.and this.cawrote:

The dwData member of COPYDATASTRUCT is not an int. It is a pointer sized
type so you must use IntPtr in your struct definition.