Interop problem

[awilliams]

I am trying to implement an interface into a database API.
The API was originally designed for C, but has been used perfectly successfully in many programming languages (C++,Delphi/pascal even Cobol and VB)

The API is like this:
One declares a structure (actually a utility generates the declarations), which in C would be something like this

struct Customer
{
  short l;
  short typenr;
  /* more header fields */
  /* data fields */
  double cust_acc_nr;
  char short_name[10];
  char tel_nr[12];
  char address[4][30];  
  double balance;
  /* more data fields */
}

Then one might do something like

memcpy(customer.cust_ref,"XYZA1234",8);
fetch(&customer,EQUAL);

The appropriate customer record is read from the database, directly into the data structure. The API also allows one to update, insert and delete records. Typically the database will have many hundreds of tables, so
the application may also have calls like

fetch(&employee) where employee is an analogous declaration for another table.

The API "knows" which table the application is working with from the header fields - in particular the
customer.l specifies the size of the customer record buffer, and the customer.typenr is a number unique to the customer table.

Now I am trying to implement this API in C#. Unfortunately C# makes it very difficult to handle fixed size character arrays. I made my program for generating declarations do the following for C#

  [StructLayout(LayoutKind.Explicit)]
  public class _Customer: DP4.ITable
  {
    public static short _CUSTOMER=4117;
    [FieldOffset(0)]
    ushort l = 312;
    [FieldOffset(2)]
    short typenr = _CUSTOMER;
    [FieldOffset(4)]
    ushort generation;
    [FieldOffset(6)]
    ushort flags;
    [FieldOffset(8)]
    int timestamp;
      [FieldOffset(16)]
    internal double _cust_acc_nr;
      [FieldOffset(24)]
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 10)]
    internal char [] _short_name;
      [FieldOffset(34)]
    [MarshalAs(UnmanagedType.ByValArray, SizeConst = 12)]
    internal char [] _tel_nr;
      [FieldOffset(48)]
    double balance;
    [DllImport(Provider.name, EntryPoint = Provider.rec_fetch, CharSet = CharSet.None)]
    static extern int rec_fetch(IntPtr dconn,
                                    [MarshalAs(UnmanagedType.LPStruct), In, Out ] _Customer rec,
                                    int flags, int index, int index_role, int match_keys);
 
    public int Fetch(DP4.Database dconn,TableFetchFlags flags, int index,int index_role,int match_keys)
    {
      return rec_fetch(dconn.Dconn,this,(int)flags,index,index_role,match_keys);
    }
 
  }

The Fetch() method, which is really generic has to be declared for every table so that C# can marshal the data. In fact there are many more similar declarations for other database methods. Provider is a class that just declares a large number of const strings, so that I can generate declarations without worrying about what the real names of the DLL and the entry points are (because, unfortunately these are different depending on whether the application is 32-bit or 64-bit, or designed to run on Windows CE). I'm only showing a low-level API here. The declarations above would be wrapped with something to provide get() and set() methods for a property corresponding to each member, and implement a more "object-oriented" API that would be to the taste of a C# programmer.

When I prototyped this, it seemed like it was going to work OK. Unfortunately when I tried it with a more complex table, although the code compiled OK, it refuses to run. I get this message:

Unhandled Exception: System.TypeLoadException: Could not load type 'Itim.DP4.Schemas.TABLES.INP._Customer' from assembly 'p4, Version=0.0.0.0, Culture=neutral,PublicKeyToken=null' because it contains an object field at offset 34 that is in correctly aligned or overlapped by a non-object field.
at Itim.NETTest2.Program.Main(String[] args)

I'm confused about this error message. I thought that the declaration I'm generating above was managing to declare the analogue of the C structure at the top. But the error message implies that what I'm really getting is something where _short_name is some kind of object being allocated in the heap, so that the name would be elsewhere. But my prototype worked fine, so that cannot be the case. So now I'm thinking that maybe C# has more onerous data alignment requirements than C/C++, and wants short_name to start on a four byte boundary or something. (There's no reason why it should - I'm laying out data in a manner that would satisfy all the alignment restrictions of any processor I've ever heard of)

I'd be really grateful for any insight and advice C# experts can offer. I'm hoping this is the only time in my life I have to write any C#, so I don't want to get into a religious debate about whether my C code should be transferring data like this. I need to implement this API, and the DLL has to be able to return some kind of customer object preferably with as little interference from C# marshalling as possible.

I know C# allows fixed char declarations. But that can only be used in unsafe code, which is not acceptable, and also they can only be used in structs and since I need a class object that would complicate things. So Layout.Explicit layout seems the way to go.

As long as I know how C# wants to lay out the data I can supply the data in that format. The API is more sophisticated that my explanation may have suggested, and can "swizzle" or "swab" data into whatever format a particular language demands. But I have to be able to return data in a single block. I can't create some complicated object containing sub-objects since the database itself is unmanaged C code.

If the worst comes to the worst I'd even be prepared to declare _tel_nr (for example) as 12 individual chars, since I can always provide get() and set() methods that turn it into a TelNr string property. But obviously, I'd prefer not to have to do that if at all possible. The code for generating C# declarations is already about five times more complex than the code for doing the C++ API, which in turn is twice as complex the C code.

Sorry this is a long post. Thanks in advance to anyone who can offer help.

[awilliams]

I have some more information

1) I changed my database tables so that all the fields would naturally begin on 4 byte boundaries. Then the program loads and runs correctly. So my best option may be to tell the database to lay the data out like that. It will waste some space for applications written in C/C++ but this is a fairly minor issue.

2) I then discovered the Pack=1 attribute, so I tried that on my original database. This makes no difference unless I change to LayoutKind Sequential and then the application also works! So it seems that the compiler IS enforcing an unnecessary alignment restriction.

I'm reluctant to use LayoutKind sequential because its not clear to me that I'm always going to guess right about exactly how data is arranged in memory for C# (even though I have to do exactly that for every other programming language). LayoutKind explicit is really what I should be using since it is essential the C code and C# code should agree about the contents of the structure.

It seems to me that C# should be fixed so that Pack=1 works OK with explicit classes/structures. But even then I'd probably avoid it since apparently Pack=1 is not supported on the Compact Framework one gets on Windows CE.

[tlhintoq]

TIP: When you are writing your question, there is a button on the tool bar that wraps the [code] tags around your copy/pasted code. It helps a bunch. Its the button with a '#' on it. More on tags. They're cool. Check'em out.

[awilliams]

Thanks for the tip. I wrote the question in notepad and didn't realise my indentation would disappear. Next time I post I'll be sure to use the "go Advanced" button and make a bit more effort to make sure the question will look OK. I notice you can edit my posts, but I don't seem to have an "edit" button. That would be handy for correcting typos and other mistakes I notice too late.
Is that just for moderators or will I be able to do that kind of thing when I'm not a newbie any more or am I just being stupid and not seeing some obvious way of doing that?

[tlhintoq]

Is that just for moderators or will I be able to do that kind of thing when I'm not a newbie any more or am I just being stupid and not seeing some obvious way of doing that?

I'm embarressed to admit that I don't know.

Hello... Does one of the more experienced moderators know why this user can't edit his own posts?