473,324 Members | 2,179 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,324 software developers and data experts.

Mapping a struct to a memory block....

I have a multi-field struct and a memory block (created using malloc) that
contains structured data. If I map the struct to the memory block, am I
guaranteed that the struct fields will be filled properly or do I have to
worry about the compiler padding the struct?

Example:

typedef struct
{
int16 field1; // 2 bytes
int32 field2; // 4 bytes
} st;

// memblock contains exactly 6 bytes
st* var = (st*) memblock;

Apr 20 '07 #1
14 8748
barcaroller wrote:
I have a multi-field struct and a memory block (created using malloc) that
contains structured data. If I map the struct to the memory block, am I
guaranteed that the struct fields will be filled properly or do I have to
worry about the compiler padding the struct?

Example:

typedef struct
{
int16 field1; // 2 bytes
int32 field2; // 4 bytes
} st;

// memblock contains exactly 6 bytes
st* var = (st*) memblock;
On many systems, sizeof(struct st) will be 8 due to alignment padding.
Alignment and padding aren't standardised, so doing what you propose is
not a good idea.

--
Ian Collins.
Apr 20 '07 #2
"barcaroller" <ba*********@music.netwrites:
I have a multi-field struct and a memory block (created using malloc) that
contains structured data. If I map the struct to the memory block, am I
guaranteed that the struct fields will be filled properly or do I have to
worry about the compiler padding the struct?
Compilers are allowed to insert padding between structure members
or after the end of a structure. The C language doesn't provide
a way to prevent or control this (although your implementation
might).
typedef struct
{
int16 field1; // 2 bytes
int32 field2; // 4 bytes
} st;
The standard type names for these types would be int16_t and
int32_t, declared in <stdint.hin C99 implementations that
support them.
--
int main(void){char p[]="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuv wxyz.\
\n",*q="kl BIcNBFr.NKEzjwCIxNJC";int i=sizeof p/2;char *strchr();int putchar(\
);while(*q){i+=strchr(p,*q++)-p;if(i>=(int)sizeof p)i-=sizeof p-1;putchar(p[i]\
);}return 0;}
Apr 20 '07 #3
"barcaroller" <ba*********@music.netwrites:
I have a multi-field struct and a memory block (created using malloc) that
contains structured data. If I map the struct to the memory block, am I
guaranteed that the struct fields will be filled properly or do I have to
worry about the compiler padding the struct?
[...]

You have to worry about the compiler padding the struct.

See question 2.12 in the comp.lang.c FAQ, <http://www.c-faq.com/>.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Apr 20 '07 #4

"Ian Collins" <ia******@hotmail.comwrote in message
news:58**************@mid.individual.net...
On many systems, sizeof(struct st) will be 8 due to alignment padding.
Alignment and padding aren't standardised, so doing what you propose is
not a good idea.
Does this also apply to reading/writing a file? I have seen code where
structs are being written to and read from files. If the compiler is
allowed to use padding then neither the writes nor the reads are safe.
Apr 21 '07 #5
barcaroller wrote:
"Ian Collins" <ia******@hotmail.comwrote in message
news:58**************@mid.individual.net...
>>On many systems, sizeof(struct st) will be 8 due to alignment padding.
Alignment and padding aren't standardised, so doing what you propose is
not a good idea.


Does this also apply to reading/writing a file? I have seen code where
structs are being written to and read from files. If the compiler is
allowed to use padding then neither the writes nor the reads are safe.

They are assuming the same compiler on the same system is being used to
generate the code that reads and writes the struct.

--
Ian Collins.
Apr 21 '07 #6

"Ian Collins" <ia******@hotmail.comwrote in message
news:58**************@mid.individual.net...
They are assuming the same compiler on the same system is being used to
generate the code that reads and writes the struct.
If I understood you correctly, you are saying that if the compiler uses
padding, it will do so in a consistent manner. Hence, if I use a struct to
write data into a memory buffer, then I am safe to read the data as long as
I use exactly the same struct (even if I read from a memory buffer that was
copied from the original memory buffer using memcpy).
Apr 21 '07 #7
barcaroller wrote:
"Ian Collins" <ia******@hotmail.comwrote in message
news:58**************@mid.individual.net...
>>They are assuming the same compiler on the same system is being used to
generate the code that reads and writes the struct.


If I understood you correctly, you are saying that if the compiler uses
padding, it will do so in a consistent manner. Hence, if I use a struct to
write data into a memory buffer, then I am safe to read the data as long as
I use exactly the same struct (even if I read from a memory buffer that was
copied from the original memory buffer using memcpy).
Probably. I've never seen a case where this isn't true, but you never know!

--
Ian Collins.
Apr 21 '07 #8
"barcaroller" <ba*********@music.netwrites:
"Ian Collins" <ia******@hotmail.comwrote in message
news:58**************@mid.individual.net...
>They are assuming the same compiler on the same system is being used to
generate the code that reads and writes the struct.

If I understood you correctly, you are saying that if the compiler uses
padding, it will do so in a consistent manner.
Strictly speaking, I think that's only required within a single
execution of a program, but practically speaking you can pretty much
depend on binary files being compatible as long as the same compiler
was used to compile the code that writes them and the code that reads
them. (But some compilers may provide command-line options that
affect layout.)

It's likely, but by no means guaranteed, that layouts will be
consistent across different compilers on the same system. Compiler
vendors are motivated to make things easy for their users.

Across different platforms, though, all bets are off unless you use
text-based formats.
Hence, if I use a struct to
write data into a memory buffer, then I am safe to read the data as long as
I use exactly the same struct (even if I read from a memory buffer that was
copied from the original memory buffer using memcpy).
Yes.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Apr 21 '07 #9
On Fri, 20 Apr 2007 20:10:53 -0400, in comp.lang.c , "barcaroller"
<ba*********@music.netwrote:
>
"Ian Collins" <ia******@hotmail.comwrote in message
news:58**************@mid.individual.net...
>On many systems, sizeof(struct st) will be 8 due to alignment padding.
Alignment and padding aren't standardised, so doing what you propose is
not a good idea.

Does this also apply to reading/writing a file? I have seen code where
structs are being written to and read from files. If the compiler is
allowed to use padding then neither the writes nor the reads are safe.
Its safe provided both reads and writes are being performed by the
same version of the same software on the same hardware, compiled by
the same compiler...

People make good money writing tools to convert binary data of this
sort between versions, platforms and applications.
--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan
Apr 21 '07 #10
On Fri, 20 Apr 2007 20:23:56 -0400, in comp.lang.c , "barcaroller"
<ba*********@music.netwrote:
>
"Ian Collins" <ia******@hotmail.comwrote in message
news:58**************@mid.individual.net...
>They are assuming the same compiler on the same system is being used to
generate the code that reads and writes the struct.

If I understood you correctly, you are saying that if the compiler uses
padding, it will do so in a consistent manner.
Provided you don't change any of the compile or link flags.
Or upgrade to a new version or patchlevel of the compiler.

--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan
Apr 21 '07 #11
On Apr 20, 10:54 pm, Ben Pfaff <b...@cs.stanford.eduwrote:
"barcaroller" <barcarol...@music.netwrites:
Compilers are allowed to insert padding between structure members
or after the end of a structure. The C language doesn't provide
a way to prevent or control this (although your implementation
might).
That's true, though you can work out how much padding there is using
the famous offsetof macro I was asking about a while ago. This seems
like a pretty good use of offsetof to me - lets you unpack a struct to
write a portable binary representation to file.
typedef struct
{
int16 field1; // 2 bytes
int32 field2; // 4 bytes
} st;

The standard type names for these types would be int16_t and
int32_t, declared in <stdint.hin C99 implementations that
support them.
--
int main(void){char p[]="ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuv wxyz.\
\n",*q="kl BIcNBFr.NKEzjwCIxNJC";int i=sizeof p/2;char *strchr();int putchar(\
);while(*q){i+=strchr(p,*q++)-p;if(i>=(int)sizeof p)i-=sizeof p-1;putchar(p[i]\
);}return 0;}

Apr 22 '07 #12
Fr************@googlemail.com wrote:
On Apr 20, 10:54 pm, Ben Pfaff <b...@cs.stanford.eduwrote:
>>"barcaroller" <barcarol...@music.netwrites:
Compilers are allowed to insert padding between structure members
or after the end of a structure. The C language doesn't provide
a way to prevent or control this (although your implementation
might).


That's true, though you can work out how much padding there is using
the famous offsetof macro I was asking about a while ago. This seems
like a pretty good use of offsetof to me - lets you unpack a struct to
write a portable binary representation to file.
No it doesn't, you are assuming both the reader and writer use the same
padding, endian representation and data model or the information is in
the file.
>>--
int
Your sig is broken, the separator should be "-- "

--
Ian Collins.
Apr 22 '07 #13
Ian Collins <ia******@hotmail.comwrites:
Fr************@googlemail.com wrote:
>On Apr 20, 10:54 pm, Ben Pfaff <b...@cs.stanford.eduwrote:
>>>--
int

Your sig is broken, the separator should be "-- "
There's nothing wrong with my sig. Someone else misquoted it.
--
"Am I missing something?"
--Dan Pop
Apr 22 '07 #14
Ben Pfaff wrote:
Ian Collins <ia******@hotmail.comwrites:

>>Fr************@googlemail.com wrote:
>>>On Apr 20, 10:54 pm, Ben Pfaff <b...@cs.stanford.eduwrote:

--
int

Your sig is broken, the separator should be "-- "


There's nothing wrong with my sig. Someone else misquoted it.
Sorry, I didn't spot it was quoted.

--
Ian Collins.
Apr 22 '07 #15

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

16
by: Alfonso Morra | last post by:
Hi, I am at the end of my tether now - after spending several days trying to figure how to do this. I have finally written a simple "proof of concept" program to test serializing a structure...
7
by: Zero | last post by:
If we have a structure like: struct something{ int *a; int b; }; We allocate mempry for a using malloc or calloc. The question is when we want to know the size of the structure,...
3
by: Nadav | last post by:
Hi I wonder... Is it possible to define the address to which shared memory will be mapped In other words is it possible to apriory define the address MapViewOfFile returns Dlls are being loaded...
11
by: redefined.horizons | last post by:
First, I would thank all of those that took the time to answer my question about creating an array based on a numeric value stored in a variable. I realize after reading the responses and doing...
1
by: Vikas | last post by:
Hi I have been browsing C struct to C# mapping emails on the newsgroups, but I haven't been able to find a solution to my problem. My C structure looks like this: struct myCstruct { char *...
8
by: barcaroller | last post by:
I have a pointer to a memory block. Is there a way I can map a vector<Tto this memory block? I would like to take advantage of the powerful vector<T> member functions. Please note that I cannot...
3
by: dreiko466 | last post by:
(sorry about my english...) I am a newbie in C (3 month expierience) I have wrote a simple test programm in VS2005, what i do wrong?Please... In this programm i create a double linked list.Then ...
4
by: hugo.arregui | last post by:
Hi! I have two struts like that: struct { int num; int num2; struct b arrayOfB; } a;
5
by: michelqa | last post by:
Hi, I need to call a lot of different native SendMessage to retreive informations from non managed application. Some win32 messages use struct pointer for lparam....how to create and...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.