472,804 Members | 1,107 Online

# A malloc question, part 2

Thanks to those who have answered my original question. I thought I
understood the answer and set out to write some code to prove my
understanding. The code was written without any error checking.

---
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

typedef struct {
char s[23];
int n;
} SomeStruct;

typedef struct {
char user[32];
int age;
char others[100];
} UserStruct;

void my_test_func(int cnt)
{
SomeStruct *p, *q;
UserStruct *x, *y;

printf("Size of SomeStruct=0x%X\n", sizeof(SomeStruct));

p = malloc(cnt*sizeof(SomeStruct));
q = p + 1;
printf("p=0x%X, q=0x%X. q-p=0x%X\n", p, q, (int)q-(int)p);

x = (UserStruct *)(p);
y = x + 1;
printf("x=0x%X, y=0x%X. y-x=0x%X\n", x, y, (int)y-(int)x);

free(p);
}

int main(int argc, char argv[])
{
printf("Size of UserStruct=0x%X\n", sizeof(UserStruct));
my_test_func(sizeof(UserStruct));
return 1;
}
---

The outputs are:
Size of UserStruct=0x88
Size of SomeStruct=0x1C
p=0x20F10, q=0x20F2C. q-p=0x1C
x=0x20F10, y=0x20F98. y-x=0x88

My question is, it seems that I was able to get the pointer to point to
the start of the second struct correctly. I used p, q for SomeStruct
followed by UserStruct; x, y for UserStruct followed by SomeStruct. In
both cases, the pointer has advanced the exact number of bytes
calculated by sizeof().

I think I can cast (p+1) or (x+1) and then access the second struct
previously. Where did I get it wrong this time?

/Why Tea

Jan 2 '07 #1
25 2161
"Why Tea" <yt****@gmail.comwrote in message
Thanks to those who have answered my original question. I thought I
understood the answer and set out to write some code to prove my
understanding. The code was written without any error checking.

---
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

typedef struct {
char s[23];
int n;
} SomeStruct;

typedef struct {
char user[32];
int age;
char others[100];
} UserStruct;

void my_test_func(int cnt)
{
SomeStruct *p, *q;
UserStruct *x, *y;

printf("Size of SomeStruct=0x%X\n", sizeof(SomeStruct));

p = malloc(cnt*sizeof(SomeStruct));
q = p + 1;
printf("p=0x%X, q=0x%X. q-p=0x%X\n", p, q, (int)q-(int)p);

x = (UserStruct *)(p);
y = x + 1;
printf("x=0x%X, y=0x%X. y-x=0x%X\n", x, y, (int)y-(int)x);

free(p);
}

int main(int argc, char argv[])
{
printf("Size of UserStruct=0x%X\n", sizeof(UserStruct));
my_test_func(sizeof(UserStruct));
return 1;
}
---

The outputs are:
Size of UserStruct=0x88
Size of SomeStruct=0x1C
p=0x20F10, q=0x20F2C. q-p=0x1C
x=0x20F10, y=0x20F98. y-x=0x88

My question is, it seems that I was able to get the pointer to point to
the start of the second struct correctly. I used p, q for SomeStruct
followed by UserStruct; x, y for UserStruct followed by SomeStruct. In
both cases, the pointer has advanced the exact number of bytes
calculated by sizeof().

I think I can cast (p+1) or (x+1) and then access the second struct
previously. Where did I get it wrong this time?

/Why Tea
I don't understand your question. You've demonstrated that you can cast one
pointer type to another and then that the compiler will treat the new
pointer as it is supposed to treat it.

When you use a statement such as:

pointer++;

or

pointer += integer_offset;

the compiler has to consider the pointer type (what it points to) when
deciding how to offset the pointer. So, at the machine arithmetic level, a
pointer to apples will get treated differently than a pointer to pears.

More specifically:

Pear *ppear;
Apple *papple;

ppear = (Pear *)(papple = malloc(sizeof(Apple * 100)));

/* ppear and papple are now equal */

ppear++;
papple++;

/* ppear and papple are now probably not equal if Pear and Apple are not of
equal size */

will create unequal pointers (in general).

What you have posted is no surprise. This is the way the compiler should
behave.

You've thrown an apple up in the air, and it came back down. Newton knew
this. I know this. We all know this.

What is your question? You've demonstrated that the compiler behaves as it
should.

Jan 2 '07 #2
What you have posted is no surprise. This is the way the compiler should
behave.

You've thrown an apple up in the air, and it came back down. Newton knew
this. I know this. We all know this.

What is your question? You've demonstrated that the compiler behaves as it
should.

I am sorry David. It was related to a question that I posted previously
with the same topic. I got some excellent answers from Eric Sosman,
Richard Heathfield, etc. But there is still something I'm not totally
clear, that's why I posted part two of the question. I should probably
have ping'ed Eric and Richard on the topic.

Jan 2 '07 #3
"Why Tea" <yt****@gmail.comwrote in message
>
>What you have posted is no surprise. This is the way the compiler should
behave.

clearly.
You've thrown an apple up in the air, and it came back down. Newton knew
this. I know this. We all know this.

What is your question? You've demonstrated that the compiler behaves as
it
should.

I am sorry David. It was related to a question that I posted previously
with the same topic. I got some excellent answers from Eric Sosman,
Richard Heathfield, etc. But there is still something I'm not totally
clear, that's why I posted part two of the question. I should probably
have ping'ed Eric and Richard on the topic.
I'm not annoyed. I'm just curious. No apology necessary.

What aspect of the compiler's behavior has surprised you? It seems to me
that, based on what you posted, it is behaving as it should.
Jan 2 '07 #4

Why Tea wrote:
Thanks to those who have answered my original question. I thought I
understood the answer and set out to write some code to prove my
understanding. The code was written without any error checking.
<snip>
typedef struct {
char s[23];
int n;
} SomeStruct;

typedef struct {
char user[32];
int age;
char others[100];
} UserStruct;

void my_test_func(int cnt)
{
SomeStruct *p, *q;
UserStruct *x, *y;

printf("Size of SomeStruct=0x%X\n", sizeof(SomeStruct));

p = malloc(cnt*sizeof(SomeStruct));
q = p + 1;
printf("p=0x%X, q=0x%X. q-p=0x%X\n", p, q, (int)q-(int)p);

x = (UserStruct *)(p);
y = x + 1;
printf("x=0x%X, y=0x%X. y-x=0x%X\n", x, y, (int)y-(int)x);

free(p);
}
<snip>
My question is, it seems that I was able to get the pointer to point to
the start of the second struct correctly. I used p, q for SomeStruct
followed by UserStruct; x, y for UserStruct followed by SomeStruct. In
both cases, the pointer has advanced the exact number of bytes
calculated by sizeof().
Yes. This is guaranteed (assuming the conversion of pointers to ints
works the way you expect -- there are plenty of systems where pointers
are bigger than ints)
I think I can cast (p+1) or (x+1) and then access the second struct
previously. Where did I get it wrong this time?
Certainly you can store a SomeStruct through q or a UserStruct through
y.
That' s guaranteed by malloc(). The memory is correctly aligned for
storing an
array of anything you like.

What isn't guaranteed is that q is correctly aligned for storing a
UserStruct
or that y is correctly aligned for storing a SomeStruct, though in
this case
they probably would be. That is, you may not be able to store a
SomeStruct
through (SomeStruct *) y or a UserStruct through (UserStruct *) q.
This
was the point of the discussion of your previous question.

The block you malloc()ed can be used for storing SomeStructs, or for
storing UserStructs, but it is not guaranteed that it can be used for
storing
both in alternation.

I can't think of any reasonable way that the particular definitions you
used
could cause trouble, but the much simpler

typedef struct { double d;} SomeStruct ;
typedef struct { char c;} UserStruct ;

could. With these definitions, on a machine that required four-byte
alignment
for doubles you might have sizeof(SomeStruct) == 4 and
sizeof(UserStruct) == 1. In this case y would not be at a four-byte
boundary
and so you could not store a SomeStruct through (SomeStruct *) y.

-thomas

Jan 2 '07 #5

Why Tea wrote:
Thanks to those who have answered my original question. I thought I
understood the answer and set out to write some code to prove my
understanding. The code was written without any error checking.

---
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

typedef struct {
char s[23];
int n;
} SomeStruct;

typedef struct {
char user[32];
int age;
char others[100];
} UserStruct;

void my_test_func(int cnt)
{
SomeStruct *p, *q;
UserStruct *x, *y;

printf("Size of SomeStruct=0x%X\n", sizeof(SomeStruct));
You'll need to cast the sizeof expression to (unsigned long)
and use the %lu conversion specifier. If you are using
c99 features, and if the 'z' modifier is supported, you can
use %zu
>
p = malloc(cnt*sizeof(SomeStruct));
q = p + 1;
Maybe someone else can provide a definitive response
but I don't think you can rely on this. p + 1 may not
be correctly aligned for a UserStruct object.
printf("p=0x%X, q=0x%X. q-p=0x%X\n", p, q, (int)q-(int)p);
See below.

>
x = (UserStruct *)(p);
y = x + 1;
printf("x=0x%X, y=0x%X. y-x=0x%X\n", x, y, (int)y-(int)x);
You need the %p conversion specifier to display
the implementation-defined values that represent
If you don't, you invoke undefined behavior.

Also, though I'm not too sure at the moment, I think
(int)y-(int)x will invoke undefined behavior as per 6.5#5
(if I recall correctly), if the values cannot be represented
by the object type int.

--
aegis

Jan 2 '07 #6
>
Also, though I'm not too sure at the moment, I think
(int)y-(int)x will invoke undefined behavior as per 6.5#5
(if I recall correctly), if the values cannot be represented
by the object type int.
This is an interesting topic on its own. He apparently wanted to calculate
the numerical difference between two pointers. I'm not sure if there is a
"legal" way to do this.
Jan 2 '07 #7
"Thomas Lumley" <th****@drizzle.netwrote in message
>
What isn't guaranteed is that q is correctly aligned for storing a
UserStruct
or that y is correctly aligned for storing a SomeStruct, though in
this case
they probably would be. That is, you may not be able to store a
SomeStruct
through (SomeStruct *) y or a UserStruct through (UserStruct *) q.
This
was the point of the discussion of your previous question.
The entire line of questioning seemed odd to me (why is he exploring this
particular aspect of compiler behavior?).

The only thing that comes to mind is that if he really wants to do what in
some contexts is called a "variant record", the way to do this in 'C' is
with a union. In the case of a union, the alignment requirements would
automatically be met.

In a variant record, it is normal to have a "fixed" part. This can be
accomplished by having a structure with a fixed part and then a union.

http://www.cacs.louisiana.edu/~mgr/4...man/node68.htm

http://en.wikipedia.org/wiki/Variant_record
Jan 2 '07 #8
Why Tea wrote:
What you have posted is no surprise. This is the way the compiler should
behave.

You've thrown an apple up in the air, and it came back down. Newton knew
this. I know this. We all know this.

What is your question? You've demonstrated that the compiler behaves as it
should.

I am sorry David. It was related to a question that I posted previously
with the same topic. I got some excellent answers from Eric Sosman,
Richard Heathfield, etc. But there is still something I'm not totally
clear, that's why I posted part two of the question. I should probably
have ping'ed Eric and Richard on the topic.
Given that you're apparently posting through Google Groups, why don't
search for the old thread and post to it? IIRC, it's just a couple of
days old and most news-servers ought to still have it.

Jan 2 '07 #9

David T. Ashley wrote:

Also, though I'm not too sure at the moment, I think
(int)y-(int)x will invoke undefined behavior as per 6.5#5
(if I recall correctly), if the values cannot be represented
by the object type int.

This is an interesting topic on its own. He apparently wanted to calculate
the numerical difference between two pointers. I'm not sure if there is a
"legal" way to do this.
We have the object type 'ptrdiff_t' but that is used
where we take the difference between the locations
of two objects of an array object. And even still, I think
we invoke undefined behavior if the value cannot be
represented by 'ptrdiff_t'.

--
aegis

Jan 2 '07 #10
On Jan 2, 6:03 pm, "David T. Ashley" <d...@e3ft.comwrote:
This is an interesting topic on its own. He apparently wanted to calculate
the numerical difference between two pointers. I'm not sure if there is a
"legal" way to do this.
There is, but with limitations. From C89:
When two pointers to members of the same array object are subtracted,
the difference is divided by the size of a member. The result
represents the difference of the subscripts of the two array members.
The size of the result is implementation-defined, and its type (a
signed integral type) is ptrdiff_t defined in the <stddef.hheader. As
with any other arithmetic overflow, if the result does not fit in the
space provided, the behavior is undefined. If two pointers that do not
point to members of the same array object are subtracted, the behavior
is undefined. However, if P points either to a member of an array
object or one past the last member of an array object, and Q points to
the last member of the same array object, the expression (Q+1) - P has
the same value as (Q-P) + 1 , even though Q+1 does not point to a
member of the array object.
--
WYCIWYG - what you C is what you get

Jan 2 '07 #11
I think I can cast (p+1) or (x+1) and then access the second struct
previously. Where did I get it wrong this time?

Certainly you can store a SomeStruct through q or a UserStruct through
y.
That' s guaranteed by malloc(). The memory is correctly aligned for
storing an
array of anything you like.

What isn't guaranteed is that q is correctly aligned for storing a
UserStruct
or that y is correctly aligned for storing a SomeStruct, though in
this case
they probably would be. That is, you may not be able to store a
SomeStruct
through (SomeStruct *) y or a UserStruct through (UserStruct *) q.
This
was the point of the discussion of your previous question.
Yes, this was the point of discussion. This is also something I
couldn't get a grasp of. If I can store SomeStruct in p and UserStruct
in x, then why I can't I store UserStruct in q and SomeStruct in y?
What happens if I go ahead and do it? Will the system crash?

Please be patient with me while I'm trying to draw a picture of the
memory allocation according to my understanding.

xxx : memory malloc()ed
<<<: SomeStruct
>>>: UserStruct (assumed to be larger than SomeStruct)
p,x
|xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx|

p q
|<<<<<<<<<<<<|>>>>>>>>>>>>>>>>>>>>>>>>>>|

x y
|>>>>>>>>>>>>>>>>>>>>>>>>>>|<<<<<<<<<<<<|
The three diagrams are showing the same memory. If it's possible to
store SomeStruct in p and UserStruct in x, why can't UserStruct be
stored in q and SomeStruct in y?
The block you malloc()ed can be used for storing SomeStructs, or for
storing UserStructs, but it is not guaranteed that it can be used for
storing
both in alternation.

I can't think of any reasonable way that the particular definitions you
used
could cause trouble, but the much simpler
above.

typedef struct { double d;} SomeStruct ;
typedef struct { char c;} UserStruct ;

could. With these definitions, on a machine that required four-byte
alignment
for doubles you might have sizeof(SomeStruct) == 4 and
sizeof(UserStruct) == 1. In this case y would not be at a four-byte
boundary
and so you could not store a SomeStruct through (SomeStruct *) y.
OK. I understood this part.

Jan 2 '07 #12

"matevzb" <ma*****@gmail.comwrote in message
On Jan 2, 6:03 pm, "David T. Ashley" <d...@e3ft.comwrote:
>This is an interesting topic on its own. He apparently wanted to
calculate
the numerical difference between two pointers. I'm not sure if there is
a
"legal" way to do this.
There is, but with limitations. From C89:
When two pointers to members of the same array object are subtracted,
the difference is divided by the size of a member. The result
represents the difference of the subscripts of the two array members.
The size of the result is implementation-defined, and its type (a
signed integral type) is ptrdiff_t defined in the <stddef.hheader. As
with any other arithmetic overflow, if the result does not fit in the
space provided, the behavior is undefined. If two pointers that do not
point to members of the same array object are subtracted, the behavior
is undefined. However, if P points either to a member of an array
object or one past the last member of an array object, and Q points to
the last member of the same array object, the expression (Q+1) - P has
the same value as (Q-P) + 1 , even though Q+1 does not point to a
member of the array object.
These limitations are probably too restrictive for the OPs need's. He was
apparently interested in getting under the hood and seeing what the actual
machine-specific/compiler-specific difference was.
Jan 2 '07 #13
"Why Tea" <yt****@gmail.comwrote in message

The issue isn't what you "can" do--the issue is what you "want to" do.

Most of us want to write code that will execute on all platforms without any
risk of odd behavior.

Using a pointer to data type X and effectively assigning to it data type Y
isn't something most 'C' programmers want to do. It leads to The Blue
Screen Of Death or its equivalent.

If you have a scenario where you need to use the same pointer to access two
qualitatively different data type, the two best options are:

a)If you don't mind the compiler allocating enough space for the largest of
the possibilities, use unions (effectively, variant records).

b)If the space is a problem, break the record involved into two pieces (the
fixed and the variant parts) and allocate the variant parts separately and
precisely.

I think certainly it is critical as a 'C' programmer to be able to
understand what the hardware is doing. It is very helpful to be able to
wade through assembly-language and understand the relationship between the
'C' statements and the assembly-language. I don't dispute the teaching
value of the question you posed. It is a valuable question.

But as a practical matter, it is not usually necessary to trick the compiler
into doing odd things. Using a pointer of type *X to access an object of
type Y is extremely dangerous.

All that being said:
Yes, this was the point of discussion. This is also something I
couldn't get a grasp of. If I can store SomeStruct in p and UserStruct
in x, then why I can't I store UserStruct in q and SomeStruct in y?
What happens if I go ahead and do it? Will the system crash?
Most systems won't crash. But you could easily run into the following
scenario:

The return value from malloc() is guaranteed to be suitable for alignment of
any data type. However, an individual element in an array (i.e. p +
aribtrary_integer) is not guaranteed to have such alignment.

Assume that sizeof(SomeStruct) == 2 and sizeof(UserStruct) == 8.

It could very well be (on a machine that requires modulo 8 alignment) that
using a pointer suitable for a SomeStruct is not always suitable for a
UserStruct. The compiler would be very justified when generating code for
using a (* UserStruct) in assuming that it has modulo 8 alignment. (It can
safely assume this because the only valid ways to get such a pointer will
all ensure the correct alignment.)

Using a not-suitably-aligned pointer to access a UserStruct might lead to
trouble.

Again, if you _must_ do this, use approach (a) or (b) above. It is unwise
to deprive the compiler of the information it needs.
Jan 2 '07 #14
But as a practical matter, it is not usually necessary to trick the compiler
into doing odd things. Using a pointer of type *X to access an object of
type Y is extremely dangerous.

All that being said:
Yes, this was the point of discussion. This is also something I
couldn't get a grasp of. If I can store SomeStruct in p and UserStruct
in x, then why I can't I store UserStruct in q and SomeStruct in y?
What happens if I go ahead and do it? Will the system crash?

Most systems won't crash. But you could easily run into the following
scenario:

The return value from malloc() is guaranteed to be suitable for alignment of
any data type. However, an individual element in an array (i.e. p +
aribtrary_integer) is not guaranteed to have such alignment.

Assume that sizeof(SomeStruct) == 2 and sizeof(UserStruct) == 8.

It could very well be (on a machine that requires modulo 8 alignment) that
using a pointer suitable for a SomeStruct is not always suitable for a
UserStruct. The compiler would be very justified when generating code for
using a (* UserStruct) in assuming that it has modulo 8 alignment. (It can
safely assume this because the only valid ways to get such a pointer will
all ensure the correct alignment.)

Using a not-suitably-aligned pointer to access a UserStruct might lead to
trouble.'s

Again, if you _must_ do this, use approach (a) or (b) above. It is unwise
to deprive the compiler of the information it needs.
It's not what I want to do. As I said in my previous post, I read a
piece of code written by someone else which malloc()s two structs such
as SomeStruct and UserStruct (note, in a single malloc). My first
instinct was that there could be trouble. So I posted the question here
for expert comments. Everyone told me there would be trouble, but no
one could be more specific about the "trouble". When the code starts to
fill (p+1) with the second struct after a cast, will it over step the
end of the malloc()ed memory and POSSIBLY corrupt other data? What will
happen when data is stored into a misaligned memory?

Jan 2 '07 #15
"Why Tea" <yt****@gmail.comwrote in message
>
>But as a practical matter, it is not usually necessary to trick the
compiler
into doing odd things. Using a pointer of type *X to access an object of
type Y is extremely dangerous.

All that being said:
Yes, this was the point of discussion. This is also something I
couldn't get a grasp of. If I can store SomeStruct in p and UserStruct
in x, then why I can't I store UserStruct in q and SomeStruct in y?
What happens if I go ahead and do it? Will the system crash?

Most systems won't crash. But you could easily run into the following
scenario:

The return value from malloc() is guaranteed to be suitable for alignment
of
any data type. However, an individual element in an array (i.e. p +
aribtrary_integer) is not guaranteed to have such alignment.

Assume that sizeof(SomeStruct) == 2 and sizeof(UserStruct) == 8.

It could very well be (on a machine that requires modulo 8 alignment)
that
using a pointer suitable for a SomeStruct is not always suitable for a
UserStruct. The compiler would be very justified when generating code
for
using a (* UserStruct) in assuming that it has modulo 8 alignment. (It
can
safely assume this because the only valid ways to get such a pointer will
all ensure the correct alignment.)

Using a not-suitably-aligned pointer to access a UserStruct might lead to
trouble.'s

Again, if you _must_ do this, use approach (a) or (b) above. It is
unwise
to deprive the compiler of the information it needs.

It's not what I want to do. As I said in my previous post, I read a
piece of code written by someone else which malloc()s two structs such
as SomeStruct and UserStruct (note, in a single malloc). My first
instinct was that there could be trouble. So I posted the question here
for expert comments. Everyone told me there would be trouble, but no
one could be more specific about the "trouble". When the code starts to
fill (p+1) with the second struct after a cast, will it over step the
end of the malloc()ed memory and POSSIBLY corrupt other data? What will
happen when data is stored into a misaligned memory?
It depends on the processor and the operating system. Here are all the
possibilities I'm aware of:

a)No effect whatsoever (usually this happens when either the processor has
been improved over time to tolerate misalignment and has outgrown the
compilers and/or the documentation, or when the hardware design uses RAM in
a configuration not using the full data bus width, or when using a variant
of the processor with a narrower data bus). No effect because (a)the
processor will tolerate it, and (b)the hardware is such that it can't get a
performance bonus from properly-aligned data, so misalignment doesn't cost
anything.

b)Speed decrease. Many processors are designed so that they properly handle
misaligned data, but because it requires multiple fetches the instructions
run slower. Some members of the NEC V850 family are this way. (BTW, you
wouldn't notice the speed decrease, generally. It happens only for some
instructions and only on misaligned data ... we're not talking something
like your program runs half as fast.)

c)Incorrect behavior. Some processors are designed so that the digital
logic ignores the least significant bits of operands for instructions where
alignment is required, usually indices in CPU registers. For example, this
might mean that if you ask for memory location 8, you'll get 8; ask for 9,
and you'll get 8, ask for 10 and you'll get 8, ask for 11 and you'll get 8,
ask for 12 and you'll get 12, ask for 13 and you'll get 12, etc.

d)Exception handled by the operating system (and the behavior can naturally
vary). I would assume the normal case is process termination.

The biggest danger of what you've described is code that will run on one CPU
(or variant) satisfactorily and won't run correctly on another.

I'm not aware of a processor where misalignment _should_ lead to an
operating system crash. But whether it might is another question.

Dave.
Jan 2 '07 #16
On 2 Jan 2007 11:41:36 -0800, "Why Tea" <yt****@gmail.comwrote:
>
>But as a practical matter, it is not usually necessary to trick the compiler
into doing odd things. Using a pointer of type *X to access an object of
type Y is extremely dangerous.

All that being said:
Yes, this was the point of discussion. This is also something I
couldn't get a grasp of. If I can store SomeStruct in p and UserStruct
in x, then why I can't I store UserStruct in q and SomeStruct in y?
What happens if I go ahead and do it? Will the system crash?

Most systems won't crash. But you could easily run into the following
scenario:

The return value from malloc() is guaranteed to be suitable for alignment of
any data type. However, an individual element in an array (i.e. p +
aribtrary_integer) is not guaranteed to have such alignment.

Assume that sizeof(SomeStruct) == 2 and sizeof(UserStruct) == 8.

It could very well be (on a machine that requires modulo 8 alignment) that
using a pointer suitable for a SomeStruct is not always suitable for a
UserStruct. The compiler would be very justified when generating code for
using a (* UserStruct) in assuming that it has modulo 8 alignment. (It can
safely assume this because the only valid ways to get such a pointer will
all ensure the correct alignment.)

Using a not-suitably-aligned pointer to access a UserStruct might lead to
trouble.'s

Again, if you _must_ do this, use approach (a) or (b) above. It is unwise
to deprive the compiler of the information it needs.

It's not what I want to do. As I said in my previous post, I read a
piece of code written by someone else which malloc()s two structs such
as SomeStruct and UserStruct (note, in a single malloc). My first
instinct was that there could be trouble. So I posted the question here
for expert comments. Everyone told me there would be trouble, but no
one could be more specific about the "trouble".
Because there is no specific answer. Even in a particular
implementation the actual behavior may depend on many factors.
>When the code starts to
fill (p+1) with the second struct after a cast, will it over step the
end of the malloc()ed memory and POSSIBLY corrupt other data?
Maybe. Maybe not. It depends, and unless you wrote the compiler, you
don't even know what it may depend on.
>What will
happen when data is stored into a misaligned memory?
It depends. (Familiar answer, eh?) It may do nothing but overwrite
other unused memory. It may cause a bus error. It may do something
else.

Don't jump off that cliff. I can't tell you whether you'll die of a
broken neck, a crushed skull, or some other cause, but it won't
matter.

--
Al Balmer
Sun City, AZ
Jan 2 '07 #17
Why Tea <yt****@gmail.comwrote:
>Yes, this was the point of discussion. This is also something I
couldn't get a grasp of. If I can store SomeStruct in p and UserStruct
in x, then why I can't I store UserStruct in q and SomeStruct in y?
What happens if I go ahead and do it? Will the system crash?
Others have gone into detail, and at this point it is not clear
what p, q, x, and y might be anyway, but ...

The fundamental problem here is the thing called "alignment".

pointer, avoid casts, and avoid cheating with "void *", you always
end with a well-alignment pointer.

The malloc() function always either returns NULL (no more memory
available), or a "well-aligned" pointer, suitably aligned for use
as *any* data type, including user-defined types. So you start
with a "well-aligned" pointer in this case, and if you avoid casts
(and do not use "void *" intermediates as a "cheat" -- more on this
later), you will have a well-aligned pointer.

The C standard says that if you use a well-aligned pointer, it has
to work. So any compiler that can claim to implement C has to make
it work, no matter what that takes.

On the other hand, if you use casts to *change* pointer types while
doing arithmetic on them, the C standard does *not* say that the
result will be suitably aligned. The behavior becomes officially
"undefined".

In practice, the actual behavior depends on the implementation --
both the CPU and the compiler. So does what "suitably aligned"
means. On some hardware, *everything* is "suitably aligned", and
it always works no matter what. (For instance, the x86 is like
this. Sometimes the code runs a little slower, is all.) On
other hardware, different things happen.

But consider instead the PowerPC architectures (such as in a Mac
G4 or G5). Here, the machines are less forgiving. If you use a
"load byte" or "store byte" instruction, you can point to any byte
in memory at all. But if you use a "load halfword" or "store
halfword" instruction (to access a 16-bit item, e.g., a "short" in
most C compilers for these), the hardware refuses to do it if the
address is not even (zero mod 2, mathematically). If you use a
"load word" or "store word" instruction (to access a 32-bit item),
the hardware refuses to do it if the address is not a multiple of
4.

The form of the refusal -- its effect on your program when you run
it -- depends on various items. Using OS X or Linux, you will get
a signal that will normally terminate your program (usually with a
"core dump"). If your program runs deep enough in the operating
system, you could indeed get a "system crash", though.

On yet another kind of hardware, the ARM family of CPUs, something
even weirder happens. If you create a "poorly aligned" pointer,
and then ask the CPU to use it, the CPU *does* use it -- but it
first "shaves off" the bits that caues it to be poorly aligned,
resulting in a well-aligned pointer that it *can* use. The resulting
well-aligned pointer no longer points where the original, poorly-aligned
pointer used to point. As with the PowerPC, alignment requirements
are a function of data access size -- so the result is that:

*(int *)p = 42;

is equivalent to:

*(int *)((intptr_t)p & ~3) = 42;

Because the C standard says that the effect of an unaligned pointer
is "undefined", *all* of these behaviors are allowed. All of these
systems can call themselves "C implementations", even though the
x86, PowerPC, and ARM all do something *different*.

The C standard says, in effect, that it is your job -- the C
programmer's job -- to make sure you never produce an unaligned
returns a well-aligned pointer (for any use), and "simple" (castless)
pointer arithmetic keeps a well-aligned pointer (for whatever use
it has now) well-aligned. So:

T *p; /* for any type T */

p = malloc(N * sizeof *p); /* obtain room for N items of type T */
if (p == NULL) ... handle error ...

*(p + k) = value; /* is OK: there are no casts */

If you find you need a cast, for any reason, this is a sign that
waters. :-) So:

T *p;
T2 *q;

p = malloc(sizeof *p + N); /* room for 1 p, plus N extra bytes */
if (p == NULL) ... handle error ...
*p = whatever; /* OK */

q = (T2 *)(p + 1); /* danger */

While "p+1" is well-aligned for use as a type "T", it is not
*necessarily* well-aligned for use as a type "T2". It is not
necessarily badly-aligned either. The C standard simply does not
say, one way or the other (with one exception: bytes -- "char"s,
specifically unsigned char although plain and signed char get
carried along for the ride -- are always well-aligned, no matter
what else is going on).

Now, you can avoid the casts by "cheating" with a "void *"
temporary variable:

T *p;
void *tmp;
T2 *q;

p = malloc(sizeof *p + N); /* room for 1 p, plus N extra bytes */
if (p == NULL) ... handle error ...
*p = whatever; /* OK */

tmp = p + 1; /* OK */

q = tmp; /* danger */

Even though the cast has now been removed, the danger remains.
The problem is -- still -- that "p + 1" is well-aligned for use as
a "T" (because p came from malloc, so it was well-aligned to start
with), but not necessarily for use as a "T2".

Again, the underlying fundamental problem -- which is hardware
and sometimes compiler dependent -- is "alignment".

It would perhaps be nice if C had a tool for checking whether
something was "well-aligned":

tmp = p + 1;
if (IS_WELL_ALIGNED_FOR_USE_AS_A_T2(tmp))
... proceed ...
else
... something ...

except this leaves two problems, one obvious:

- what do we put in the "else"?

and one slightly less obvious:

- what would we need to do to *guarantee* that tmp is well-aligned
for use as a T2?

That second question is really the important one. Suppose that
you had a way to predict the alignment requirements for "T2"s.
We know (from C's basic design) that there must be some number
of bytes ("char"s) that we can use as padding, placed after a
T and before a T2, so that the T2 that comes after the (possibly
empty) padding and is now well-aligned:

<object of type T><padding><object of type T2>

which is what you would get if you did this:

struct combined {
T first;
/* compiler inserts padding here if needed */
T2 second;
};

Then you could do this:

char *x;
...
p = malloc(sizeof *p + sizeof(T2) + worst_case_padding);
... the usual check for NULL ...
x = (char *)(p + 1);
q = (T2 *)x;

But if you know the types up-front like this, you can always just
*make* a "struct combined", and replace all of the above with:

struct combined *comb;

comb = malloc(sizeof *comb);
if (comb == NULL) ...

p = &comb->first;
q = &comb->second;

So this is not quite as useful as it seems.

The real problem comes in when you do *not* know the types up-front.
Suppose, for instance, you are writing a replacement malloc() (which
you might name "nmalloc", perhaps). You will probably want to make
the same guarantee that C does about malloc(): that it returns a
pointer that is well-aligned for use as *any* type, no matter what
that type may be. But all malloc() -- and hence nalloc() -- gets
is a size in bytes. How can it find the type? Or, equivalently,
how can it find the "maximum" alignment required by the compiler
and hardware?

The only real Standard C answer is "it can't". You cannot write
malloc() in Standard C. You need at least one piece of "NonStandard
C": you need to know the underlying alignment constraints of the
hardware.

So, if you want to write a general-purpose allocator, you have
two choices:

- forget it, or
- "don't use Standard C".

The first one is often not satisfactory. Note that in abandoning
the C standard while writing this allocator, you need not abandon
Standard C *everywhere*. You can do it just in this one little
place, in this one little way, and hope (with good justification:
you know your implementor had to do this too) that this does not
cause the whole program to fall apart somehow.

Note, though, that the first approach ("forget it") is actually
often a good one. For instance, *if* you can make a "struct
combined", as I did above, you can then obtain your two pointers
(p and q above) in one malloc() call, and know that both are
well-aligned. The big limitation here is that you must already
have the types of all the sub-objects (T and T2, in this case) set
in stone.
--
In-Real-Life: Chris Torek, Wind River Systems
Salt Lake City, UT, USA (40°39.22'N, 111°50.29'W) +1 801 277 2603
Reading email is like searching for food in the garbage, thanks to spammers.
Jan 2 '07 #18
"Chris Torek" <no****@torek.netwrote in message
news:en*********@news1.newsguy.com...
Why Tea <yt****@gmail.comwrote:
>>Yes, this was the point of discussion. This is also something I
couldn't get a grasp of. If I can store SomeStruct in p and UserStruct
in x, then why I can't I store UserStruct in q and SomeStruct in y?
What happens if I go ahead and do it? Will the system crash?

Others have gone into detail, and at this point it is not clear
what p, q, x, and y might be anyway, but ...

The fundamental problem here is the thing called "alignment".

pointer, avoid casts, and avoid cheating with "void *", you always
end with a well-alignment pointer.

The malloc() function always either returns NULL (no more memory
available), or a "well-aligned" pointer, suitably aligned for use
as *any* data type, including user-defined types. So you start
with a "well-aligned" pointer in this case, and if you avoid casts
(and do not use "void *" intermediates as a "cheat" -- more on this
later), you will have a well-aligned pointer.
If I'm reading your statement correctly, I'm not sure that it is true.

For example, even on the x86 architecture, doing pointer arithmetic with a
pointer to a character will take it out of the alignment desired for larger
data types.

I don't believe in general that it is guaranteed that pointer arithmetic
won't generate a pointer less-well-aligned than malloc() returns.

The malloc() return value is guaranteed to have suitable alignment for any
data array.

A result of pointer arithmetic is guaranteed to have suitable alignment so
long as you were truthful with the compiler about what the data type is.

BUT ...

I don't believe it is guaranteed that if you:

a)Declare a pointer as:

TYPE_A *ptr;

b)Get a pointer via malloc(), i.e.:

ptr = malloc(sizeof(TYPE_A) * 10);

c)Do arithmetic on it, i.e.

ptr++;

that it will then be suitably aligned to point to another data type (besides
TYPE_A).

The value returned by malloc() is guaranteed to be suitable for any data
type.

The results of pointer arithmetic are not guaranteed to be equally suitable.

Whack on me if you don't agree.
Jan 2 '07 #19
"Why Tea" <yt****@gmail.comwrites:
Thanks to those who have answered my original question. I thought I
understood the answer and set out to write some code to prove my
understanding. The code was written without any error checking.
There are also some dangerous things in your code. I'll point them
---
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

typedef struct {
char s[23];
int n;
} SomeStruct;

typedef struct {
char user[32];
int age;
char others[100];
} UserStruct;

void my_test_func(int cnt)
{
SomeStruct *p, *q;
UserStruct *x, *y;

printf("Size of SomeStruct=0x%X\n", sizeof(SomeStruct));
The "%X" format expects an unsigned int. You're giving it a size_t,
which is not necessarily the same type. Try this:

printf("Size of SomeStruct=0x%lX\n",
(unsigned long)sizeof(SomeStruct));
p = malloc(cnt*sizeof(SomeStruct));
q = p + 1;
printf("p=0x%X, q=0x%X. q-p=0x%X\n", p, q, (int)q-(int)p);
Here you're using "%X" to print pointer values; that's also a no-no.
Use "%p" and convert each pointer to void*:

printf("p=%p, q=%p\n", (void*)p, (void*)q);

You're also converting pointer values to type int, which is legal, but
the results are implementation-defined (and I've worked on systems
with odd pointer formats where the results of such conversions are not
particularly meaningful).

But pointer subtraction is defined by the language; it yields a result
of type ptrdiff_t (which is a typedef for some signed integer type).
The result of q-p is going to be 1, because the result is scaled by
the size of the pointed-to type. If you want the difference in bytes,
convert each pointer to char*:

printf("p=%p, q=%p, q-p=%ld\n",
(void*)p, (void*)q, (char*)q-(char*)p);

Note that we convert to char*, not to void*, since pointer arithmetic
is not defined for void* (the pointed-to type has no size) -- though
some compilers may support it as an extension.
x = (UserStruct *)(p);
y = x + 1;
printf("x=0x%X, y=0x%X. y-x=0x%X\n", x, y, (int)y-(int)x);
As above.
>
free(p);
}

int main(int argc, char argv[])
The correct declaration for argv is "char *argv[]" or "char **argv".
But since you're not actually using either argc or argv, you might as
well declare it as:

int main(void)
{
printf("Size of UserStruct=0x%X\n", sizeof(UserStruct));
"%X" expects an unsigned int, but you're giving it a size_t; see above.
my_test_func(sizeof(UserStruct));
my_test_func() takes an argument of type int, but you're passing it a
size_t. This isn't likely to be a problem in this case; since
my_test_func() isn't variadic, and its declaration is visible, the
compiler will automatically generate an implicit conversion from
size_t to int, which is likely to preserve the correct value in this
case. Nevertheless, it would be better to declare my_test_func() with
a parameter of type size_t.
return 1;
Why are you returning the value 1 from main()? On some systems, this
indicates an error condition; on others, it may be meaningless. The
only portable values you can return from main() are 0, EXIT_SUCCESS,
and EXIT_FAILURE.
}
[snip]

It's likely that none of these errors will cause any actual problems
*on many systems*. But your code could break badly if you try to
compile and execute it on a system with different characteristics
(e.g., where int and pointers have different sizes, or are passed as
arguments in different ways).

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Jan 3 '07 #20
On 2 Jan 2007 22:00:59 GMT, Chris Torek wrote:
>Now, you can avoid the casts by "cheating" with a "void *"
temporary variable:

T *p;
void *tmp;
T2 *q;

p = malloc(sizeof *p + N); /* room for 1 p, plus N extra bytes */
if (p == NULL) ... handle error ...
*p = whatever; /* OK */

tmp = p + 1; /* OK */

q = tmp; /* danger */

Even though the cast has now been removed, the danger remains.
The problem is -- still -- that "p + 1" is well-aligned for use as
a "T" (because p came from malloc, so it was well-aligned to start
with), but not necessarily for use as a "T2".

Again, the underlying fundamental problem -- which is hardware
and sometimes compiler dependent -- is "alignment".

It would perhaps be nice if C had a tool for checking whether
something was "well-aligned":

tmp = p + 1;
if (IS_WELL_ALIGNED_FOR_USE_AS_A_T2(tmp))
... proceed ...
else
... something ...

except this leaves two problems, one obvious:

- what do we put in the "else"?

and one slightly less obvious:

- what would we need to do to *guarantee* that tmp is well-aligned
for use as a T2?

#define IS_WELL_ALIGNED_FOR_USE_AS_A_T2(tmp) !((tmp)& (sizeof(T2)-1))

should be ok for all the above cases (x86, powerpc, arm)
Jan 3 '07 #21
Chris Torek wrote:
On yet another kind of hardware, the ARM family of CPUs, something
even weirder happens. If you create a "poorly aligned" pointer,
and then ask the CPU to use it, the CPU *does* use it -- but it
first "shaves off" the bits that caues it to be poorly aligned,
resulting in a well-aligned pointer that it *can* use. The resulting
well-aligned pointer no longer points where the original, poorly-aligned
pointer used to point. As with the PowerPC, alignment requirements
are a function of data access size -- so the result is that:

*(int *)p = 42;

is equivalent to:

*(int *)((intptr_t)p & ~3) = 42;
On the Archimedes and RISC PC family of ARM machines, it is (or perhaps
was) even weirder, if I recall correctly.

The pointer is shaved, the word is fetched, and /then/ the shaved
address bits are used to rotate the fetched data bits so that the
low byte of the data is the byte a load-byte instruction would have
fetched.

clear out the high halfword. (Storing, not so simple.)

I think it was some collaboration between the CPU and the memory
management unit, rather than a purely CPU operation. But time
passes and bits degrade, so I may be worng in the details.

--
Chris "hopefully not Pyecroft" Dollin
The "good old days" used to be much better.

Jan 3 '07 #22
av <av@ala.awrites:

#define IS_WELL_ALIGNED_FOR_USE_AS_A_T2(tmp) !((tmp)& (sizeof(T2)-1))

should be ok for all the above cases (x86, powerpc, arm)
Its safe but there are false negative: the alignment is a divisor of the
size as reported by sizeof.

Yours,

--
Jean-Marc
Jan 3 '07 #23
David T. Ashley wrote:
>Also, though I'm not too sure at the moment, I think
(int)y-(int)x will invoke undefined behavior as per 6.5#5
(if I recall correctly), if the values cannot be represented
by the object type int.

This is an interesting topic on its own. He apparently wanted to calculate
the numerical difference between two pointers. I'm not sure if there is a
"legal" way to do this.
Any object in C may be accessed as if it were an array of unsigned char
by converting a pointer within that object to type (unsigned char *).

If you do that to both y and x, and subtract them, you will get their
difference in bytes, since the size of unsigned char is by definition 1.

Casting them to int, even if the operation is valid, is not guaranteed
to give the same result. Generally this is a problem if the size of int
is smaller than the size of a pointer.

--
Simon.
Jan 3 '07 #24
av wrote:
On 2 Jan 2007 22:00:59 GMT, Chris Torek wrote:
>>
- what would we need to do to *guarantee* that tmp is well-aligned
for use as a T2?

#define IS_WELL_ALIGNED_FOR_USE_AS_A_T2(tmp) !((tmp)& (sizeof(T2)-1))

should be ok for all the above cases (x86, powerpc, arm)
Try it with any of

typedef struct { double x, y, z; } T2;
typedef int T2[42];
typedef char T2[8192];

.... and you'll see that the test is too stringent. It can be
shown that the alignment requirement for a type is a divisor
of the type's sizeof (otherwise arrays wouldn't work), but the
sizeof itself may overstate the requirement. (Also, the `&(N-1)'
trick only works for N a power of two; the % operator would be
an improvement.)

--
Eric Sosman
es*****@acm-dot-org.invalid
Jan 3 '07 #25
Simon Biber <ne**@ralmin.ccwrites:
David T. Ashley wrote:
>>Also, though I'm not too sure at the moment, I think
(int)y-(int)x will invoke undefined behavior as per 6.5#5
(if I recall correctly), if the values cannot be represented
by the object type int.
This is an interesting topic on its own. He apparently wanted to
calculate the numerical difference between two pointers. I'm not
sure if there is a "legal" way to do this.

Any object in C may be accessed as if it were an array of unsigned
char by converting a pointer within that object to type (unsigned char
*).

If you do that to both y and x, and subtract them, you will get their
difference in bytes, since the size of unsigned char is by definition
1.

Casting them to int, even if the operation is valid, is not guaranteed
to give the same result. Generally this is a problem if the size of
int is smaller than the size of a pointer.
It can also be a problem if pointers have a non-obvious
representation, and pointer-to-integer conversion just preserves the
bits.

I once ran across a statement in production software that assumed that
converting pointers to int and subtracting them was a valid way to
determine their difference. This failed to work on one of the
platforms that the software was expected to support. I changed it to
a simple pointer subtraction, which worked on all platforms.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Jan 3 '07 #26

This thread has been closed and replies have been disabled. Please start a new discussion.