Address syntax

[Adam Warner]

Hi all,

In the code snippet below I successfully determine the address of val1:*

struct o val1=l_SYM_2B(& a).o[0];
print_aesthetic (&val1);

The structure o is heavyweight. I understand (hopefully correctly) that
(barring compiler optimisations) C will shallow copy the structure into
val1.

As I merely wished to pass the structure to print_aesthetic I combined the
statements to avoid creating an intermediate copy of the structure. This
was my attempt:

print_aesthetic (&(l_SYM_2B(&a) .o[0]));

I discovered that the only way to get GCC 3.4 to accept this syntax was to
append the -std=c99 compiler option. Otherwise GCC produced the error
message "invalid operands to binary +", i.e. it appeared to be trying to
interpret the address operator as a binary AND.

Is this a bug in GCC's pre-C99 support or has the syntax of C been changed
in C99 to support the code I wrote above? If so, thank goodness!

Regards,
Adam

* Footnote:
a is a local variable of type struct o.
l_SYM_2B(struct o * arg0) returns struct v1, defined as { struct o o[1]; };
print_aesthetic has the prototype struct v1 print_aesthetic (struct o *);

[Dietmar Schindler]

Adam Warner wrote:

In the code snippet below I successfully determine the address of val1:*

struct o val1=l_SYM_2B(& a).o[0];
print_aesthetic (&val1);

The structure o is heavyweight. I understand (hopefully correctly) that
(barring compiler optimisations) C will shallow copy the structure into
val1.

As I merely wished to pass the structure to print_aesthetic I combined the
statements to avoid creating an intermediate copy of the structure. This
was my attempt:

print_aesthetic (&(l_SYM_2B(&a) .o[0]));

I discovered that the only way to get GCC 3.4 to accept this syntax was to
append the -std=c99 compiler option. Otherwise GCC produced the error
message "invalid operands to binary +", i.e. it appeared to be trying to
interpret the address operator as a binary AND.

Is this a bug in GCC's pre-C99 support or has the syntax of C been changed
in C99 to support the code I wrote above? If so, thank goodness!
...
* Footnote:
a is a local variable of type struct o.
l_SYM_2B(struct o * arg0) returns struct v1, defined as { struct o o[1]; };
print_aesthetic has the prototype struct v1 print_aesthetic (struct o *);

If l_SYM_2B is a function, you may get an intermediate copy of the
structure anyway. If it is a macro, it might work without ():

print_aesthetic (&l_SYM_2B(&a). o[0]);

[Keith Thompson]

Adam Warner <us****@consult ing.net.nz> writes:

In the code snippet below I successfully determine the address of val1:*

struct o val1=l_SYM_2B(& a).o[0];
print_aesthetic (&val1);

The structure o is heavyweight. I understand (hopefully correctly) that
(barring compiler optimisations) C will shallow copy the structure into
val1.

As I merely wished to pass the structure to print_aesthetic I combined the
statements to avoid creating an intermediate copy of the structure. This
was my attempt:

print_aesthetic (&(l_SYM_2B(&a) .o[0]));

I discovered that the only way to get GCC 3.4 to accept this syntax was to
append the -std=c99 compiler option. Otherwise GCC produced the error
message "invalid operands to binary +", i.e. it appeared to be trying to
interpret the address operator as a binary AND.

Is this a bug in GCC's pre-C99 support or has the syntax of C been changed
in C99 to support the code I wrote above? If so, thank goodness!

I think the "+" it's complaining about is the one implicit in the
indexing operator. x[y] is equivalent to *(x+y). (Usually x is an
address, often resulting from the implicit conversion of an array
name, and y is an integer.)

--
Keith Thompson (The_Other_Keit h) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.

[Adam Warner]

Hi Dietmar Schindler,
On Wed, 29 Dec 2004 15:27:27 +0100, Dietmar Schindler wrote:

* Footnote:
a is a local variable of type struct o.
l_SYM_2B(struct o * arg0) returns struct v1, defined as { struct o o[1]; };
print_aesthetic has the prototype struct v1 print_aesthetic (struct o *);

If l_SYM_2B is a function, you may get an intermediate copy of the
structure anyway. If it is a macro, it might work without ():

print_aesthetic (&l_SYM_2B(&a). o[0]);

l_SYM_2B is a function (l_SYM is a namespace prefix and underscore is the
escape character for non-alphanumerics. 0x2B is the ASCII code for +. It
is to be my dynamic Lisp function +).

v1, v2, ... are structures for stack allocating multiple return values:

struct v1 {
struct o o[1];
};

struct v2 {
struct o o[2];
};

struct v3 {
struct o o[3];
};

l_SYM_2B() returns the structure v1, i.e. leaves room for one object o on
the stack, aligned to the same position as if struct o itself was returned.

I sincerely hope no compiler will ever create a copy of struct o before
returning its address. I am only using the struct v1 for symmetry with v2,
v3, etc. The address of the first object o is expected to be the same as
the v1 structure. As I am only obtaining the address of o[0] within the
structure v1 I cannot see how C semantics could result in an intermediate
copy of o.

Regards,
Adam

[Andrey Tarasevich]

Adam Warner wrote:

In the code snippet below I successfully determine the address of val1:*

struct o val1=l_SYM_2B(& a).o[0];
print_aesthetic (&val1);

The structure o is heavyweight. I understand (hopefully correctly) that
(barring compiler optimisations) C will shallow copy the structure into
val1.

As I merely wished to pass the structure to print_aesthetic I combined the
statements to avoid creating an intermediate copy of the structure. This
was my attempt:

print_aesthetic (&(l_SYM_2B(&a) .o[0]));

I discovered that the only way to get GCC 3.4 to accept this syntax was to
append the -std=c99 compiler option. Otherwise GCC produced the error
message "invalid operands to binary +", i.e. it appeared to be trying to
interpret the address operator as a binary AND.

Is this a bug in GCC's pre-C99 support or has the syntax of C been changed
in C99 to support the code I wrote above? If so, thank goodness!

AFAIK the code is supposed to be ill-formed in both C89/C90 and C99
because you are trying to apply the address-of operator to a non-lvalue
argument. (Although I couldn't find an exact place in either document
that would explicitly state that function return value is not an lvalue
or prohibit this application of '&' in some other way. Anyone?).

However, even if we'd assume for a second that this code is well-formed,
we'd still have to come to a conclusion that the code is broken at least
in C99. According to C99 (6.5.2.2/5), an attempt to access the returned
value after the next sequence point results in undefined behavior. If
inside 'print_aestheti c' you are actually trying to access the value
pointed by the argument (i.e. the return value of 'l_SYM_2B'), you are
in violation of that rule, since 'print_aestheti c' has a sequence point
at the entry. (Again, I couldn't find an equivalent statement in C89/C90.)

--
Best regards,
Andrey Tarasevich

[Andrey Tarasevich]

Andrey Tarasevich wrote:

Adam Warner wrote:

In the code snippet below I successfully determine the address of val1:*

struct o val1=l_SYM_2B(& a).o[0];
print_aesthetic (&val1);

The structure o is heavyweight. I understand (hopefully correctly) that
(barring compiler optimisations) C will shallow copy the structure into
val1.

As I merely wished to pass the structure to print_aesthetic I combined the
statements to avoid creating an intermediate copy of the structure. This
was my attempt:

print_aesthetic (&(l_SYM_2B(&a) .o[0]));

I discovered that the only way to get GCC 3.4 to accept this syntax was to
append the -std=c99 compiler option. Otherwise GCC produced the error
message "invalid operands to binary +", i.e. it appeared to be trying to
interpret the address operator as a binary AND.

Is this a bug in GCC's pre-C99 support or has the syntax of C been changed
in C99 to support the code I wrote above? If so, thank goodness!

AFAIK the code is supposed to be ill-formed in both C89/C90 and C99
because you are trying to apply the address-of operator to a non-lvalue
argument. (Although I couldn't find an exact place in either document
that would explicitly state that function return value is not an lvalue
or prohibit this application of '&' in some other way. Anyone?).

However, even if we'd assume for a second that this code is well-formed,
we'd still have to come to a conclusion that the code is broken at least
in C99. According to C99 (6.5.2.2/5), an attempt to access the returned
value after the next sequence point results in undefined behavior. If
inside 'print_aestheti c' you are actually trying to access the value
pointed by the argument (i.e. the return value of 'l_SYM_2B'), you are
in violation of that rule, since 'print_aestheti c' has a sequence point
at the entry. (Again, I couldn't find an equivalent statement in C89/C90.)

On the second thought, the code should probably circumvent the "lvalue
check" since it actually accesses an element of an array aggregated by
the returned struct. Array element access in C decays to pointer
arithmetics, which will formally turn the '&'s argument into an lvalue
(i.e. 'l_SYM_2B(&a).o[0]' is '*(l_SYM_2B(&a) .o + 0)' and the result of
'*' is always an lvalue).

However, the second problem - UB - is still there.

--
Best regards,
Andrey Tarasevich

[Adam Warner]

Hi Andrey Tarasevich,
On Wed, 29 Dec 2004 16:27:26 -0800, Andrey Tarasevich wrote:

Adam Warner wrote:

In the code snippet below I successfully determine the address of val1:*

struct o val1=l_SYM_2B(& a).o[0];
print_aesthetic (&val1);

The structure o is heavyweight. I understand (hopefully correctly) that
(barring compiler optimisations) C will shallow copy the structure into
val1.

As I merely wished to pass the structure to print_aesthetic I combined the
statements to avoid creating an intermediate copy of the structure. This
was my attempt:

print_aesthetic (&(l_SYM_2B(&a) .o[0]));

I discovered that the only way to get GCC 3.4 to accept this syntax was to
append the -std=c99 compiler option. Otherwise GCC produced the error
message "invalid operands to binary +", i.e. it appeared to be trying to
interpret the address operator as a binary AND.

Is this a bug in GCC's pre-C99 support or has the syntax of C been changed
in C99 to support the code I wrote above? If so, thank goodness!

AFAIK the code is supposed to be ill-formed in both C89/C90 and C99
because you are trying to apply the address-of operator to a non-lvalue
argument. (Although I couldn't find an exact place in either document
that would explicitly state that function return value is not an lvalue
or prohibit this application of '&' in some other way. Anyone?).

However, even if we'd assume for a second that this code is well-formed,
we'd still have to come to a conclusion that the code is broken at least
in C99. According to C99 (6.5.2.2/5), an attempt to access the returned
value after the next sequence point results in undefined behavior. If
inside 'print_aestheti c' you are actually trying to access the value
pointed by the argument (i.e. the return value of 'l_SYM_2B'), you are
in violation of that rule, since 'print_aestheti c' has a sequence point
at the entry. (Again, I couldn't find an equivalent statement in C89/C90.)

I've been considering similar issues after I tried to compile this to show
the addresses were the same:

printf ("&(l_SYM_2B(&a )) is %p\n", &(l_SYM_2B(&a)) );
printf ("&(l_SYM_2B(&a ).o[0]) is %p\n", &(l_SYM_2B(&a). o[0]));

The first statement correctly generates the error: "invalid lvalue in
unary `&'" according to 6.5.3.2 (a meta question is *why* it should be
prohibited to obtain the address of a return value on the stack. Sometimes
C feels frustratingly high level :-)

The second one may however be legitimate in C99. The semantics state (note
that this the final draft): <http://dev.unicals.com/c99-draft.html#6.5. 3.2>

Similarly, if the operand is the result of a [] operator, neither the &
operator nor the unary * that is implied by the [] is evaluated and the
result is as if the & operator were removed and the [] operator were
changed to a + operator. Otherwise, the result is a pointer to the
object or function designated by its operand.

The (l_SYM_2B(&a).o[0]) operand appears to be the result of a [] operator.
So the result is evaluated as if the & operator were removed and the []
operator were changed to a + operator.

print_aesthetic does try to access the value pointed by the argument.
It has the prototype: struct v1 print_aesthetic (struct o *);

There's no reason this can't become struct v1 print_aesthetic (struct o[])
because arrays are also passed by reference. If you believe this change to
print_aesthetic is necessary to be conforming I can make the change. I'd
like to avoid it because the syntax is messier.

With the function argument "struct o * arg" I can use structure
deferencing notation, e.g. arg->type, and follow pointer members to the
next object. With the function argument "struct o * arg[]" I have to
reference members as arg[0].type, arg[1].type, etc.

Regards,
Adam

[Andrey Tarasevich]

Adam Warner wrote:

...
AFAIK the code is supposed to be ill-formed in both C89/C90 and C99
because you are trying to apply the address-of operator to a non-lvalue
argument. (Although I couldn't find an exact place in either document
that would explicitly state that function return value is not an lvalue
or prohibit this application of '&' in some other way. Anyone?).

However, even if we'd assume for a second that this code is well-formed,
we'd still have to come to a conclusion that the code is broken at least
in C99. According to C99 (6.5.2.2/5), an attempt to access the returned
value after the next sequence point results in undefined behavior. If
inside 'print_aestheti c' you are actually trying to access the value
pointed by the argument (i.e. the return value of 'l_SYM_2B'), you are
in violation of that rule, since 'print_aestheti c' has a sequence point
at the entry. (Again, I couldn't find an equivalent statement in C89/C90.)
I've been considering similar issues after I tried to compile this to show
the addresses were the same:

printf ("&(l_SYM_2B(&a )) is %p\n", &(l_SYM_2B(&a)) );
printf ("&(l_SYM_2B(&a ).o[0]) is %p\n", &(l_SYM_2B(&a). o[0]));

The first statement correctly generates the error: "invalid lvalue in
unary `&'" according to 6.5.3.2 (a meta question is *why* it should be
prohibited to obtain the address of a return value on the stack. Sometimes
C feels frustratingly high level :-)

The second one may however be legitimate in C99. The semantics state (note
that this the final draft): <http://dev.unicals.com/c99-draft.html#6.5. 3.2>

Similarly, if the operand is the result of a [] operator, neither the &
operator nor the unary * that is implied by the [] is evaluated and the
result is as if the & operator were removed and the [] operator were
changed to a + operator. Otherwise, the result is a pointer to the
object or function designated by its operand.

The (l_SYM_2B(&a).o[0]) operand appears to be the result of a [] operator.
So the result is evaluated as if the & operator were removed and the []
operator were changed to a + operator.

I found one key difference between the C89/C90 and C99 that must be at
play here.

The property of "being a non-lvalue" propagates from the left side of
'.' operator to its result in both C89/90 and C99. This means that the
following expression

l_SYM_2B(&a).o

is not an lvalue in both C89/90 and C99.

Now, the key moment: in C89/C90 the array-to-pointer conversion was only
applicable to _lvalues_ of array type (see 6.2.2.1). In C99 this
limitation was removed (see 6.3.2.1/3), and now the array-to-pointer
conversion is applicable to any values of array type.

This means that the following expression

l_SYM_2B(&a).o[0]

is ill-formed in C89/90 (there's no way to apply operator '[]' to the
result of previous expression) and well-formed in C99. For example, the
following simple code will be rejected by Comeau Online compiler in
C89/C90 mode and accepted in C99 mode for this very reason

struct S { int i[5]; };

struct S foo() { struct S s = { 0 }; return s; }

int main() { int i = foo().i[1]; }

Note, that this code does not involve operator '&' at all.

However, this also means that you original code (with intermediate
variable 'val1') was also ill-formed in C89/C90. But you said that you
could compile it without any problems. Apparently, some quirks of
concrete compilers are also at play here.

print_aesthetic does try to access the value pointed by the argument.
It has the prototype: struct v1 print_aesthetic (struct o *);
Once again, if I'm not missing something, that definitely leads to UB in
both C89/C90 and C99 for reasons explained in my previous messages.
There's no reason this can't become struct v1 print_aesthetic (struct o[])
because arrays are also passed by reference.
This is an exact equivalent of the previous declaration. It won't make
any difference.
If you believe this change to
print_aesthetic is necessary to be conforming I can make the change. I'd
like to avoid it because the syntax is messier. With the function argument "struct o * arg" I can use structure
deferencing notation, e.g. arg->type, and follow pointer members to the
next object. With the function argument "struct o * arg[]" I have to
reference members as arg[0].type, arg[1].type, etc.

Did you mean 'struct o arg[]'? If yes, then you are wrong. The following
two declarations

struct v1 print_aesthetic (struct o *arg)
struct v1 print_aesthetic (struct o arg[])

are exactly equivalent in C. In both cases you can use 'arg->type' and
'arg[0].type'.

--
Best regards,
Andrey Tarasevich

[Adam Warner]

Hi Andrey Tarasevich,
On Wed, 29 Dec 2004 16:37:05 -0800, Andrey Tarasevich wrote:

On the second thought, the code should probably circumvent the "lvalue
check" since it actually accesses an element of an array aggregated by
the returned struct. Array element access in C decays to pointer
arithmetics, which will formally turn the '&'s argument into an lvalue
(i.e. 'l_SYM_2B(&a).o[0]' is '*(l_SYM_2B(&a) .o + 0)' and the result of
'*' is always an lvalue).
I'm glad we agree that this part is legal C99.
However, the second problem - UB - is still there.

6.5.2.2/5: "If an attempt is made to modify the result of a function call
or to access it after the next sequence point, the behavior is undefined."

Therefore I have to copy the return value into a local variable so I can
pass it by reference to a function, thereby defeating the benefits of pass
by reference (which were to avoid copying heavyweight structures and
permit mutation of all objects via function calls).

My mistake is that I can't rely upon a return value remaining on the stack
until the _calling_ function exits (and when you think about it the stack
could blow up horribly otherwise!)

<http://madchat.org/osdevl/stack.html>

As you can see, the data still is on the stack, but once the pop
operation is completed, we consider that part of the data invalid.
Thus, the next push operation overwrites this data. But that's OK,
because we assume that after a pop operation, the data that's popped
off is considered garbage.

If you've ever made the error of returning a pointer to a local
variable or to a parameter that was passed by value and wondered why
the value stayed valid initially, but later on got corrupted, you
should now know the reason.

The data still stays on the garbage part of the stack until the next
push operation overwrites it (that's when the data gets corrupted).

My method, while supremely fast and apparently correct with simple
benchmarks, would have resulted in hideously difficult to track down
data corruption.

Thanks for enlightening me.

Regards,
Adam

[Adam Warner]

Hi Andrey Tarasevich,
On Wed, 29 Dec 2004 17:45:01 -0800, Andrey Tarasevich wrote:

Did you mean 'struct o arg[]'?
I did, sorry!
If yes, then you are wrong. The following two declarations

struct v1 print_aesthetic (struct o *arg)
struct v1 print_aesthetic (struct o arg[])

are exactly equivalent in C. In both cases you can use 'arg->type' and
'arg[0].type'.

That's great to know! Thanks for helping me appreciate the undefined
behaviour of calling an (at least not inlined) function with the address
of the return value of a nested function.

Regards,
Adam