By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
448,538 Members | 909 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 448,538 IT Pros & Developers. It's quick & easy.

traumatized by pointer casting

P: n/a
I was reading page 720 of unix network programming,
volume one, second edition. In this udp_write function
he does the following:

void udp_write(char *buf, <everything else omitted)

struct udpiphdr *ui;
struct ip *ip;

ip = (struct ip *) buf;
ui = (struct udpiphdr *) buf;

I couldn't believe what I was seeing at first.
When I checked the headers for their type definitions
one of them contained bit fields in the struct
and the other struct just contained some unsigned shorts.

A pointer to char is not guaranteed to be
properly aligned for a pointer to struct. In fact
I don't see how this could even work period, even on the
most perverse of systems. I'm right, right? This really
shouldn't work?
Nov 14 '05 #1
Share this Question
Share on Google+
20 Replies


P: n/a
On Fri, 9 Jul 2004, j0mbolar wrote:
A pointer to char is not guaranteed to be
properly aligned for a pointer to struct. In fact
I don't see how this could even work period, even on the
most perverse of systems. I'm right, right? This really
shouldn't work?


It's not guaranteed by any standard, but in practical terms,
assuming the same programming model, a pointer is a pointer
(at least as far as their size is concerned). That is, one
32-bit or 64-bit address is the same size as any other 32-bit
or 64-bit address.

--
Rich Teer, SCNA, SCSA

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-online.net
Nov 14 '05 #2

P: n/a
On 9 Jul 2004 17:45:54 -0700 j0******@engineer.com (j0mbolar) wrote:
A pointer to char is not guaranteed to be
properly aligned for a pointer to struct. In fact
I don't see how this could even work period, even on the
most perverse of systems. I'm right, right? This really
shouldn't work?


Right! How can this work?

For the folks who didn't get it:

char buf[100];
struct foo {
long a;
long b;
char c;
int d;
};
struct foo *foo_p = (struct foo *) buf;

has the problem that buf has byte-alignment, whereas struct foo has
long alignment.

foo_p->a is not guaranteed to be referenceable. On some arches it would
cause an unaligned read (slow), on some it just wouldn't work at all.

/fc
Nov 14 '05 #3

P: n/a
Rich Teer <ri*******@rite-group.com> writes:
On Fri, 9 Jul 2004, j0mbolar wrote:
A pointer to char is not guaranteed to be
properly aligned for a pointer to struct. In fact
I don't see how this could even work period, even on the
most perverse of systems. I'm right, right? This really
shouldn't work?


It's not guaranteed by any standard, but in practical terms,
assuming the same programming model, a pointer is a pointer
(at least as far as their size is concerned). That is, one
32-bit or 64-bit address is the same size as any other 32-bit
or 64-bit address.


No, not really. It's common for all pointer types to have the same
size and representation, but there are alignment requirements.
For example:

char buf[100];
char *ptr = buf + 1;

*((int*)ptr) = 42;

is likely to cause a trap on many systems.

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Nov 14 '05 #4

P: n/a
Frank Cusack <fc*****@fcusack.com> writes:
On 9 Jul 2004 17:45:54 -0700 j0******@engineer.com (j0mbolar) wrote:
A pointer to char is not guaranteed to be
properly aligned for a pointer to struct. In fact
I don't see how this could even work period, even on the
most perverse of systems. I'm right, right? This really
shouldn't work?


Right! How can this work?

For the folks who didn't get it:

char buf[100];
struct foo {
long a;
long b;
char c;
int d;
};
struct foo *foo_p = (struct foo *) buf;

has the problem that buf has byte-alignment, whereas struct foo has
long alignment.

foo_p->a is not guaranteed to be referenceable. On some arches it would
cause an unaligned read (slow), on some it just wouldn't work at all.


It's certainly true that foo_p isn't guaranteed to be aligned to any
boundary bigger than a byte. On the other hand, a declared array
object is likely to be word-aligned. (If you count on this, of
course, your program will fail at the most inconvenient possible
moment.)

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <*> <http://users.sdsc.edu/~kst>
We must do something. This is something. Therefore, we must do this.
Nov 14 '05 #5

P: n/a
Keith Thompson wrote:
Frank Cusack <fc*****@fcusack.com> writes:
On 9 Jul 2004 17:45:54 -0700 j0******@engineer.com (j0mbolar) wrote:
A pointer to char is not guaranteed to be
properly aligned for a pointer to struct. In fact
I don't see how this could even work period, even on the
most perverse of systems. I'm right, right? This really
shouldn't work?


Right! How can this work?

For the folks who didn't get it:

char buf[100];
struct foo {
long a;
long b;
char c;
int d;
};
struct foo *foo_p = (struct foo *) buf;

has the problem that buf has byte-alignment, whereas struct foo has
long alignment.

foo_p->a is not guaranteed to be referenceable. On some arches it would
cause an unaligned read (slow), on some it just wouldn't work at all.

It's certainly true that foo_p isn't guaranteed to be aligned to any
boundary bigger than a byte. On the other hand, a declared array
object is likely to be word-aligned. (If you count on this, of
course, your program will fail at the most inconvenient possible
moment.)


Now, you have confused me.

Why would an array be word-aligned (16 bits)? And a structure be
byte-aligned (8 bits)?

Of course, to avoid an argument, I am making the assumption that a byte
equals 8 bits. From a previous thread on this newsgroup, I know that is
not always the case, but let's just use it this once.

I reviewed the draft of C99. And I'm not sure what determines
the boundaries.

The definition from the draft reads:

alignment
requirement that objects of a particular type be located on
storage boundaries with addresses that are particular
multiples of a byte address

Thanks,

Brian


--
* Remove x's to send me mail *
Nov 14 '05 #6

P: n/a
On Fri, 09 Jul 2004 21:18:56 -0700, tweak <xb**********@cox.net> wrote
in comp.lang.c:
Keith Thompson wrote:
Frank Cusack <fc*****@fcusack.com> writes:
On 9 Jul 2004 17:45:54 -0700 j0******@engineer.com (j0mbolar) wrote:

A pointer to char is not guaranteed to be
properly aligned for a pointer to struct. In fact
I don't see how this could even work period, even on the
most perverse of systems. I'm right, right? This really
shouldn't work?

Right! How can this work?

For the folks who didn't get it:

char buf[100];
struct foo {
long a;
long b;
char c;
int d;
};
struct foo *foo_p = (struct foo *) buf;

has the problem that buf has byte-alignment, whereas struct foo has
long alignment.

foo_p->a is not guaranteed to be referenceable. On some arches it would
cause an unaligned read (slow), on some it just wouldn't work at all.

It's certainly true that foo_p isn't guaranteed to be aligned to any
boundary bigger than a byte. On the other hand, a declared array
object is likely to be word-aligned. (If you count on this, of
course, your program will fail at the most inconvenient possible
moment.)


Now, you have confused me.


You are misinterpreting what Keith wrote.
Why would an array be word-aligned (16 bits)? And a structure be
byte-aligned (8 bits)?
The array might not be word-aligned. It might have an odd address.
When you cast the array by name to a pointer to struct, it would
retain that odd address.

On an implementation for a processor, say something widely used like
an ARM that requires that longs (which do contain 4 8-bit bytes) be
aligned on an address that is evenly divisible by 4. A real struct
foo would start on an address evenly divisible by 4.

So if you have a pointer to foo with an odd address, and you try to
read or write to foo_p->a (a long) at that odd address, the processor
will not slow down. Instead it will generate a hardware trap.
Of course, to avoid an argument, I am making the assumption that a byte
equals 8 bits. From a previous thread on this newsgroup, I know that is
not always the case, but let's just use it this once.

I reviewed the draft of C99. And I'm not sure what determines
the boundaries.
The implementation determines the boundaries. They must conform to
what is physically possible with the underlying processor hardware,
but they might be more strict than the hardware requires. There are
quite a few modern RISC and DSP processors that have a general
requirement that types larger than one byte be aligned on addresses
that are a multiple of their size. They don't waste the extra
transistors on automatic circuitry to make slower access to unaligned
memory, they use them for more useful things and generate a fault or
just plain read or write the wrong memory instead.
The definition from the draft reads:

alignment
requirement that objects of a particular type be located on
storage boundaries with addresses that are particular
multiples of a byte address

Thanks,

Brian


--
Jack Klein
Home: http://JK-Technology.Com
FAQs for
comp.lang.c http://www.eskimo.com/~scs/C-faq/top.html
comp.lang.c++ http://www.parashift.com/c++-faq-lite/
alt.comp.lang.learn.c-c++
http://www.contrib.andrew.cmu.edu/~a...FAQ-acllc.html
Nov 14 '05 #7

P: n/a
>On Fri, 09 Jul 2004 21:18:56 -0700, tweak <xb**********@cox.net> wrote
in comp.lang.c:
Why would an array be word-aligned (16 bits)? And a structure be
byte-aligned (8 bits)?


A quick question; but isn't a word's size processor dependant?
For example, a 32-bit Intel's word size is 32 bits, while a
64-bit SPARC's word is 64 bits?

--
Eric Enright /"\
ericAtiptsoftDcom \ / ASCII Ribbon Campaign
X Against HTML E-Mail
Public Key: 0xBEDF636F / \
Nov 14 '05 #8

P: n/a
Jack Klein wrote:
On Fri, 09 Jul 2004 21:18:56 -0700, tweak <xb**********@cox.net> wrote
in comp.lang.c:

Keith Thompson wrote:
Frank Cusack <fc*****@fcusack.com> writes:
On 9 Jul 2004 17:45:54 -0700 j0******@engineer.com (j0mbolar) wrote:
>A pointer to char is not guaranteed to be
>properly aligned for a pointer to struct. In fact
>I don't see how this could even work period, even on the
>most perverse of systems. I'm right, right? This really
>shouldn't work?

Right! How can this work?

For the folks who didn't get it:

char buf[100];
struct foo {
long a;
long b;
char c;
int d;
};
struct foo *foo_p = (struct foo *) buf;

has the problem that buf has byte-alignment, whereas struct foo has
long alignment.

foo_p->a is not guaranteed to be referenceable. On some arches it would
cause an unaligned read (slow), on some it just wouldn't work at all.
It's certainly true that foo_p isn't guaranteed to be aligned to any
boundary bigger than a byte. On the other hand, a declared array
object is likely to be word-aligned. (If you count on this, of1
course, your program will fail at the most inconvenient possible
moment.)


Now, you have confused me.

You are misinterpreting what Keith wrote.

Why would an array be word-aligned (16 bits)? And a structure be
byte-aligned (8 bits)?

The array might not be word-aligned. It might have an odd address.
When you cast the array by name to a pointer to struct, it would
retain that odd address.


it == address of first element in the array?

Let me try to word how I am understanding what you are saying. The
addresses within the struct mentioned above occupy memory addresses
determined by their type. For example

a b c d
[4 bytes] [4 bytes] [1 byte] [2 bytes]

In theory, the address pointed to would be the first item in the
structure, right? Thus, the first item in the structure will always
have an address divisible by 4. And the first item in a structure
determines the alignment of the structure?

So to guarantee that the typecast will work, the buf (array) size has to
have the same divisibility as the first item in the structure?
Now, the array buf contains 100 bytes each of type char, so the location
in memory of the first element in the array can have an odd address
(address divisible by 1).

So when buf is typecasted to struct foo *, the first item pointed to
long a may not be aligned correctly since &buf[0] can have an odd
address? Am I following what you are saying okay?

I reviewed the draft of C99. And I'm not sure what determines
the boundaries.

The implementation determines the boundaries. They must conform to
what is physically possible with the underlying processor hardware,
but they might be more strict than the hardware requires. There are
quite a few modern RISC and DSP processors that have a general
requirement that types larger than one byte be aligned on addresses
that are a multiple of their size. They don't waste the extra
transistors on automatic circuitry to make slower access to unaligned
memory, they use them for more useful things and generate a fault or
just plain read or write the wrong memory instead.


So char can have odd addresses, int can have addresses divisible by 2,
long can have addresses divisible by 4 and so on?

I will write a program tomorrow with intentional mis-alignment, so that
I can debug it and see the mis-alignment. I hope I can do this with
gdb.

Brian

P.S. The FAQ is brilliant. I wish I knew C as well as you.

--
* Remove x's to send me mail *
Nov 14 '05 #9

P: n/a

"Eric Enright" <er**@tiptsoft.com> wrote in message
Why would an array be word-aligned (16 bits)? And a structure be
byte-aligned (8 bits)?


A quick question; but isn't a word's size processor dependant?
For example, a 32-bit Intel's word size is 32 bits, while a
64-bit SPARC's word is 64 bits?

The underlying hardware reads memory in chunks. The smallest chunk of memory
that can be read in one go is a "byte", the largest can be called the "word
size".
Since efficency is very important to C programmers, usually compilers
writers make the size of common types, such as int, match up with the word
size. However often there is a choice - for instance a processor may have
instructions that read 32-bit or a 64-bit word equally fast.

On some architectures, but not all, word access has to be aligned on memory
boundaries. On other architectures access to an aligned word is faster. If
you declare a structure containing bit fields, or many chars, a compiler
might very well decide to align it less strictly than an array. For instnace
bitfield access is so slow that there is no point speeding up the memory
access by inserting padding to enforce a stricter alignment.
Nov 14 '05 #10

P: n/a
Malcolm wrote:

"Eric Enright" <er**@tiptsoft.com> wrote in message
>> Why would an array be word-aligned (16 bits)? And a structure be
>> byte-aligned (8 bits)?
A quick question; but isn't a word's size processor dependant?
For example, a 32-bit Intel's word size is 32 bits, while a
64-bit SPARC's word is 64 bits?

The underlying hardware reads memory in chunks. The smallest chunk of memory
that can be read in one go is a "byte", the largest can be called the "word
size".
Since efficency is very important to C programmers, usually compilers
writers make the size of common types, such as int, match up with the word
size


I see, this nicely illuminates to me why much of the assembly
code I have looked at uses the .align directive (my
understanding of that was always fairly sketchy).
However often there is a choice - for instance a processor may have
instructions that read 32-bit or a 64-bit word equally fast.
Which is a key feature in backwards compatability with legacy
applications (in this case, 32-bit binaries on a 64-bit
processor), and performance, I take it.
On some architectures, but not all, word access has to be aligned on memory
boundaries. On other architectures access to an aligned word is faster. If
you declare a structure containing bit fields, or many chars, a compiler
might very well decide to align it less strictly than an array. For instnace
bitfield access is so slow that there is no point speeding up the memory
access by inserting padding to enforce a stricter alignment.


Yes, I can imagine on architectures which do not require word
access to be aligned, what might be a single instruction to
fetch a word would end up being two or more!

Thank you for your enlightening article Malcolm.

I can only hope that in the future I get to play with a more
diverse set of processors than the Intel and (little bit of)
Alpha that I do now.

--
Eric Enright /"\
ericAtiptsoftDcom \ / ASCII Ribbon Campaign
X Against HTML E-Mail
Public Key: 0xBEDF636F / \
Nov 14 '05 #11

P: n/a

"j0mbolar" <j0******@engineer.com> wrote in message
news:2d**************************@posting.google.c om...
I was reading page 720 of unix network programming,
volume one, second edition. In this udp_write function
he does the following:

void udp_write(char *buf, <everything else omitted)

struct udpiphdr *ui;
struct ip *ip;

ip = (struct ip *) buf;
ui = (struct udpiphdr *) buf;

I couldn't believe what I was seeing at first.
When I checked the headers for their type definitions
one of them contained bit fields in the struct
and the other struct just contained some unsigned shorts.
Right, that's because a UDP packet is guaranteed to have a specific
layout at the bit/byte level.
A pointer to char is not guaranteed to be
properly aligned for a pointer to struct.
Nevertheless, there is in fact a UDP packet beginning at that address,
and it does in fact have the layout. This is not just any pointer to char,
it's a pointer that he probably got back from a function like 'malloc',
which means it's guaranteed to be adequately aligned to point to any type.
In fact
I don't see how this could even work period, even on the
most perverse of systems. I'm right, right? This really
shouldn't work?


It will work on most systems. However, it could break on systems where
the buffer is not sufficiently aligned. I'm betting his code made sure the
buffer is sufficiently aligned.

DS
Nov 14 '05 #12

P: n/a
On 9 Jul 2004 17:45:54 -0700, j0******@engineer.com (j0mbolar) wrote:
I was reading page 720 of unix network programming,
volume one, second edition. In this udp_write function
he does the following:

void udp_write(char *buf, <everything else omitted)

struct udpiphdr *ui;
struct ip *ip;

ip = (struct ip *) buf;
ui = (struct udpiphdr *) buf;

I couldn't believe what I was seeing at first.
When I checked the headers for their type definitions
one of them contained bit fields in the struct
and the other struct just contained some unsigned shorts.

A pointer to char is not guaranteed to be
properly aligned for a pointer to struct. In fact
I don't see how this could even work period, even on the
most perverse of systems. I'm right, right? This really
shouldn't work?


It will work on any platform which doesn't enforce alighment
requirements, and on any platform at all provided the pointer passed
is suitably aligned, as I imagine the book's code will do.

If you really want to ensure it's always aligned, add a wrapper
something like this:

void a_udp_write(char *buf,size_t size,...)
{
char *abuf,*tbuf=buf;
int a=0;

if ((buf&15)!=0)
{
a++;
abuf=(char*)malloc(size+16);
tbuf=abuf & ~15;
memcpy(tbuf,buf,size);
}
udp_write(tbuf,size,...);
if (a!=0)
free(abuf);
}

Then you can be certain udp_write() is always working with an aligned
buffer (on a 16 byte boundary here), even if the caller didn't.
James.
Nov 14 '05 #13

P: n/a

On Sat, 10 Jul 2004, Eric Enright wrote:
On Fri, 09 Jul 2004 21:18:56 -0700, tweak <xb**********@cox.net> wrote
in comp.lang.c:
Why would an array be word-aligned (16 bits)? And a structure be
byte-aligned (8 bits)?


A quick question; but isn't a word's size processor dependant?
For example, a 32-bit Intel's word size is 32 bits, while a
64-bit SPARC's word is 64 bits?


Yes, this is the "general" meaning of the word 'word' in computer
architecture AIUI. But in the [W]intel world, for hysterical reasons,
'word' often means specifically 16 bits (two bytes), no matter
what the "word size" of the processor in question is. For example,
the 32-bit integer type supplied by many Microsoft languages is
called 'DWORD', short for "double word," even though those languages
usually run on platforms where a 'machine word' is 32 bits!

The Intel hierarchy goes: byte(8), word(16), dword(32), qword(64)...
and has very little to do with the actual "word size" involved on
any modern machine.

HTH,
-Arthur
Nov 14 '05 #14

P: n/a
j0mbolar wrote:
A pointer to char is not guaranteed to be
properly aligned for a pointer to struct. In fact
I don't see how this could even work period, even on the
most perverse of systems. I'm right, right? This really
shouldn't work?


Yes but, we're talking platform-specific stuff here. This sort of thing
is so common in UNIX socket programs, that there just aren't likely to
be implementations where it doesn't work. It would destroy too much
legacy code.

The moral, don't look to books like that for ISO standard C. Look to it
as one of the finest tutorials in network programming around. Trust that
the code he has in there has been used and abused for many a year.


Brian Rodenborn
Nov 14 '05 #15

P: n/a
On Sat, 10 Jul 2004 00:06:30 -0700, tweak <xb**********@cox.net> wrote
in comp.lang.c:
Jack Klein wrote:
On Fri, 09 Jul 2004 21:18:56 -0700, tweak <xb**********@cox.net> wrote
in comp.lang.c:

Keith Thompson wrote:

Frank Cusack <fc*****@fcusack.com> writes:
>On 9 Jul 2004 17:45:54 -0700 j0******@engineer.com (j0mbolar) wrote:
>
>
>>A pointer to char is not guaranteed to be
>>properly aligned for a pointer to struct. In fact
>>I don't see how this could even work period, even on the
>>most perverse of systems. I'm right, right? This really
>>shouldn't work?
>
>Right! How can this work?
>
>For the folks who didn't get it:
>
> char buf[100];
> struct foo {
> long a;
> long b;
> char c;
> int d;
> };
> struct foo *foo_p = (struct foo *) buf;
>
>has the problem that buf has byte-alignment, whereas struct foo has
>long alignment.
>
>foo_p->a is not guaranteed to be referenceable. On some arches it would
>cause an unaligned read (slow), on some it just wouldn't work at all.
It's certainly true that foo_p isn't guaranteed to be aligned to any
boundary bigger than a byte. On the other hand, a declared array
object is likely to be word-aligned. (If you count on this, of1
course, your program will fail at the most inconvenient possible
moment.)
Now, you have confused me.

You are misinterpreting what Keith wrote.

Why would an array be word-aligned (16 bits)? And a structure be
byte-aligned (8 bits)?

The array might not be word-aligned. It might have an odd address.
When you cast the array by name to a pointer to struct, it would
retain that odd address.


it == address of first element in the array?


Yes.
Let me try to word how I am understanding what you are saying. The
addresses within the struct mentioned above occupy memory addresses
determined by their type. For example

a b c d
[4 bytes] [4 bytes] [1 byte] [2 bytes]
Actually there is a good chance that internal padding would be added
between the members of the structure, which C allows for just this
reason. To keep the discussion simple, assume a platform with 8-bit
bytes, 2-byte (16-bit) ints, and 4-byte (32-bit) longs. If it has
access requirements, that 2-byte int member d will need to be on an
even address. So assuming you define one of these structures and its
address is 0x1000 hex, the compiler will probably allocate it like
this:

0x1000 - 0x1003 member a 4 bytes
0x1004 - 0x1007 member b 4 bytes
0x1008 - 0x1008 member c 1 byte
0x1009 - 0x1009 padding 1 byte
0x100A - 0x100B member d 2 bytes

The padding byte is required because d must start at an even address.
C allows the compiler to insert padding bytes between the members of a
structure, and after the last member, just not before the first
member. Padding is sometimes necessary at the end to make the size of
the entire structure a multiple of the alignment size.

Consider a processor that requires 4-byte, 32-bit longs be located on
an address divisible by 4. Now consider this structure definition and
how the compiler will pad it:

struct silly {
char c1; /* relative address 0x0000 */
/* padding 3 bytes 0x0001 - 0x0003 */
long l1; /* relative address 0x0004 - 0x0007 */
char c2; /* relative address 0x0008 */
/* padding 3 bytes 0x0009 - 0x000b */
};
In theory, the address pointed to would be the first item in the
structure, right? Thus, the first item in the structure will always
have an address divisible by 4. And the first item in a structure
determines the alignment of the structure?
No, the member of the structure with the strictest alignment
requirement determines the alignment requirement of the entire
structure, as I tried to show in struct silly above. The structure
must have an alignment at least as strict as its strictest member, and
must have padding at the end, if necessary, to make the size of the
entire structure a multiple of this alignment size.
So to guarantee that the typecast will work, the buf (array) size has to
have the same divisibility as the first item in the structure?
Yes. That is why the memory allocation functions malloc(), calloc(),
and realloc() are guaranteed to return pointers to memory blocks
suitable aligned for any data type.

Now, the array buf contains 100 bytes each of type char, so the location
in memory of the first element in the array can have an odd address
(address divisible by 1).

So when buf is typecasted to struct foo *, the first item pointed to
long a may not be aligned correctly since &buf[0] can have an odd
address? Am I following what you are saying okay?


Yes.
I reviewed the draft of C99. And I'm not sure what determines
the boundaries.

The implementation determines the boundaries. They must conform to
what is physically possible with the underlying processor hardware,
but they might be more strict than the hardware requires. There are
quite a few modern RISC and DSP processors that have a general
requirement that types larger than one byte be aligned on addresses
that are a multiple of their size. They don't waste the extra
transistors on automatic circuitry to make slower access to unaligned
memory, they use them for more useful things and generate a fault or
just plain read or write the wrong memory instead.


So char can have odd addresses, int can have addresses divisible by 2,
long can have addresses divisible by 4 and so on?

I will write a program tomorrow with intentional mis-alignment, so that
I can debug it and see the mis-alignment. I hope I can do this with
gdb.

Brian

P.S. The FAQ is brilliant. I wish I knew C as well as you.


What happens when you violate alignment requirements falls into the
category that the C standard calls undefined behavior. It is like
dividing by 0 in mathematics, you have broken the rules and there is
no correct answer.

In the actual hardware world there are three likely things that will
happen, depending on the type of the underlying processor hardware:

1. Example Intel Pentium. The processor will perform multiple memory
accesses, if necessary, and shift the bytes around between memory and
the processor core. The performance penalty can be quite severe.

2. Example ARM. The processor hardware generates a hardware
exception that causes the operating system (if there is one) to
terminate the misbehaving program.

3. Example Intel 8096. The misalignment is ignored, with incorrect
results. If you have a pointer of 0x21 and try to read or write a
16-bit int at address 0x21 and 0x22, the processor actually reads or
writes at the aligned address, bytes 0x20 and 0x21.

--
Jack Klein
Home: http://JK-Technology.Com
FAQs for
comp.lang.c http://www.eskimo.com/~scs/C-faq/top.html
comp.lang.c++ http://www.parashift.com/c++-faq-lite/
alt.comp.lang.learn.c-c++
http://www.contrib.andrew.cmu.edu/~a...FAQ-acllc.html
Nov 14 '05 #16

P: n/a
Jack Klein wrote:
[snip]
Let me try to word how I am understanding what you are saying. The
addresses within the struct mentioned above occupy memory addresses
determined by their type. For example

a b c d
[4 bytes] [4 bytes] [1 byte] [2 bytes]

Actually there is a good chance that internal padding would be added
between the members of the structure, which C allows for just this
reason. To keep the discussion simple, assume a platform with 8-bit
bytes, 2-byte (16-bit) ints, and 4-byte (32-bit) longs. If it has
access requirements, that 2-byte int member d will need to be on an
even address. So assuming you define one of these structures and its
address is 0x1000 hex, the compiler will probably allocate it like
this:

0x1000 - 0x1003 member a 4 bytes
0x1004 - 0x1007 member b 4 bytes
0x1008 - 0x1008 member c 1 byte
0x1009 - 0x1009 padding 1 byte
0x100A - 0x100B member d 2 bytes

The padding byte is required because d must start at an even address.
C allows the compiler to insert padding bytes between the members of a
structure, and after the last member, just not before the first
member. Padding is sometimes necessary at the end to make the size of
the entire structure a multiple of the alignment size.


So is it better to design the structure from the start with padding
or to let the compiler put it there? I would think that later would
make the code more portable.

Consider a processor that requires 4-byte, 32-bit longs be located on
an address divisible by 4. Now consider this structure definition and
how the compiler will pad it:

struct silly {
char c1; /* relative address 0x0000 */
/* padding 3 bytes 0x0001 - 0x0003 */
long l1; /* relative address 0x0004 - 0x0007 */
char c2; /* relative address 0x0008 */
/* padding 3 bytes 0x0009 - 0x000b */
};
So to write good code I need to make sure that when I typecast pointers
between two different types (e.g., structures and arrays), that the
starting point of each type resides in memory based upon the strictest
member of the two?

This should help me avoid having to deal with undefined behavior
exploits in my software.

[snip] What happens when you violate alignment requirements falls into the
category that the C standard calls undefined behavior. It is like
dividing by 0 in mathematics, you have broken the rules and there is
no correct answer.
But the un-alignment remains in memory and the implementation (OS and/or
architecture) will have to determine how to handle the undefined behavior?

In the actual hardware world there are three likely things that will
happen, depending on the type of the underlying processor hardware:

1. Example Intel Pentium. The processor will perform multiple memory
accesses, if necessary, and shift the bytes around between memory and
the processor core. The performance penalty can be quite severe.


Since this is the platform I am dealing with, when I step through the
code (I still need to write it tonight), I should see everything
aligned correctly, but I should suffer a penality due to the multiple
memory accesses to fix the alignment?

How do you catch these errors in large projects like web browsers if the
processor fixes them for you in real time and they pass the compiler?

I am writing some network software right now, and I'm making corrections
non-stop as folks here point out things I am doing wrong. The realm
of undefined behavior that is not caught by the compiler can put serious
security problems in software.

Thanks for taking me through this lesson.

Brian

P.S. I took the same C classes that the CS majors in my university took,
and these items never came up. Is there alot of code floating
around with undefined behavior not caught by compilers?
--
* Remove x's to send me mail *
Nov 14 '05 #17

P: n/a
tweak <xb**********@cox.net> wrote:
Jack Klein wrote:
address is 0x1000 hex, the compiler will probably allocate it like
this:

0x1000 - 0x1003 member a 4 bytes
0x1004 - 0x1007 member b 4 bytes
0x1008 - 0x1008 member c 1 byte
0x1009 - 0x1009 padding 1 byte
0x100A - 0x100B member d 2 bytes

The padding byte is required because d must start at an even address.
C allows the compiler to insert padding bytes between the members of a
structure, and after the last member, just not before the first
member. Padding is sometimes necessary at the end to make the size of
the entire structure a multiple of the alignment size.
So is it better to design the structure from the start with padding
or to let the compiler put it there? I would think that later would
make the code more portable.
You can't really "design" with padding because, unless you only want
to run this program on some specific hardware and you exactly know
the alignment requirements on that architecture, you will have no
knowledge of how much padding you're going to need in all possible
cases (and the compiler is still entitled to add some more). The
compiler will automatically insert as much padding as it thinks is
needed (unless you explicitely tell it not to, at least some com-
pilers allow you to do that). The only thing you can do portably is
to try to avoid having e.g. an int follow an uneven number of chars,
so instead of defining a structure like this

stuct A {
char c1;
int i1;
char c2;
int i2;
};

you might end up with a shorter structure when you define it as

struct B {
int i1;
int i2;
char c1;
char c2;
};

but that's probably only getting interesting if you're creating huge
arrays of such structures (and if sizeof(char) != sizeof(int))...
Consider a processor that requires 4-byte, 32-bit longs be located on
an address divisible by 4. Now consider this structure definition and
how the compiler will pad it:

struct silly {
char c1; /* relative address 0x0000 */
/* padding 3 bytes 0x0001 - 0x0003 */
long l1; /* relative address 0x0004 - 0x0007 */
char c2; /* relative address 0x0008 */
/* padding 3 bytes 0x0009 - 0x000b */
}; So to write good code I need to make sure that when I typecast pointers
between two different types (e.g., structures and arrays), that the
starting point of each type resides in memory based upon the strictest
member of the two?
Typically, you should never need to cast between pointers of different,
unrelated types - it often shows that you have done something wrong.
But when you have to your statement is correct. So when you see code
like this

int x = * ( int * ) ( char * ) buf + 3 );

you better get very suspicious of it, quite often it was written by
someone not aware of alignment issues and trying to cut corners, with
the net effect that it only runs on certain architectures and there
possibly even slow.

Where things are unproblematic is when you convert from a pointer
you got from malloc() because addresses returned by malloc() always
are aligned for the strictest requirements on the machine. So e.g.

char *buf = malloc( 100 );
struct xyz *p = buf;

can't get you in trouble because of alignment issues.
What happens when you violate alignment requirements falls into the
category that the C standard calls undefined behavior. It is like
dividing by 0 in mathematics, you have broken the rules and there is
no correct answer.

But the un-alignment remains in memory and the implementation (OS and/or
architecture) will have to determine how to handle the undefined behavior?


It's going to do something, of course. Undefined behavior just
means that it's not defined by the C standard what's going to
happpen. So the code has a chance to run seemingly flawless on
your machine or it could result in e.g. a bus error or something
completely different. If you're targeting your code to a single
architecture only and a certain compiler you may get away with it.
But when you later want to port the program to a different archi-
tecture/OS/compiler combination you're likely to find out about
the errors of your ways;-)

Things of course are different when you write e.g. a device driver
for a certain architecture. These are places where invoking unde-
fined behaviour sometimes is even necessary to get the work done.
But then you are supposed to know exactly how the machine deals with
the type of undefined behaviour you're invoking. In that case it's
undefined behaviour in the C sense, but it's well defined what the
machine is going to do.
Regards, Jens
--
\ Jens Thoms Toerring ___ Je***********@physik.fu-berlin.de
\__________________________ http://www.toerring.de
Nov 14 '05 #18

P: n/a
tweak wrote:
Since this is the platform I am dealing with, when I step through the
code (I still need to write it tonight), I should see everything
aligned correctly, but I should suffer a penality due to the multiple
memory accesses to fix the alignment?

How do you catch these errors in large projects like web browsers if the
processor fixes them for you in real time and they pass the compiler?

I am writing some network software right now, and I'm making corrections
non-stop as folks here point out things I am doing wrong. The realm
of undefined behavior that is not caught by the compiler can put serious
security problems in software.

Thanks for taking me through this lesson.

Brian

P.S. I took the same C classes that the CS majors in my university took,
and these items never came up. Is there alot of code floating
around with undefined behavior not caught by compilers?


Just remember that structures should not be used to input or
output data to the physical world. This is largely due to the padding
issue between fields in the structure (and maybe at the end).

The best method is to declare a structure to "model" the data within
the program and fill the fields individually from the physical
source (or output to the physical source). This will also help
ensure better portability when taking legacy code and porting to
a different platform especially when the other platform has different
native integer bit widths (e.g. converting 16-bit code to 32-bit
processor).

I've seen may a code that tries to patch the I/O from the physical
source by inserting padding bytes between the structures. This
still causes problems. Other resolutions include designing the
physical format for better alignment (such as one char followed
by a second char so that the next int field is aligned). Again,
this is platform dependent and its advantages turn into headaches
when the platform changes.

From my experience, the best method is to limit the I/O to the
physical source to as few places as possible (preferably one).
This reduces the locations that need to be modified when the
code is ported (which in most cases it will; rarely is code
ever rewritten from scrath). Also, dealing with a structure
inside the program makes the program faster. One doesn't have
to keep worrying about extracting 16 bits out of 32 or converting
from little endian to big endian. So convert the data onece as
it is input and once again before it is output.

--
Thomas Matthews

C++ newsgroup welcome message:
http://www.slack.net/~shiva/welcome.txt
C++ Faq: http://www.parashift.com/c++-faq-lite
C Faq: http://www.eskimo.com/~scs/c-faq/top.html
alt.comp.lang.learn.c-c++ faq:
http://www.raos.demon.uk/acllc-c++/faq.html
Other sites:
http://www.josuttis.com -- C++ STL Library book
http://www.sgi.com/tech/stl -- Standard Template Library

Nov 14 '05 #19

P: n/a
In <2d**************************@posting.google.com > j0******@engineer.com (j0mbolar) writes:
I was reading page 720 of unix network programming,
volume one, second edition. In this udp_write function
he does the following:

void udp_write(char *buf, <everything else omitted)

struct udpiphdr *ui;
struct ip *ip;

ip = (struct ip *) buf;
ui = (struct udpiphdr *) buf;

I couldn't believe what I was seeing at first.
When I checked the headers for their type definitions
one of them contained bit fields in the struct
and the other struct just contained some unsigned shorts.

A pointer to char is not guaranteed to be
properly aligned for a pointer to struct. In fact
I don't see how this could even work period, even on the
most perverse of systems. I'm right, right? This really
shouldn't work?


It is the caller's job to ensure that the first argument of udp_write()
is properly aligned for these struct's. If dynamical allocation is used,
this condition is trivially satisfied.

Dan
--
Dan Pop
DESY Zeuthen, RZ group
Email: Da*****@ifh.de
Nov 14 '05 #20

P: n/a
On Sat, 10 Jul 2004 16:49:55 +0100, James <ja*@spamcop.net> wrote:
On 9 Jul 2004 17:45:54 -0700, j0******@engineer.com (j0mbolar) wrote:
[Stevens v1 2e p720]
void udp_write(char *buf, <everything else omitted)

struct udpiphdr *ui;
struct ip *ip;

ip = (struct ip *) buf;
ui = (struct udpiphdr *) buf;
It will work on any platform which doesn't enforce alighment
requirements, and on any platform at all provided the pointer passed
is suitably aligned, as I imagine the book's code will do.

If you really want to ensure it's always aligned, add a wrapper
something like this:

void a_udp_write(char *buf,size_t size,...)
The parameter could better be const char *, but back in the '80s that
wasn't reliably portable, and before that didn't exist at all.
{
char *abuf,*tbuf=buf;
int a=0;

if ((buf&15)!=0)
That requires an explicit conversion (cast) -- and then formally as
far as C is concerned the result is implementation-defined, although
it should be correct on any Unix and probably any platform where
BSD-ish sockets make sense at all.
{
a++;
abuf=(char*)malloc(size+16);
Cast not needed in C if malloc is correctly declared, as it must be,
preferably by #include'ing <stdlib.h>; (clc) FAQ 7.6. Should check the
return from malloc() etc. for nonnull before using.
tbuf=abuf & ~15;
Not needed; the value returned by {m,c,re}alloc, if successful, must
be sufficiently aligned for any object type (including struct of
anything) on the implementation. And if it was needed it would be
backwards; you would need to round *up*, into the space allocated.

In general there is no way to automatically determine what the
strictest alignment is, although you can get an upper bound for any
predetermined (finite) set of types by something like
offsetof ( struct { char a; union { long l; double d; } u } , u ).
For IPv4+UDP in particular, we know that no header field requires more
than 4-byte alignment.
memcpy(tbuf,buf,size);
}
udp_write(tbuf,size,...);
if (a!=0)
free(abuf);
Instead of a separate flag variable a, why not just initialize abuf to
NULL, and then unconditionally free() it -- if (still) null it does
nothing, and if non-null it frees the malloc'ed space.

Or, since UDP is lossy anyway, could just do
if( (ulong)buf & 15 ) return;
or to be "nice" about it
... { errno = EPULLTHEOTHERONE; return; }
:-)
}

Then you can be certain udp_write() is always working with an aligned
buffer (on a 16 byte boundary here), even if the caller didn't.
James.


- David.Thompson1 at worldnet.att.net
Nov 14 '05 #21

This discussion thread is closed

Replies have been disabled for this discussion.