Seting unsigned long value causes coredump

[bryan]

I think I'm missing something fundamental here. I'm trying to set an
unsigned long value via a u_long pointer, however it coredumps
everytime I get to that instruction. Here is a sample program that
demonstrates the issue:

--- snip ---
#include <unistd.h>

int main() {
char buf[512];
u_char *char_p;
u_long *long_p;

memset(buf, 0x0, sizeof(buf));
memset(buf, 'A', 50);

char_p=buf;
long_p = (unsigned long *) (char_p+=strlen(buf));

*long_p = 0xffaa00dd;
}
--- snip ---

I've even debuged it right up to the store call:

--- session ---
[sundev2: user]~$ gdb blah
GNU gdb 6.0
Copyright 2003 Free Software Foundation, Inc.
....
(gdb) disassemble main
Dump of assembler code for function main:
....
0x00010708 <main+104>: sethi %hi(0xffaa0000), %g1
0x0001070c <main+108>: or %g1, 0xdd, %g1 ! 0xffaa00dd
0x00010710 <main+112>: st %g1, [ %o5 ]
0x00010714 <main+116>: mov %g1, %i0
0x00010718 <main+120>: ret
0x0001071c <main+124>: restore
End of assembler dump.
(gdb) break *0x00010710
Breakpoint 1 at 0x10710: file blah.c, line 14.
(gdb) run
Starting program: /export/home/s182301/labroom/test/cde/blah

Breakpoint 1, 0x00010710 in main () at blah.c:14
14 *long_p = 0xffaa00dd;
(gdb) info register g1 o5
g1 0xffaa00dd -5635875
o5 0xffbffaba -4195654
(gdb) x/2wx 0xffbffaba - 4
0xffbffab6: 0x41414141 0x00000000
(gdb) stepi

Program received signal SIGSEGV, Segmentation fault.
0x00010710 in main () at blah.c:14
14 *long_p = 0xffaa00dd;
(gdb)
--- session ---

So, I understand that "st %g1, [ %o5 ]" will move the contents of %g1
into the address of %o5. So, to make sure i'm at the right location,
I examine the word at and before %o5 (0xffbffaba), and it looks like
i'm in the right location, immediatly after my memset 'A's. But the
instruction causes a SIGSEGV, which I really dont understand.

For your information, I'm running this on an UltraSparc I, using
Solaris 9, gcc 3.3.2, and latest patches.

Thank you in advanced for any assistance you can provide.
Bryan.

[Eric Sosman]

bryan wrote:

I think I'm missing something fundamental here. I'm trying to set an
unsigned long value via a u_long pointer, however it coredumps
everytime I get to that instruction. Here is a sample program that
demonstrates the issue:

--- snip ---
#include <unistd.h>

int main() {
char buf[512];
u_char *char_p;
u_long *long_p;

memset(buf, 0x0, sizeof(buf));
memset(buf, 'A', 50);

char_p=buf;
long_p = (unsigned long *) (char_p+=strlen(buf));

*long_p = 0xffaa00dd;
}

The fundamental thing you've missed is the magic
word "alignment." Some machines will permit a `long'
to begin at any arbitrary address, but others require
that it begin at an address that's a multiple of four,
or eight, or something of the sort. (One can deduce
that the required alignment for an object is a divisor
of the object's size, but that's another topic.)

It appears that the machine you're using is of the
latter type: a `long' can't start "just anywhere," but only
at an address that satisfies a divisibility constraint.
It further appears that the value you generate for `long_p'
(fifty bytes past the start of `buf', whose own alignment
is not known) doesn't meet this constraint -- boom!

Legalistically speaking, anything at all can happen
once you've gone off the rails this way. In practice,
I've come across three different behaviors:

- The program crashes.

- The program works, but more slowly than if the
pointer had been properly aligned.

- The program runs, but the value is stored "near"
rather than "at" the place you intended.

Now to the big question: What are you trying to do?
Explain your goal, and perhaps someone can chart a path
to it that doesn't take you through the Circles of Hell.

--
Er*********@sun.com

[Martin Dickopp]

br***@bueterfamily.org (bryan) writes:

I think I'm missing something fundamental here. I'm trying to set an
unsigned long value via a u_long pointer, however it coredumps
everytime I get to that instruction. Here is a sample program that
demonstrates the issue:

--- snip ---
#include <unistd.h>
Not a standard C header (and not needed by your code). Include
<string.h> instead to get declarations for `memset' and `strlen'.
int main() {
char buf[512];
u_char *char_p;
u_long *long_p;
`u_char' and `u_long' are not standard C type. I'll assume they
correspond to `unsigned char' and `unsigned long', respectively.
memset(buf, 0x0, sizeof(buf));
memset(buf, 'A', 50);

char_p=buf;
long_p = (unsigned long *) (char_p+=strlen(buf));

*long_p = 0xffaa00dd;
}

If you convert a pointer to an object type to a pointer to another
object type, the resulting pointer is not guaranteed to be correctly
aligned. Therefore, your cast causes undefined behavior, i.e. anything
could happen.

What are you really trying to achieve? If you want to serialize some
data, copy it bytewise (which is referred to as accessing the
"representation" of the data):

unsigned long data = 0xffaa00dd;
/* ... */
memcpy (char_p += strlen (buf), &data, sizeof data);

If the contents of the buffer are potentially read on another machine
later, use the /value/ (as opposed to the /representation/) of the data:

/* The following assumes 4 byte longs and 8 bit bytes. */
*char_p++ = (data >> 24) & 0xFF;
*char_p++ = (data >> 16) & 0xFF;
*char_p++ = (data >> 8) & 0xFF;
*char_p++ = data & 0xFF;

Martin
--
,--. Martin Dickopp, Dresden, Germany ,= ,-_-. =.
/ ,- ) http://www.zero-based.org/ ((_/)o o(\_))
\ `-' `-'(. .)`-'
`-. Debian, a variant of the GNU operating system. \_/

[Mac]

On Thu, 06 May 2004 19:58:44 +0200, Martin Dickopp wrote:

If you convert a pointer to an object type to a pointer to another
object type, the resulting pointer is not guaranteed to be correctly
aligned. Therefore, your cast causes undefined behavior, i.e. anything
could happen.

You can, however, convert any pointer to a pointer to unsigned char and
then inspect the object byte by byte by way of the unsigned char pointer.
I point this out for the benefit of the OP, as I believe Martin already
knows this.
Martin

--Mac

[bryan]

>Eric Sosman <Er*********@sun.com> wrote in message >news:<40***************@sun.com>...

The fundamental thing you've missed is the magic
word "alignment." Some machines will permit a `long'
to begin at any arbitrary address, but others require
that it begin at an address that's a multiple of four,
or eight, or something of the sort. (One can deduce
that the required alignment for an object is a divisor
of the object's size, but that's another topic.)
Ok, this answers my question fully. So instead of writing a whole
long word I'll just write it one char at a time:

--- snip ---
#include <unistd.h>

int main() {
char buf[512];
u_char *char_p;
u_long *long_p;

memset(buf, 0x0, sizeof(buf));
memset(buf, 'A', 50);

char_p=buf;
long_p = (unsigned long *) (char_p+=strlen(buf));

/* *long_p = 0xffaa00dd; */
*char_p++ = 0xff ;
*char_p++ = 0xaa ;
*char_p++ = 0x00 ;
*char_p++ = 0xdd ;
}
--- snip ---

Its a bit more tedious, but it doesnt crash like it did previously,
and I dont have to worry about where I end up after filling buf[].
Now to the big question: What are you trying to do?
Explain your goal, and perhaps someone can chart a path
to it that doesn't take you through the Circles of Hell.

As for what I'm trying to do? Sticking an unsigned long value (memory
address) into an arbitrary location at the end of buf[]. So
eventually I'll want:

--- snip ---
....
void *from, *to;
u_char *char_p;
u_long *long_p,l_value;

char_p=buf;
long_p = (unsigned long *) (char_p+=strlen(buf));

/* *long_p = 0xffaa00dd; */
l_value = (long)(&string);

from = (void *)(&l_value);
to = (void *)(long_p);

memcpy(to, from, 4);
....
--- snip ---

And thats that. Thanks much.
Bryan.

[Eric Sosman]

bryan wrote:

Eric Sosman <Er*********@sun.com> wrote in message >news:<40***************@sun.com>...

Now to the big question: What are you trying to do?
Explain your goal, and perhaps someone can chart a path
to it that doesn't take you through the Circles of Hell.
As for what I'm trying to do? Sticking an unsigned long value (memory
address) into an arbitrary location at the end of buf[].

Guy walks into a bar and orders a beer, and instructs the
bartender to put a spoonful of salt in the bottom of the glass
before pouring. "Why do you want me to do that?" asks the
bartender. "To make sure it dissolves," says the guy.

In other words, when I asked "What are you trying to do?"
it was already obvious that you were trying to stuff a `long'
at an arbitrary location, just as it was obvious to the bartender
that the customer wanted salt in his beer. But just as it's weird
to want salt in beer, it's weird to want to stuff a `long' value
at a non-`long' place. The bartender didn't want to know why the
salt was to go into the glass first, but why it was to go in at all.
I'm similarly wondering why you want to do this weird thing
with `long' values.
So
eventually I'll want:

void *from, *to;
u_char *char_p;
u_long *long_p,l_value;

char_p=buf;
long_p = (unsigned long *) (char_p+=strlen(buf));

/* *long_p = 0xffaa00dd; */
l_value = (long)(&string);

from = (void *)(&l_value);
to = (void *)(long_p);

memcpy(to, from, 4);
...
--- snip ---

And thats that. Thanks much.

Boyoboyoboy are *you* living in a fool's paradise! It
appears that you haven't yet discovered "endianness," nor
have you discovered that `sizeof(long)' might not be four.
And from your remark that this `long' is actually "a memory
address," it appears you haven't quite understood that a
pointer might not be the same size as a `long'. You are
setting yourself up for lots of trouble here (and working
much too hard at it, too), so again I ask: Why?

--
Er*********@sun.com

[bryan]

>Eric Sosman <Er*********@sun.com> wrote in message >news:<40***************@sun.com>...

Guy walks into a bar and orders a beer, and instructs the
bartender to put a spoonful of salt in the bottom of the glass
before pouring. "Why do you want me to do that?" asks the
bartender. "To make sure it dissolves," says the guy.

In other words, when I asked "What are you trying to do?"
it was already obvious that you were trying to stuff a `long'
at an arbitrary location, just as it was obvious to the bartender
that the customer wanted salt in his beer. But just as it's weird
to want salt in beer, it's weird to want to stuff a `long' value
at a non-`long' place. The bartender didn't want to know why the
salt was to go into the glass first, but why it was to go in at all.
I'm similarly wondering why you want to do this weird thing
with `long' values.
ha, good analogy!

So, what am I trying to do? Well, first off, I was just tryig to side
step a straight forward question. I didnt want to get flamed by
saying that I was trying to exploit a bug in libDtHelp:
http://www.kb.cert.org/vuls/id/575804

Let me back that up by saying, I'm not trying to be malicious, I was
just bored at work and reading a few articles on buffer overflows, so
I thought I would play with it a little and learn somthing along the
way:
http://www.securityfocus.com/archive/1/12734

Furthermore, I havnt touched C in a while and this has been a good
practice exersice for getting acustomed to it again. Also, i've
taught myself how to use debuggers (both gdb and mdb), learned a
little assembly, some CPU architecture, and via these posts, learned a
few things about "alignment".

Boyoboyoboy are *you* living in a fool's paradise! It
appears that you haven't yet discovered "endianness," nor
have you discovered that `sizeof(long)' might not be four.
And from your remark that this `long' is actually "a memory
address," it appears you haven't quite understood that a
pointer might not be the same size as a `long'. You are
setting yourself up for lots of trouble here (and working
much too hard at it, too), so again I ask: Why?

--
Er*********@sun.com

Now... Yes, I do know about "endianness". Sparc is big endian, so I
neednt worry about byte order. I've viewed several examples from
memory dumps on how it is stored, so I also know its four bytes.
Maybe different areas are different lenghts, but I havnt run into that
yet, so no biggie.

I also understand that most of what I'm doing is not portable. I
havnt been down that road yet, so i'm not too concerned. This is for
one specific instance, and that is enough of a challenge for me. I am
a beginner C programmer, and I'm probably prone to do stupid things.
I'm ok with that for now.

So, at any rate. Thanks for your help, I've made some progress on the
overflow. If your interested I'll share what i've done. I probably
wont get it fully working, most likely because of the many unknown
technical details that I'm just not aware of, but i'm ok with that
too. Its fun anyway.
Bryan.

ps. Sorry for the delay in posting, but i'm stuck using google for
now.

[Eric Sosman]

bryan wrote:

Eric Sosman <Er*********@sun.com> wrote in message >news:<40***************@sun.com>...
[...] It
appears that you haven't yet discovered "endianness," nor
have you discovered that `sizeof(long)' might not be four. [...]

Now... Yes, I do know about "endianness". Sparc is big endian, so I
neednt worry about byte order. I've viewed several examples from
memory dumps on how it is stored, so I also know its four bytes.
Maybe different areas are different lenghts, but I havnt run into that
yet, so no biggie.

As it happens, sizeof(long) on SPARC is either four *or* eight,
depending on the compiler options. Be wary of what you "know!"

--
Er*********@sun.com