Different Variables Use Same Memory

[MajorSetback]

I am using the Redhat version of Linux and GNU C++.

It is not clear to me whether this is a Linux issue or a C++ issue. I
do not have this problem running the same program on Windows but
tweaking the C++ code appears to fix the problem on Linux.

I have a static array that is declared privately in one class and a
structure that is declared publicly in another class. The program uses
both classes. I was getting core dumps and traced the problem down to
the fact that the array shares the same memory as the structure so
that, when an array element is written to, the structure is
overwritten. I found that the problem appears to go away when I change
thet declaration of the static array to public. However, I am not sure
I understand the issues involved. In fact, I am not really clear of
the advantages of declaring a variable as private as opposed to public,
apart from not wanting someone to modify the variable erroneously.
However, it appears that declaring it as private makes the program go
ahead and erroneously change it anyway.

I should also point out that the class where the static array is
declared privately, has the class with the publicaly declared structure
as one of its class objects. I have no idea why the compiled code
would make them use the same memory.

Any clarification of this/these issues would be greatly appreciated.

Thanks,
Peter.

[Daniel T.]

Ma**********@excite.com wrote:

I am using the Redhat version of Linux and GNU C++.

It is not clear to me whether this is a Linux issue or a C++ issue. I
do not have this problem running the same program on Windows but
tweaking the C++ code appears to fix the problem on Linux.

I have a static array that is declared privately in one class and a
structure that is declared publicly in another class. The program uses
both classes. I was getting core dumps and traced the problem down to
the fact that the array shares the same memory as the structure so
that, when an array element is written to, the structure is
overwritten. I found that the problem appears to go away when I change
thet declaration of the static array to public. However, I am not sure
I understand the issues involved. In fact, I am not really clear of
the advantages of declaring a variable as private as opposed to public,
apart from not wanting someone to modify the variable erroneously.
However, it appears that declaring it as private makes the program go
ahead and erroneously change it anyway.

The fact that the variable is private limits the places you have to look
when trying to figure out what you did wrong to cause the array writes
to stomp on the structure. That's a huge benefit in my book.

I should also point out that the class where the static array is
declared privately, has the class with the publicaly declared structure
as one of its class objects. I have no idea why the compiled code
would make them use the same memory.

Any clarification of this/these issues would be greatly appreciated.

They "use the same memory" because you are accessing one or both of them
incorrectly. It seems to go away because when you move one to the
"public" section of the class you are changing the order that they are
laid out in memory, thus you are overwriting some other part of memory
instead of the particular structure you noticed the problem in. What you
need to do is find out where you are jumping the bounds of your array
and fix that code.

One solution would be to replace the array with a vector and use the
"at" member-function or create some sort of checked array class that
alerts you when you write outside of it.

--
To send me email, put "sheltie" in the subject.

[Steve Pope]

This almost has to be a compiler or environment problem
rather than a language or programming problem (unless
there is something you're not telling us).

Even so it might be useful to try to create the minimal
program that exhibits the problem and analyze that.

Steve

[Salt_Peter]

[e-mail] wrote:

I am using the Redhat version of Linux and GNU C++.

It is not clear to me whether this is a Linux issue or a C++ issue. I
do not have this problem running the same program on Windows but
tweaking the C++ code appears to fix the problem on Linux.

I have a static array that is declared privately in one class and a
structure that is declared publicly in another class. The program uses
both classes. I was getting core dumps and traced the problem down to
the fact that the array shares the same memory as the structure so
that, when an array element is written to, the structure is
overwritten. I found that the problem appears to go away when I change
thet declaration of the static array to public. However, I am not sure
I understand the issues involved. In fact, I am not really clear of
the advantages of declaring a variable as private as opposed to public,
apart from not wanting someone to modify the variable erroneously.
However, it appears that declaring it as private makes the program go
ahead and erroneously change it anyway.

I should also point out that the class where the static array is
declared privately, has the class with the publicaly declared structure
as one of its class objects. I have no idea why the compiled code
would make them use the same memory.

Any clarification of this/these issues would be greatly appreciated.

Thanks,
Peter.

How did you initialize the static array?
Why is the array static?
Why can't you provide minimal code?

#include <iostream>

class A
{
static int narray[10];
public:
static int get(int index) { return A::narray[index]; }
static void set(int index, int val) { narray[index] = val; }
};

int A::narray[] = { 0,1,2,3,4,5,6,7,8,9 }; // initialized array

int main()
{
A a;
a.set(5, 50);
for( size_t index = 0; index < 10; ++index)
{
std::cout << a.get(index) << std::endl;
}
return 0;
}

/*
0
1
2
3
4
50
6
7
8
9
*/

[Bernd Strieder]

Hello,
Ma**********@excite.com wrote:

I have a static array that is declared privately in one class and a
structure that is declared publicly in another class. The program
uses both classes.

I was getting core dumps and traced the problem down to
the fact that the array shares the same memory as the structure so
that, when an array element is written to, the structure is
overwritten.

Any clarification of this/these issues would be greatly appreciated.

There is pretty surely a programming error in your code leading to
undefined behaviour. This could be due to overruns of some memory area,
it could be due to keeping some area in use after freeing or deleting.
These kinds of errors can be notoriously hard to identify. It is almost
impossible to create minimal examples to make it easy for others to
help, because minimal changes in the code might cause the crash going
away.

You have basically two options:

1. Do it the hard way, and nail it down: Learn about setting watchpoints
in gdb, or whatever your favourite debugger is. Make sure you have
deterministic failing runs of your code, i.e. all adresses and values
stay constant between different runs and then try to find out all
occasions, where the corrupt memory area has been in use before the
crash, and check, whether all parties play fair.

It looks like you have done some part of this already, you have found
the data structures with the conflict, now you have to find out which
one is first, and how the other one gets into the same area.

2. Use valgrind to find possible errors in using heap memory. It is
quite probable, that valgrind will identify your problem, but there is
no warrant.

Additionally you could write some procedures checking your data
structures, i.e. scanning them and touching the memory. Insert calls to
those checking procedures at viable places in your code. This can help
both basic options to get to the interesting places earlier.

Most probably you will have to know in detail, at least for the data
structures involved in the crash, where and when memory is allocted and
freed again, i.e. the full lifecycle.

Bernd Strieder

[Jean-David Beyer]

Ma**********@excite.com wrote:

I am using the Redhat version of Linux and GNU C++.

It is not clear to me whether this is a Linux issue or a C++ issue.

I bet it is neither Linux nor C++; most likely a programming error.
Are you an experienced C++ user, or are you a beginner?

I
do not have this problem running the same program on Windows but
tweaking the C++ code appears to fix the problem on Linux.

I spent several years working on the optimizer for C (and, implicitly, C++)
for the UNIX OS. People persisted in filing trouble reports on the optimizer
(which ran between the compiler and assembler steps in that system) because
their programs worked unoptimized (or so they said) and failed when optimized.

It turned out that the problems were in their source programs. They were
faulty. One thing to note about an optimizer is that _for correct programs_,
the optimized program should give the same results as the unoptimized ones.
And our optimizer did that. But it did not guarantee to give the same
results for incorrect programs.

One example is that for optimized programs, we did register allocation even
if the programmer did not explicitly declare a variable to go into a
register. And we overloaded registers. If a variable needed to be in a
register in one part of a function but not in the rest, and another variable
needed to be in a register in another part of a function, they could both be
assigned to the same register. Now consider what happens when the program
uses a variable, to which no value has been assigned, as a pointer. While
_not required_, a memory location is typically zero until a value is
assigned to it. Now in the case of overloaded registers, if a variable has
no value assigned to it, what the program gets is just whatever was in the
register, which is usually non-zero. So in the unoptimized case, the program
gets a null pointer (which should crash the program), where in the optimized
case, the program just uses some memory location somewhere.

This is a simple case of a programming error that runs differently on
different machines; different in this case being simply the unoptimized
machine and the optimized machine -- if you care to look at it that way.

Now in C++, the normal place to allocate memory is in the constructor
functions and the normal place to deallocate memory is in the destructor
functions. And I try to never let pointers to such memory escape from any of
the methods on an object. If you program like that, all your memory
allocations will be in the constructors and destructors, which should make
debugging easier: put tests in them to verify what is going on. And be sure
your constructors initialize everything.

>
I have a static array that is declared privately in one class and a
structure that is declared publicly in another class. The program uses
both classes. I was getting core dumps and traced the problem down to
the fact that the array shares the same memory as the structure so
that, when an array element is written to, the structure is
overwritten. I found that the problem appears to go away when I change
thet declaration of the static array to public. However, I am not sure
I understand the issues involved. In fact, I am not really clear of
the advantages of declaring a variable as private as opposed to public,
apart from not wanting someone to modify the variable erroneously.
However, it appears that declaring it as private makes the program go
ahead and erroneously change it anyway.

I normally have all variables in a class be private. Now I may have
functions in the class that return the value of a variable, but that is not
the same as making the variable public. Similarly, though less likely, I
might have a function that sets the value of a variable. But normally, that
is getting too fine: I might, in a reimplementation of the class, remove the
variable entirely. This should not affect the users of the class.

>
I should also point out that the class where the static array is
declared privately, has the class with the publicaly declared structure
as one of its class objects. I have no idea why the compiled code
would make them use the same memory.

Any clarification of this/these issues would be greatly appreciated.

Thanks,
Peter.

--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey http://counter.li.org
^^-^^ 07:30:01 up 10 days, 9:57, 3 users, load average: 4.41, 3.78, 3.53

[The Natural Philosopher]

Jean-David Beyer wrote:

Ma**********@excite.com wrote:

>I am using the Redhat version of Linux and GNU C++.

It is not clear to me whether this is a Linux issue or a C++ issue.

I bet it is neither Linux nor C++; most likely a programming error.
Are you an experienced C++ user, or are you a beginner?

>I
do not have this problem running the same program on Windows but
tweaking the C++ code appears to fix the problem on Linux.

I spent several years working on the optimizer for C (and, implicitly, C++)
for the UNIX OS. People persisted in filing trouble reports on the optimizer
(which ran between the compiler and assembler steps in that system) because
their programs worked unoptimized (or so they said) and failed when optimized.

It turned out that the problems were in their source programs. They were
faulty. One thing to note about an optimizer is that _for correct programs_,
the optimized program should give the same results as the unoptimized ones.
And our optimizer did that. But it did not guarantee to give the same
results for incorrect programs.

One example is that for optimized programs, we did register allocation even
if the programmer did not explicitly declare a variable to go into a
register. And we overloaded registers. If a variable needed to be in a
register in one part of a function but not in the rest, and another variable
needed to be in a register in another part of a function, they could both be
assigned to the same register. Now consider what happens when the program
uses a variable, to which no value has been assigned, as a pointer. While
_not required_, a memory location is typically zero until a value is
assigned to it. Now in the case of overloaded registers, if a variable has
no value assigned to it, what the program gets is just whatever was in the
register, which is usually non-zero. So in the unoptimized case, the program
gets a null pointer (which should crash the program), where in the optimized
case, the program just uses some memory location somewhere.

This is a simple case of a programming error that runs differently on
different machines; different in this case being simply the unoptimized
machine and the optimized machine -- if you care to look at it that way.

Now in C++, the normal place to allocate memory is in the constructor
functions and the normal place to deallocate memory is in the destructor
functions. And I try to never let pointers to such memory escape from any of
the methods on an object. If you program like that, all your memory
allocations will be in the constructors and destructors, which should make
debugging easier: put tests in them to verify what is going on. And be sure
your constructors initialize everything.

>I have a static array that is declared privately in one class and a
structure that is declared publicly in another class. The program uses
both classes. I was getting core dumps and traced the problem down to
the fact that the array shares the same memory as the structure so
that, when an array element is written to, the structure is
overwritten. I found that the problem appears to go away when I change
thet declaration of the static array to public. However, I am not sure
I understand the issues involved. In fact, I am not really clear of
the advantages of declaring a variable as private as opposed to public,
apart from not wanting someone to modify the variable erroneously.
However, it appears that declaring it as private makes the program go
ahead and erroneously change it anyway.

I normally have all variables in a class be private. Now I may have
functions in the class that return the value of a variable, but that is not
the same as making the variable public. Similarly, though less likely, I
might have a function that sets the value of a variable. But normally, that
is getting too fine: I might, in a reimplementation of the class, remove the
variable entirely. This should not affect the users of the class.

>I should also point out that the class where the static array is
declared privately, has the class with the publicaly declared structure
as one of its class objects. I have no idea why the compiled code
would make them use the same memory.

Any clarification of this/these issues would be greatly appreciated.

Thanks,
Peter.

Yep seen all that sort of stuff.

Another good one is to overwrite memory that 'doesn't belong to you' -
like putting a string too long in a buffer of fixed size. If its a local
variable it will smash the sack and crash the machine, but if its a
global or static variable, there is no guarantee WHAT will get
overwritten. It mighty be memory thats never used for anything, it might
be memory that is utterly crucial for the program to work - -say an
array of pointers to functions.. call into those and its 'bye bye kansas'

C at least (I am not a C++ man) does not define the contents of
temporary variables, or the order in real memory and precise location of
static or temporary variables. I am not sure about 'malloc()'ed) memory
either.

Nothing is easier to declare a local pointer, and access memory it
points to without initialising it first.

{
char *p;
strcpy(p, "where will this end up");
}

[Frederick Gotham]

The Natural Philosopher:

C at least (I am not a C++ man) does not define the contents of
temporary variables,

Yes it does. The following "temporary" has a well-defined value:

(int)53.5667

or the order in real memory and precise location of
static or temporary variables.

That's up to the implementation -- and it's irrelevant to the internal
workings of the program.

I am not sure about 'malloc()'ed) memory either.

Well you can store the address in a void*, and that's all you need to know.

Nothing is easier to declare a local pointer, and access memory it
points to without initialising it first.

{
char *p;
strcpy(p, "where will this end up");
}

That's not easy, not if you're very familiar with initialising pointers.
You could say that it's very easy to turn the kettle on without filling it
with water first. . . but what percentage of people don't check that
there's water?

--

Frederick Gotham

[MajorSetback]

Bernd Strieder wrote:

Hello,
Ma**********@excite.com wrote:

I have a static array that is declared privately in one class and a
structure that is declared publicly in another class. The program
uses both classes.

I was getting core dumps and traced the problem down to
the fact that the array shares the same memory as the structure so
that, when an array element is written to, the structure is
overwritten.

Any clarification of this/these issues would be greatly appreciated.

There is pretty surely a programming error in your code leading to
undefined behaviour. This could be due to overruns of some memory area,
it could be due to keeping some area in use after freeing or deleting.
These kinds of errors can be notoriously hard to identify. It is almost
impossible to create minimal examples to make it easy for others to
help, because minimal changes in the code might cause the crash going
away.

You have basically two options:

1. Do it the hard way, and nail it down: Learn about setting watchpoints
in gdb, or whatever your favourite debugger is. Make sure you have
deterministic failing runs of your code, i.e. all adresses and values
stay constant between different runs and then try to find out all
occasions, where the corrupt memory area has been in use before the
crash, and check, whether all parties play fair.

It looks like you have done some part of this already, you have found
the data structures with the conflict, now you have to find out which
one is first, and how the other one gets into the same area.

2. Use valgrind to find possible errors in using heap memory. It is
quite probable, that valgrind will identify your problem, but there is
no warrant.

Additionally you could write some procedures checking your data
structures, i.e. scanning them and touching the memory. Insert calls to
those checking procedures at viable places in your code. This can help
both basic options to get to the interesting places earlier.

Most probably you will have to know in detail, at least for the data
structures involved in the crash, where and when memory is allocted and
freed again, i.e. the full lifecycle.

Bernd Strieder

Thanks very much for your help. I was not previously familiar with
valgrind and was very impressed. It is easy to use and found 30 memory
errors in my program. The error report was easy to understand and it
enabled me to fix the errors with little difficulty. Unfortunately,
the problem had not manifested itself after I reprivatised the static
arrays (by undoing the publicising changes). However, I could
certainly see the errors in the code after valgrind drew my attention
to them.

Thanks again,
Peter.

[MajorSetback]

Daniel T. wrote:

>
The fact that the variable is private limits the places you have to look
when trying to figure out what you did wrong to cause the array writes
to stomp on the structure. That's a huge benefit in my book.

Good point.

I should also point out that the class where the static array is
declared privately, has the class with the publicaly declared structure
as one of its class objects. I have no idea why the compiled code
would make them use the same memory.

They "use the same memory" because you are accessing one or both of them
incorrectly. It seems to go away because when you move one to the
"public" section of the class you are changing the order that they are
laid out in memory, thus you are overwriting some other part of memory
instead of the particular structure you noticed the problem in. What you
need to do is find out where you are jumping the bounds of your array
and fix that code.

Unfortunately, the problem did not manifest after I undid the changes
that unprivatized the static arrays. Hence, I was not able to tell
with certainly whether certain changes fixed the problem. However, I
found 30 memory errors with valgrind and have fixed them.

Thanks,
Peter.

[MajorSetback]

Salt_Peter wrote:

>
How did you initialize the static array?
Why is the array static?

Maybe I'm not using the correct term. I am declaring the array as
float faArray[64];
rather than
float *fpArray;
with a subsequent call to malloc() (or whatever).

Why can't you provide minimal code?

The code involved in the program is huge. I was not clear where the
problem was occurring although it did appear to be early in the
program. hence I asked the question in general terms. Valgrind found
about 30 memory errors and they may have been the cause. If not, I
will post the code where the problem appears to be occurring.

Thanks,
Peter.

[Jean-David Beyer]

Ma**********@excite.com wrote:

Salt_Peter wrote:

>How did you initialize the static array? Why is the array static?

Maybe I'm not using the correct term. I am declaring the array as float
faArray[64]; rather than float *fpArray; with a subsequent call to
malloc() (or whatever).

Those are rather different. The first declares an array of size 64 floats.
The second declares a pointer to a float.

Are you writing in C or C++?

>

>Why can't you provide minimal code?

The code involved in the program is huge. I was not clear where the
problem was occurring although it did appear to be early in the program.
hence I asked the question in general terms. Valgrind found about 30
memory errors and they may have been the cause. If not, I will post the
code where the problem appears to be occurring.

30 is a lot of errors. I never heard of valgrind, but it seems to be a
really useful program if you have problems like that.

--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey http://counter.li.org
^^-^^ 13:55:01 up 10 days, 16:22, 3 users, load average: 4.33, 4.26, 4.20

[MajorSetback]

Jean-David Beyer wrote:

Are you an experienced C++ user, or are you a beginner?

It's relative. I have been programming in C++ for a number of years
but have not taken any formal course-work or training. There could be
things I overlooked.

Now in C++, the normal place to allocate memory is in the constructor
functions and the normal place to deallocate memory is in the destructor
functions. And I try to never let pointers to such memory escape from any of
the methods on an object. If you program like that, all your memory
allocations will be in the constructors and destructors, which should make
debugging easier: put tests in them to verify what is going on. And be sure
your constructors initialize everything.

I initialize pointers to NULL in the constructor and free them in the
destructor if they are no longer NULL. I have a number of classes that
contain pointers that may not be used in an application or which may be
used but with different memory allocations. Hence, I may allocate
memory other than in the constructor. In this case, I check to see if
they are NULL, and free them otherwise, before I allocate memory.

Thanks,
Peter.

[Jean-David Beyer]

The Natural Philosopher wrote:

Another good one is to overwrite memory that 'doesn't belong to you' -
like putting a string too long in a buffer of fixed size.

IIRC, the OP claimed to be writing in C++ and if he does that, he would
never overwrite a string too long since there are string objects in the C++
Standard Library and they always arrange to get themselves large enough
unless the program runs out of virtual memory space, in which case you find
out about it fast.

If its a local
variable it will smash the sack and crash the machine, but if its a
global or static variable, there is no guarantee WHAT will get
overwritten. It mighty be memory thats never used for anything, it might
be memory that is utterly crucial for the program to work - -say an
array of pointers to functions.. call into those and its 'bye bye kansas'

Well, you never know in any case, since it depends on the implementation of
the compilation system, the hardware, etc. We used to arrange that programs
never used the first page of hardware addresses. And we set the hardware
memory management unit to trap if there was an attempt to access that page.
And all uninitialized memory was initialized to zero (null). So if you
dereferenced a null pointer, it tried to address the first page and you got
a core dump.

>
C at least (I am not a C++ man) does not define the contents of
temporary variables, or the order in real memory and precise location of
static or temporary variables. I am not sure about 'malloc()'ed) memory
either.

Nothing is easier to declare a local pointer, and access memory it
points to without initialising it first.

{
char *p;
strcpy(p, "where will this end up");
}

On the systems where I was involved with code generation, that would get you
a core dump when strcpy tried to store the "w". Of course another department
set up UNIX to ignore those interrupts, which I thought was insane, but they
said they got too many core dumps. INSANE!

--
.~. Jean-David Beyer Registered Linux User 85642.
/V\ PGP-Key: 9A2FC99A Registered Machine 241939.
/( )\ Shrewsbury, New Jersey http://counter.li.org
^^-^^ 13:55:01 up 10 days, 16:22, 3 users, load average: 4.33, 4.26, 4.20

[MajorSetback]

Jean-David Beyer wrote:

Ma**********@excite.com wrote:

Salt_Peter wrote:

How did you initialize the static array? Why is the array static?

Maybe I'm not using the correct term. I am declaring the array as float
faArray[64]; rather than float *fpArray; with a subsequent call to
malloc() (or whatever).

Those are rather different. The first declares an array of size 64 floats.
The second declares a pointer to a float.

What I meant was that the memory was statically, rather than
dynamically, allocated.

>
Are you writing in C or C++?

C++. But I was a C programmer so I may still have a lot of "baggage".

Why can't you provide minimal code?

The code involved in the program is huge. I was not clear where the
problem was occurring although it did appear to be early in the program.
hence I asked the question in general terms. Valgrind found about 30
memory errors and they may have been the cause. If not, I will post the
code where the problem appears to be occurring.

30 is a lot of errors. I never heard of valgrind, but it seems to be a
really useful program if you have problems like that.

It sounds bad but I found that replacing a < with <= zapped about half
of them. It certainly is a useful program. I will be using it
regularly from now on.

Thanks,
Peter.

[red floyd]

Ma**********@excite.com wrote:

I initialize pointers to NULL in the constructor and free them in the
destructor if they are no longer NULL.

You don't have to NULL check pointers in the destructor. delete will
behave properly if passed a NULL pointer (you *are* using delete, and
not free(), right?).

[MajorSetback]

red floyd wrote:

Ma**********@excite.com wrote:

I initialize pointers to NULL in the constructor and free them in the
destructor if they are no longer NULL.

You don't have to NULL check pointers in the destructor. delete will
behave properly if passed a NULL pointer (you *are* using delete, and
not free(), right?).

I still use malloc/calloc with free. I could switch to new and delete
if there is a good reason to do so. I guess not having to worry about
checking for NULL pointers may be an advantage but I don't find that a
problem. I like malloc and calloc because you can initialise
everything to zero (with calloc) if you want to or not do so (with
malloc) if it is unnecessary.

Thanks,
Peter.

[Frederick Gotham]

MajorSetback posted:

I still use malloc/calloc with free. I could switch to new and delete
if there is a good reason to do so.

Faster compile time as you don't have to #include <cstdlib>. In general, I
always opt for a "built-in" language feature rather than a library feature.

I guess not having to worry about
checking for NULL pointers may be an advantage but I don't find that a
problem.

Well there's always the option of new(nothrow).

I like malloc and calloc because you can initialise
everything to zero (with calloc)

new int[5]()

if you want to or not do so (with
malloc) if it is unnecessary.

new int[5]

I myself only use malloc if I'm doing some fancy memory control, e.g. using
realloc also.

--

Frederick Gotham

[Måns Rullgård]

Ma**********@excite.com writes:

red floyd wrote:

>Ma**********@excite.com wrote:

I initialize pointers to NULL in the constructor and free them in the
destructor if they are no longer NULL.

You don't have to NULL check pointers in the destructor. delete will
behave properly if passed a NULL pointer (you *are* using delete, and
not free(), right?).

I still use malloc/calloc with free. I could switch to new and delete
if there is a good reason to do so. I guess not having to worry about
checking for NULL pointers may be an advantage but I don't find that a
problem.

Calling free() with a null pointer is perfectly legal.

--
Måns Rullgård
mr*@inprovide.com