The Heap?

[Markus]

Simple question.

In the book I previously read on C, it didn't go into what the heap was. In some lectures I'm watching currently, the guy talks about the heap but doesn't explain what it is (maybe he's gone over this in a previous lecture).

Do you know of any resources I could use (online, preferably) to understand the memory management, heap, etc. in C?

Thanks.

[Markus]

Furthermore, what happens with the following 2 examples (are both stored on the heap, or..)?

int *arr = (int *)malloc(4 * sizeof(int)); // On the heap?
int arr[4]; // Or is this on the heap?
// Or both?
 

Mark.

P.S. I apologise for the newbie questions. Maybe I should take back my previous pseudonym of markusn00b?

P.P.S. I apologise for having to apologise.

P.P.P.S Ad infinitum..

[Banfa]

Most C and C++ implementations rely on both a heap and a stack. However you may not find it in a book on the subject because neither are mandated by the standard, how an implementation organises its own memory is implementation defined.

malloc, realloc, calloc and new all allocate from the local heap. The local heap is the memory in the heap available to the program as opposed to the system heap, the memory available to the entire system. In an embedded program the program heap and system heap are often the same but for Windows and Linux they are very different, the program rarely as access to the entire memory of the system under Windows or Linux. In the days of 16 bit Windows the size of the program heap had to be set in the linker.

Variables with automatic scope (look up the auto keyword) are allocated from the stack (if the implementation has one). The stack is another area of memory the size of the used stack grows and shrinks as function calls are made. Each time a function call is made, the current state of the processor and the return address are stored to the stack space is also reserved on the stack for any auto variables in the function just called, that is variables local to the function.

Data declared at file scope or data declared at function scope with the static keyword are stored in the data segment. This is a section of data that the C start-up code allocates (if required) and initialises. These variables are normally split into 2 sections, those with a non-zero initial value and those with a zero initial value or non initial value (which automatically get assign zero). The later section needs to have their initial values stored in the program code so that c start-up can initialise them properly, those in the latter section can just be zeroed programmatically.

As to your code, truely I should ask if that code is inside or outside a function, however since you have called malloc it must be inside a function.

Therefore the first declaration allocates an int pointer on the stack, then calls malloc to allocate data for 4 ints on the heap and stores a pointer to it in the int pointer.

The second declaration allocates an array of 4 ints on the stack.

It would normally be considered bad practice to allocate large objects (more than a few hundred bytes) on the stack. For a recursive function allocating more than a few bytes on the stack is likely to be a bad idea. However it does rather depend on how your implementation organises its memory.

[Markus]

Thanks. That's a great explanation.

Mark.

[myusernotyours]

Banfa, what about if his code was under main? Is main treated just like any other function (the first) and the automatic variables allocated on the stack?

Incidentally just today I was trying to find out the entire memory available on a programs heap on my windows (with g++ on mingw) and I could allocate an array of char[125000] 17029 times and failing on the 17030th array. That's almost 2gb. My laptop (on which I was doing this) has 3gb of RAM. So it seems like the heap is much more than you would expect. This apparently has something to do with how windows allocates memory that is 'really not there' physically. I have no idea whether it does this for the stack also.
Finally, for my 2gb allocation, the taskman showed memory usage of under 150000KB I wonder why.

Regards,

Alex

[Banfa]

Yes main is like any other function in this respect.

I have worked with embedded micro-controllers whose compilers had a switch to make all function local variables default to static allocation rather than auto allocation (effectively taking them off the stack an putting the onto the heap.

On (32bit) WIndows any program can have up to 2Gbyte of allocated memory with another 2Gbyte reserved for the system to use on behalf of the program.

That 2Gbyte can be made up using heap, stack or statically allocated data. On Windows both the heap and stack of a program is expandable to accommodate what the program allocates (up to the 2Gbyte limit).

Windows (and Linux) can supplement physical memory with a swap file on the hard drive giving the computer more available memory that its actually physical memory.

A can not explain why taskman has the number wrong, it may in fact be displaying the physical ram being used by your program rather than the amount of memory it has allocated. Try Process Explorer from SysInternals (google it).

[myusernotyours]

You are right. Taskman only shows the 'Physical memory' procexp calls it 'Working memory'. Actually I had tried procexp too and found this 'Working memory' to be the same as what taskman reported, but when you mentioned it, I checked it again and realised I had to enable a column that shows the virtual memory size (called just 'virtual size' by the program) that shows the entire 2gb allocated to the process. I had read about this 2gb-program and 2gb-OS thing but it wasn't very clear to me. Now I think it is! I suspect on machines that support 64GB virtual memory I could allocate almost half of that right?

Regards,

Alex.

[Banfa]

In 64 bit windows 32 bit applications are allocated 4Gbyte of address space by the OS, however they need to be compiled with the /LARGEADDRESSAWARE to be able to access >2Gbyte of address space.

64bit applications get 8Tbyte of address space, with 8Tbyte reserved for the OS.

Having the address space does not mean you can allocate the memory, you can only allocate the memory if you have the address space and the RAM (or swap file ram).

I am not quite sure where you got the 64Gbyte limit from.

http://support.microsoft.com/kb/294418