I wonder how the runtime allocates memory. Please excuse this extended
message.
#include <stdlib.h>
#include <stdio.h>
void buf(int n){
void* buffer = NULL;
buffer = malloc(n);
printf("buffer of size %u: \t%p", n, buffer);
if(((unsigned long)buffer) % n == 0){
printf(", address aligned to buffer size");
}
printf("\n");
free(buffer);
}
int main(int ac, char** av){
unsigned long one = 1;
int i = 0;
buf(getpagesize());
for(i=0;i<32;i++){
buf(one << i);
}
return 0;
}
I run that little program and get:
buffer of size 4096: 0x44520
buffer of size 1: 0x44520, address aligned to buffer size
buffer of size 2: 0x44520, address aligned to buffer size
buffer of size 4: 0x44520, address aligned to buffer size
buffer of size 8: 0x44520, address aligned to buffer size
buffer of size 16: 0x44520, address aligned to buffer size
buffer of size 32: 0x44520, address aligned to buffer size
buffer of size 64: 0x44520
buffer of size 128: 0x44520
buffer of size 256: 0x44520
buffer of size 512: 0x44520
buffer of size 1024: 0x44520
buffer of size 2048: 0x44520
buffer of size 4096: 0x44520
buffer of size 8192: 0x45570
buffer of size 16384: 0xa4000, address aligned to buffer size
buffer of size 32768: 0xa8000, address aligned to buffer size
buffer of size 65536: 0xb0000, address aligned to buffer size
buffer of size 131072: 0xc0000, address aligned to buffer size
buffer of size 262144: 0xe0000
buffer of size 524288: 0x120000
buffer of size 1048576: 0x1a0000
buffer of size 2097152: 0x2a0000
buffer of size 4194304: 0x4a0000
buffer of size 8388608: 0x8a0000
buffer of size 16777216: 0x10a0000
buffer of size 33554432: 0x10a0000
buffer of size 67108864: 0x10a0000
buffer of size 134217728: 0x10a0000
buffer of size 268435456: 0x10a0000
buffer of size 536870912: 0x10a0000
buffer of size 1073741824: 0x10a0000
*** malloc[931]: error: Can't allocate region
buffer of size 2147483648: 0x0, address aligned to buffer size
So, it looks like asking malloc for a block the size of the memory page
does not result in a page-aligned block. All the allocated blocks are
on a 32-byte boundary. for 16, 32, 64, and 128 kilobyte blocks it
might seem that they are allocated to be aligned to the block size,
that might be not the case except for 64 kilobytes. After 16 megabytes
the returned address is the same.
I don't free each allocating buffer, letting the process' return from
main free the memory buffers.
buffer of size 4096: 0x44520
buffer of size 1: 0x45570, address aligned to buffer size
buffer of size 2: 0x45580, address aligned to buffer size
buffer of size 4: 0x45590, address aligned to buffer size
buffer of size 8: 0x455a0, address aligned to buffer size
buffer of size 16: 0x455b0, address aligned to buffer size
buffer of size 32: 0x455d0
buffer of size 64: 0x45600, address aligned to buffer size
buffer of size 128: 0x45650
buffer of size 256: 0x456e0
buffer of size 512: 0x457f0
buffer of size 1024: 0x45a00
buffer of size 2048: 0x45e10
buffer of size 4096: 0x46620
buffer of size 8192: 0x47630
buffer of size 16384: 0xa4000, address aligned to buffer size
buffer of size 32768: 0xa8000, address aligned to buffer size
buffer of size 65536: 0xb0000, address aligned to buffer size
buffer of size 131072: 0xc0000, address aligned to buffer size
buffer of size 262144: 0xe0000
buffer of size 524288: 0x120000
buffer of size 1048576: 0x1a0000
buffer of size 2097152: 0x2a0000
buffer of size 4194304: 0x4a0000
buffer of size 8388608: 0x8a0000
buffer of size 16777216: 0x10a0000
buffer of size 33554432: 0x20a0000
buffer of size 67108864: 0x40a0000
buffer of size 134217728: 0x80a0000
buffer of size 268435456: 0x100a0000
buffer of size 536870912: 0x200a0000
*** malloc[960]: error: Can't allocate region
buffer of size 1073741824: 0x0, address aligned to buffer size
*** malloc[960]: error: Can't allocate region
buffer of size 2147483648: 0x0, address aligned to buffer size
It appears that 16 bytes, on this system, is the least alignment (gcd).
In a recent e-mail I was reading about how an allocated block sometimes
has bookkeeping information in the word or two immediately prior to the
contiguous block. I research, and find some interesting web sites
discussing this kind of thing.
http://www.memorymanagement.org/ http://ose.sourceforge.net/browse.ph...try=memory.htm
There are lots of memory managers. Their behavior is often
predictable. It's predictable that misuse of dynamic memory
allocations will cause problems.
Basically I'm not very concerned, but I do wonder about high
performance access to physical memory, but I don't have any inclination
to consider implementing a virtual memory. I look at sys/mman.h, this
is a BSD system. There appears to be discussion of pages and
"regions". Hmm... on the malloc man page, it describes a
(non-standard) function malloc_good_size. For each of the above block
sizes, the recommended "good" size is 14 bytes more than the block size
unless it is less than 8 in which case it says 14 or 16k or more in
which case it is the block size. The malloc_size function on this
implementation always returns zero, but errno is not set.
#include <stdlib.h>
#include <stdio.h>
#include <errno.h>
void buf(int n);
void info_good_size(int n){
printf("\t%u", malloc_good_size(n));
/* buf(malloc_good_size(n)); */
}
void buf(int n){
size_t msize=0;
void* buffer = NULL;
buffer = malloc(n);
printf("buffer of size %u: \t%p", n, buffer);
msize = malloc_size(buf);
if(errno) perror(NULL);
printf("\t%u", msize);
info_good_size(n);
if(((unsigned long)buffer) % n == 0){
printf(" A ");
}
printf("\n");
free(buffer);
}
int main(int ac, char** av){
....
buffer of size 4096: 0x44520 0 4110
buffer of size 1: 0x44520 0 14 A
buffer of size 2: 0x44520 0 14 A
buffer of size 4: 0x44520 0 14 A
buffer of size 8: 0x44520 0 14 A
buffer of size 16: 0x44520 0 30 A
buffer of size 32: 0x44520 0 46 A
buffer of size 64: 0x44520 0 78
buffer of size 128: 0x44520 0 142
buffer of size 256: 0x44520 0 270
buffer of size 512: 0x44520 0 526
buffer of size 1024: 0x44520 0 1038
buffer of size 2048: 0x44520 0 2062
buffer of size 4096: 0x44520 0 4110
buffer of size 8192: 0x45570 0 8206
buffer of size 16384: 0xa4000 0 16384 A
buffer of size 32768: 0xa8000 0 32768 A
buffer of size 65536: 0xb0000 0 65536 A
buffer of size 131072: 0xc0000 0 131072 A
buffer of size 262144: 0xe0000 0 262144
buffer of size 524288: 0x120000 0 524288
buffer of size 1048576: 0x1a0000 0 1048576
buffer of size 2097152: 0x2a0000 0 2097152
buffer of size 4194304: 0x4a0000 0 4194304
buffer of size 8388608: 0x8a0000 0 8388608
buffer of size 16777216: 0x10a0000 0 16777216
buffer of size 33554432: 0x10a0000 0 33554432
buffer of size 67108864: 0x10a0000 0 67108864
buffer of size 134217728: 0x10a0000 0 134217728
buffer of size 268435456: 0x10a0000 0 268435456
buffer of size 536870912: 0x10a0000 0 536870912
buffer of size 1073741824: 0x10a0000 0 1073741824
*** malloc[1078]: error: Can't allocate region
buffer of size 2147483648: 0x0 0 2147483648 A
If I uncomment the recursive call to buf() in info_good_size():
Segmentation fault
That is confusing.
So, it seems like instead of using getpagesize() as a default memory
block size, I should start at the least power of two greater than or
equal to getpagesize, and shift that left a bit at a time and compare
that to malloc_good_size of that until they are equal and use that.
While that may be so, malloc_good_size is not a portable function, or
rather not in the standard C library. I could just allocate blocks of
those test sizes until one is aligned on the block boundary, and assume
that's the region.
Basically I'm wondering the most humane way to treat malloc, _the_
standard C library dynamic heap memory allocation facility, to get
memory from it. I want some nice growable arrays, and by nice meaning
the most efficient in terms of being polite to the hardware.
Standard C might be mute about these things, but programs written in
standard C are not deaf to their concerns, or if they are, it's from
too much noise.
I'm trying to determine some good practices for coding C libraries in
general, and the dynamic memory allocation is one of the
strengths/weaknesses, features/bugs, .... This is still in mostly
ignorance about reference counting and basically multithreading and a
processor's real single thread of execution, the active node on the
flowgraph of the flow of control. With the threading being basically
non-portable, although to a large extent the thread and lock functions
have translations in each threading environment, global variables are
to be avoided like the plague, except in a framework where they're
planned with regards to a portable thread-safe or thread-friendly
design.
So anyways, towards implementing a growable array, I don't want to
implement a heavy function for each time the array context is
initialized for a default memory block size, and for portability's sake
on compilation that value will vary anyways. It could be heuristically
determined by a program at compile time and emitted to a port.h type
file, or maybe just:
#define DEFAULT_BLOCK_SIZE 16384
Advice is appreciate, madvise() might be.
Warm regards,
Ross F.