473,695 Members | 2,458 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

memory allocators & proper alignment...

Here is some info on a C++ allocator prototype I am working on:

http://groups.google.com/group/comp....eee1f61fdbb52c

Any tried-and-true techniques for calculating the correct alignment of any
C++ type the user can throw at it? For the initial code, I was assuming that
the alignment(T) == sizeof(T)... Now that I am so close to being able to
release this thing, I wanted to be able to stitch up this loose end.

All of the allocators I have worked on were always uses by entities which
were private to a library implementation. .. Any they only needed to align
structures on level-2 cache-line boundaries. So, I didn't need to align for
the type, I only align for the cache line.

Now I need to figure out how to get the correct alignment of any C++ type a
user can come up with...

Help!
:^)

--
Chris M. Thomasson
http://appcore.home.comcast.net

May 16 '07 #1
13 3622
"Chris Thomasson" <cr*****@comcas t.netwrote in message
news:mJ******** *************** *******@comcast .com...
Here is some info on a C++ allocator prototype I am working on:

http://groups.google.com/group/comp....eee1f61fdbb52c
[...]
Now I need to figure out how to get the correct alignment of any C++ type
a user can come up with...
I think I will just go ahead and post the code today or tomorrow.

May 17 '07 #2
Chris Thomasson wrote:
Here is some info on a C++ allocator prototype I am working on:

http://groups.google.com/group/comp....eee1f61fdbb52c
Any tried-and-true techniques for calculating the correct alignment of
any C++ type the user can throw at it? For the initial code, I was
assuming that the alignment(T) == sizeof(T)... Now that I am so close to
being able to release this thing, I wanted to be able to stitch up this
loose end.

All of the allocators I have worked on were always uses by entities
which were private to a library implementation. .. Any they only needed
to align structures on level-2 cache-line boundaries. So, I didn't need
to align for the type, I only align for the cache line.

Now I need to figure out how to get the correct alignment of any C++
type a user can come up with...

Help!
:^)
There is no infallible way to compute that in standard C++. I recall
reading about techniques that almost always work. This article seems to
describe one of those (although it doesn't ring any bells for me).

http://www.monkeyspeak.com/alignment/

john
May 17 '07 #3
On May 17, 11:02 am, "Chris Thomasson" <cris...@comcas t.netwrote:
"Chris Thomasson" <cris...@comcas t.netwrote in message

news:mJ******** *************** *******@comcast .com...
Here is some info on a C++ allocator prototype I am working on:
http://groups.google.com/group/comp..../thread/beeee1...

[...]
Now I need to figure out how to get the correct alignment of any C++ type
a user can come up with...
one way:
template <typename T>
struct alignof_static
{
struct Helper1 { T v; };
struct Helper2 { char c; Helper1 v; };

static const unsigned value = sizeof(Helper2)-sizeof(Helper1) ;
};
template <typename T>
unsigned alignof()
{
return alignof_static< T>::value;
}

#ifdef __GNUC__
struct X
{
int a;
} __attribute__ ((aligned (8)));
#else
typedef int X;
#endif
#include <iostream>
int main()
{
std::cout << alignof<char>() << "\n";
std::cout << alignof<short>( ) << "\n";
std::cout << alignof<int &>() << "\n";
std::cout << alignof<char &>() << "\n";
std::cout << alignof<X>() << "\n";
}

May 17 '07 #4
On May 17, 11:02 am, "Chris Thomasson" <cris...@comcas t.netwrote:
"Chris Thomasson" <cris...@comcas t.netwrote in message

news:mJ******** *************** *******@comcast .com...
Here is some info on a C++ allocator prototype I am working on:
http://groups.google.com/group/comp..../thread/beeee1...

[...]
Now I need to figure out how to get the correct alignment of any C++ type
a user can come up with...

I think I will just go ahead and post the code today or tomorrow.
I'm not sure why my previous attempt at posting this didn't show up -
if this is a second post - apologies in advance.

template <typename T>
struct alignof_static
{
struct Helper1 { T v; };
struct Helper2 { char c; Helper1 v; };

static const unsigned value = sizeof(Helper2)-sizeof(Helper1) ;
};
template <typename T>
unsigned alignof()
{
return alignof_static< T>::value;
}

#ifdef __GNUC__
struct X
{
int a;
} __attribute__ ((aligned (8)));
#else
typedef int X;
#endif
#include <iostream>
int main()
{
std::cout << alignof<char>() << "\n";
std::cout << alignof<short>( ) << "\n";
std::cout << alignof<int &>() << "\n";
std::cout << alignof<char &>() << "\n";
std::cout << alignof<X>() << "\n";
}
May 17 '07 #5
On May 17, 11:34 am, Gianni Mariani <gi3nos...@mari ani.wswrote:
On May 17, 11:02 am, "Chris Thomasson" <cris...@comcas t.netwrote:
"Chris Thomasson" <cris...@comcas t.netwrote in message
news:mJ******** *************** *******@comcast .com...
Here is some info on a C++ allocator prototype I am working on:
>http://groups.google.com/group/comp..../thread/beeee1....
[...]
Now I need to figure out how to get the correct alignment
of any C++ type a user can come up with...
one way:
template <typename T>
struct alignof_static
{
struct Helper1 { T v; };
struct Helper2 { char c; Helper1 v; };
static const unsigned value = sizeof(Helper2)-sizeof(Helper1) ;
Just curious, but why Helper1? I would have expected:

struct Helper { char c; T v ; } ;
static const size_t value = sizeof( Helper ) - sizeof( T ) ;

Word addressed machines will typically require all struct's to
be aligned at word boundaries; only character types are exempt
from the requirement. But since the above seems so obvious, I
presume that there is some reason I haven't seen for not using
it.

--
James Kanze (Gabi Software) email: ja*********@gma il.com
Conseils en informatique orientée objet/
Beratung in objektorientier ter Datenverarbeitu ng
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

May 17 '07 #6
James Kanze wrote:
On May 17, 11:34 am, Gianni Mariani <gi3nos...@mari ani.wswrote:
>On May 17, 11:02 am, "Chris Thomasson" <cris...@comcas t.netwrote:
>>"Chris Thomasson" <cris...@comcas t.netwrote in message
>>news:mJ****** *************** *********@comca st.com...
>>>Here is some info on a C++ allocator prototype I am working on:
>>>http://groups.google.com/group/comp..../thread/beeee1...
>>[...]
Now I need to figure out how to get the correct alignment
of any C++ type a user can come up with...
>one way:
template <typename T>
struct alignof_static
{
struct Helper1 { T v; };
struct Helper2 { char c; Helper1 v; };
static const unsigned value = sizeof(Helper2)-sizeof(Helper1) ;

Just curious, but why Helper1? I would have expected:

struct Helper { char c; T v ; } ;
static const size_t value = sizeof( Helper ) - sizeof( T ) ;
Think about what the expression above does when T = char &.

I attached another version which has two alignof methods - one for
dealing with the reference as a type and the other to deal with the
references and the underlying type as the same.
>
Word addressed machines will typically require all struct's to
be aligned at word boundaries; only character types are exempt
from the requirement. But since the above seems so obvious, I
presume that there is some reason I haven't seen for not using
it.
You can probably do a static assert for this - i.e. safe since it fails
to compile on platforms that don't support it.

template <typename T>
struct noreference
{
typedef T type;
};

template <typename T>
struct noreference<T&>
{
typedef T type;
};

template <typename T>
struct alignof_static
{
struct Helper1 { T v; };
struct Helper2 { char c; Helper1 v; };

static const unsigned value = sizeof(Helper2)-sizeof(Helper1) ;
};

template <typename T>
struct alignof_noref_s tatic
{
static const unsigned value = alignof_static< typename noreference<T>: :type >::value;
};

template <typename T>
unsigned alignof()
{
return alignof_static< T>::value;
}

template <typename T>
unsigned alignof_noref()
{
return alignof_noref_s tatic<T>::value ;
}

#ifdef __GNUC__
struct X
{
int a;
} __attribute__ ((aligned (8)));
#else
typedef int X;
#endif
#include <iostream>
int main()
{
std::cout << alignof<char>() << "\n";
std::cout << alignof<short>( ) << "\n";
std::cout << alignof<int &>() << "\n";
std::cout << alignof<char &>() << " noref=" << alignof_noref<c har &>() << "\n";
std::cout << alignof<X>() << "\n";
}

May 18 '07 #7
Gianni Mariani wrote:
James Kanze wrote:
....

I forgot to mention - I decided to switch news servers after my last
response didn't show for a few hours. So apologies again for double posts.
May 18 '07 #8
Gianni Mariani wrote:
James Kanze wrote:
On May 17, 11:34 am, Gianni Mariani <gi3nos...@mari ani.wswrote:
On May 17, 11:02 am, "Chris Thomasson" <cris...@comcas t.netwrote:
[...]
Now I need to figure out how to get the correct alignment
of any C++ type a user can come up with...
one way:
template <typename T>
struct alignof_static
{
struct Helper1 { T v; };
struct Helper2 { char c; Helper1 v; };
static const unsigned value = sizeof(Helper2)-sizeof(Helper1) ;
Just curious, but why Helper1? I would have expected:
struct Helper { char c; T v ; } ;
static const size_t value = sizeof( Helper ) - sizeof( T ) ;
Think about what the expression above does when T = char &.
Bingo. That's what I hadn't thought of.

I'm not sure that it's all that useful to ask the alignment of
something that the standard says might not occupy memory:-). On
the other hand, for most people, the case is probably more
likely to occur in practice that is the case of a word addressed
machine (unless you're a mainframes programmer, and your company
buys from Unisys). A good library should probably handle both
(i.e. word addressed machines, and the alignment of references)
correctly.

(The simplest solution would probably be to use both techniques,
forcing an error if the results weren't equal. That would
probably handle 99% of the cases, and would cause a compile time
error for the cases it didn't handle. Otherwise, I think that
it should be possible to use some metaprogramming tricks to use
one technique for references, and the other otherwise.)

--
James Kanze (GABI Software) email:ja******* **@gmail.com
Conseils en informatique orientée objet/
Beratung in objektorientier ter Datenverarbeitu ng
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34

May 18 '07 #9
Thank you all for your time and energy!

Humm... This thread might be of interest to this group. I say this simply
because current C++ has no concept of threads... I can allow you to create a
single-threaded allocator in C++, and transform it into a full-blown
scaleable mulit-threaded one:

http://groups.google.com/group/comp....87f9cb2cb62d84
No kidding!
So, all of your single threaded C++ allocators can be transformed into
multi-threaded versions, simply by applying a simple lock-free algorithm I
invented.
Any thoughts? Could the technology/method I invented be of use to you or
your company?
May 21 '07 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

18
6671
by: Tron Thomas | last post by:
Given the following information about memory management in C++: ----- The c-runtime dynamic memory manager (and most other commercial memory managers) has issues with fragmentation similar to a hard drive file system. Over time, the more often use call new/delete or alloc/free, there will be gaps and fragments in the heap. This can lead to inefficient use of available memory, as well as cache-hit inefficiencies.
11
2653
by: aaaaa | last post by:
Hi all, Does anybody know if STLPort or SGI STL standard allocators do memory pooling for the list, map and set? Also I have had a look at the BOOST pool_alloc (to be used as a pooling allocator for lists), but looking into the code it doesn't seem to ever release (to the global ::free) the memory that was once allocated. I can understand that such memory can be re-used if I have another list
6
1512
by: Ares Lagae | last post by:
Hello, I am trying to create a container the stl way, and I have a couple of questions. The code of the container in question can be found at http://www.cs.kuleuven.ac.be/~ares/tmp/array_2_hpp.html It is a 2d array that dynamically allocates its storage. It cannot be resized. It offers element access methods for both sequential access and 2d element access.
13
684
by: sachin_mzn | last post by:
Hi, What is the concept of memory alignment? Is memory alignment differs, If a data type is local to a function or if it is a member of structure or union? How 32 to 64 bit processor afftects the memory alignment?
29
2817
by: K. Jennings | last post by:
I would like to get the result of malloc() such that it is aligned on a given boundary - 8, in this case. Assuming that sizeof(u64) is 8, will u64 *p = malloc(N) ; do the trick in general? Here N is a positive integer, and I am assuming that malloc() succeeds in allocating the chunk of memory specified.
3
2067
by: yasmin | last post by:
I am dealing with internal memory fragmentation issues in my program (C++ program on freeBSD). It involves a large number of fixed-size structures. For example for my records which are typically fixed at 100 bytes, the default allocator allocates 128 bytes (closest power of two) wasting 28 bytes per records and this adds up to a significant amount over several thousand records. Before jumping in and writing my own custom allocator which...
2
3793
by: somenath | last post by:
Hi All, I have one question regarding the alignment of pointer returned by malloc. In K&R2 page number 186 one union is used to enforce the alignment as mentioned bellow. typedef long Align; union header { struct {
31
1671
by: Chris Thomasson | last post by:
How many C compilers provide extensions which allow for a standard implementation of the following hack? ____________________________________________________________________ #include <stdio.h> typedef union aligner_types_u aligner_types; typedef struct aligner_offset_s aligner_offset;
12
2993
by: David Given | last post by:
I have a situation where I need to be able to allocate chunks on unmapped virtual memory. Because the memory I'm allocating isn't mapped, I can't use a normal memory allocator, as most of them want to store their metadata about the free list in the unused holes in the heap. Instead, I need a memory allocator that stores its metadata out-of-line (perversely enough, a simple malloc() heap would be fine for that). Does anyone know of...
0
8998
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
8854
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
8831
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
6494
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5841
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
4345
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
3013
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2278
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
1979
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.