470,833 Members | 1,463 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 470,833 developers. It's quick & easy.

Question on use of "placement" new

Hi,

We have a class whose objects are to be allocated in shared memory. To
do that a ShmManager base class is defined so that operator new and
delete are redefined to allocate segments in shared memory. A typical
class "Foo" then inherit from ShmManager to have get this behaviour.
class ShmManager {

void* operator new(size_t size);
void operator delete(void* p, size_t size);

};

class Foo : public ShmManager
{

int fData1;
Barr fData2[16];

};

In the previous Foo example, the size of fData2 array is known at
compilation time, but we need to make this size "dynamic", but keeping
the object memory layout "flat".

We are using the "placement" new syntax doing:

class Foo1 : public ShmManager
{

int fData1;
Barr fData2[]; // will be extented using "placement" new

};

ShmManager* ptr = ShmManager::operator new(sizeof(Foo1) + num *
sizeof(Barr));
Foo1* = new(ptr) Foo1();

So that Foo1 object nows gets a dynamic "num" number of elements. This
seems to work, but is this safe to do that? Are we obliged to put the
fData2 fied as the *last* element in the Foo1? Is there any better/
safer manner to implement the same pattern?

Thanks

Stéphane Letz
Jun 27 '08 #1
11 1548
le**@grame.fr wrote in news:dd732c4c-3cd7-4319-ae60-be2cb2a487a6
@d77g2000hsb.googlegroups.com:
Hi,

We have a class whose objects are to be allocated in shared memory. To
do that a ShmManager base class is defined so that operator new and
delete are redefined to allocate segments in shared memory. A typical
class "Foo" then inherit from ShmManager to have get this behaviour.
class ShmManager {

void* operator new(size_t size);
void operator delete(void* p, size_t size);

};

class Foo : public ShmManager
{

int fData1;
Barr fData2[16];

};

In the previous Foo example, the size of fData2 array is known at
compilation time, but we need to make this size "dynamic", but keeping
the object memory layout "flat".
I guess with this you meant to say that e.g. std::vector is outlawed.
>
We are using the "placement" new syntax doing:

class Foo1 : public ShmManager
{

int fData1;
Barr fData2[]; // will be extented using "placement" new
This is not valid C++. This is valid syntax in C and some C++ compilers
might accept this as an extension. AFAIK it is not planned to include
this feature in C++0x as in C++ there are better means for dynamically
sized arrays (std::vector, ...). Thus in C++ you are obliged to declare
it as Barr fData2[1]; however, accessing elements other than 0 is
undefined behavior (which might happen to be defined by your particular
implementation). In practice I suspect that your code works on most
mainstream implementations, but this is not guaranteed by the standard.
};

ShmManager* ptr = ShmManager::operator new(sizeof(Foo1) + num *
sizeof(Barr));
Foo1* = new(ptr) Foo1();
Foo1* p_foo1 = new(ptr) Foo1();

>
So that Foo1 object nows gets a dynamic "num" number of elements. This
seems to work, but is this safe to do that? Are we obliged to put the
fData2 fied as the *last* element in the Foo1?
Sure. And better not derive from that class.
Is there any better/
safer manner to implement the same pattern?
Using C?

One option would be to let the fData2 out of the class definition,
allocate memory as you do now, and obtain the Barr array pointer by
something like:

Barr* barr = static_cast<Barr*>( static_cast<void*>(p_foo1+1));

Provided that the alignment requirements for Barr are the same or less
stringent than for Foo1, I think this should be kosher, I'm sure others
will correct me if I'm wrong.

hth
Paavo
Jun 27 '08 #2
On 17 Maj, 18:15, l...@grame.fr wrote:
Hi,

We have a class whose objects are to be allocated in shared memory. To
do that a ShmManager base class is defined so that operator new and
delete are redefined to allocate segments in shared memory. A typical
class "Foo" then inherit from ShmManager to have get this behaviour.

class ShmManager {

* * * * void* operator new(size_t size);
* * * * void operator delete(void* p, size_t size);

};

class Foo : public ShmManager
{

* * * * * int fData1;
* * * * * Barr fData2[16];

};

In the previous Foo example, the size of fData2 array is known at
compilation time, but we need to make this size "dynamic", but keeping
the object memory layout "flat".
Why must it be flat? You can't do that portably in C++ and there are
potential problems here - e.g. if Barrs alignment is not compatible
with that of int.
I will assume that a solution where fData2 is a pointer to memory not
adjacent to fData1.
>
We are using the "placement" new syntax doing:

class Foo1 : public ShmManager
{

* * * * * int fData1;
* * * * * Barr fData2[]; *// will be extented using "placement" new

};

ShmManager* ptr = ShmManager::operator new(sizeof(Foo1) + num *
sizeof(Barr));
Foo1* = new(ptr) Foo1();
There is no reason to inherit from ShmManager here. This would have
been necessary only in the case where you would be able to new Foo1
directly.
>
So that Foo1 object nows gets a dynamic "num" number of elements. This
seems to work, but is this safe to do that? Are we obliged to put the
fData2 fied as the *last* element in the Foo1? Is there any better/
safer manner to implement the same pattern?
There are problems:
* your code will have 16 calls to the Barr constructor and this is
only correct when num is 16. To solve this problem, you need to extend
your code.
* The alignment problems mentioned before.
* The nonportability. The code above is not valid C++, and you might
get into troubles for that.

If the requirement was that only the fData2 elements would have to be
contigious, much better solutions exist. Your solution is nonportable
and fragile, but sometimes you just might have to live with that.

/Peter
Jun 27 '08 #3
On 17 mai, 19:49, peter koch <peter.koch.lar...@gmail.comwrote:
On 17 Maj, 18:15, l...@grame.fr wrote:
Hi,
We have a class whose objects are to be allocated in shared memory. To
do that a ShmManager base class is defined so that operator new and
delete are redefined to allocate segments in shared memory. A typical
class "Foo" then inherit from ShmManager to have get this behaviour.
class ShmManager {
* * * * void* operator new(size_t size);
* * * * void operator delete(void* p, size_t size);
};
class Foo : public ShmManager
{
* * * * * int fData1;
* * * * * Barr fData2[16];
};
In the previous Foo example, the size of fData2 array is known at
compilation time, but we need to make this size "dynamic", but keeping
the object memory layout "flat".

Why must it be flat?
Remerber the object lives in shared mem, then having it "flat" greatly
simplify it's use: any process that needs it gets the base object
pointer and can access any field "directly".
You can't do that portably in C++ and there are
potential problems here - e.g. if Barrs alignment is not compatible
with that of int.
I will assume that a solution where fData2 is a pointer to memory not
adjacent to fData1.
If we switch to a solution where fData2 a allocated elsewhere, (that
is in another shared memory segment in our case) then we have to deal
with more complex memory access scheme, again because of the shared
memory issue.
>

If the requirement was that only the fData2 elements would have to be
contigious, much better solutions exist. Your solution is nonportable
and fragile, but sometimes you just might have to live with that.
The whole point of the initial question was: how to allow to have a
dynamic sized array as a field *and* keep the "flat" acess property.
What kind of "much better solutions exist" are you thinking of?

Thanks

Stephane Letz
Jun 27 '08 #4
On 17 Maj, 20:17, l...@grame.fr wrote:
On 17 mai, 19:49, peter koch <peter.koch.lar...@gmail.comwrote:


On 17 Maj, 18:15, l...@grame.fr wrote:
Hi,
We have a class whose objects are to be allocated in shared memory. To
do that a ShmManager base class is defined so that operator new and
delete are redefined to allocate segments in shared memory. A typical
class "Foo" then inherit from ShmManager to have get this behaviour.
class ShmManager {
* * * * void* operator new(size_t size);
* * * * void operator delete(void* p, size_t size);
};
class Foo : public ShmManager
{
* * * * * int fData1;
* * * * * Barr fData2[16];
};
In the previous Foo example, the size of fData2 array is known at
compilation time, but we need to make this size "dynamic", but keeping
the object memory layout "flat".
Why must it be flat?

Remerber the object lives in shared mem, then having it "flat" greatly
simplify it's use: any process that needs it gets the base object
pointer and can access any field "directly".

*You can't do that portably in C++ and there are
potential problems here - e.g. if Barrs alignment is not compatible
with that of int.
I will assume that a solution where fData2 is a pointer to memory not
adjacent to fData1.

If we switch to a solution where fData2 a allocated elsewhere, (that
is in another shared memory segment in our case) then we have to deal
with more complex memory access scheme, again because of the shared
memory issue.
If the requirement was that only the fData2 elements would have to be
contigious, much better solutions exist. Your solution is nonportable
and fragile, but sometimes you just might have to live with that.

The whole point of the initial question was: how to allow to have a
dynamic sized array as a field *and* keep the "flat" acess property.
What kind of "much better solutions exist" are you thinking of?
I believe now that you are in a situation where you have a piece
memory that is shared between different processes but where the adress
of each segment is different from process to process. In that case, I
believe there is not a better solution than what you proposed.
If the adress is the same in each process, I would use a std::vector
with an allocator allocating from ShmMem.

/Peter
Jun 27 '08 #5
On 17 mai, 20:31, peter koch <peter.koch.lar...@gmail.comwrote:
On 17 Maj, 20:17, l...@grame.fr wrote:
On 17 mai, 19:49, peter koch <peter.koch.lar...@gmail.comwrote:
On 17 Maj, 18:15, l...@grame.fr wrote:
Hi,
We have a class whose objects are to be allocated in shared memory. To
do that a ShmManager base class is defined so that operator new and
delete are redefined to allocate segments in shared memory. A typical
class "Foo" then inherit from ShmManager to have get this behaviour.
class ShmManager {
* * * * void* operator new(size_t size);
* * * * void operator delete(void* p, size_t size);
};
class Foo : public ShmManager
{
* * * * * int fData1;
* * * * * Barr fData2[16];
};
In the previous Foo example, the size of fData2 array is known at
compilation time, but we need to make this size "dynamic", but keeping
the object memory layout "flat".
Why must it be flat?
Remerber the object lives in shared mem, then having it "flat" greatly
simplify it's use: any process that needs it gets the base object
pointer and can access any field "directly".
*You can't do that portably in C++ and there are
potential problems here - e.g. if Barrs alignment is not compatible
with that of int.
I will assume that a solution where fData2 is a pointer to memory not
adjacent to fData1.
If we switch to a solution where fData2 a allocated elsewhere, (that
is in another shared memory segment in our case) then we have to deal
with more complex memory access scheme, again because of the shared
memory issue.
If the requirement was that only the fData2 elements would have to be
contigious, much better solutions exist. Your solution is nonportable
and fragile, but sometimes you just might have to live with that.
The whole point of the initial question was: how to allow to have a
dynamic sized array as a field *and* keep the "flat" acess property.
What kind of "much better solutions exist" are you thinking of?

I believe now that you are in a situation where you have a piece
memory that is shared between different processes but where the adress
of each segment is different from process to process. In that case, I
believe there is not a better solution than what you proposed.
If the adress is the same in each process, I would use a std::vector
with an allocator allocating from ShmMem.

/Peter
I finally found this : http://www.devmaster.net/forums/showthread.php?t=11310
that basically does what I what..
.
Any comments?

Thanks

Stephane

Jun 27 '08 #6
On 17 mai, 18:15, l...@grame.fr wrote:
We have a class whose objects are to be allocated in shared memory. To
do that a ShmManager base class is defined so that operator new and
delete are redefined to allocate segments in shared memory. A typical
class "Foo" then inherit from ShmManager to have get this behaviour.
class ShmManager {
void* operator new(size_t size);
void operator delete(void* p, size_t size);
};
class Foo : public ShmManager
{
int fData1;
Barr fData2[16];
};
In the previous Foo example, the size of fData2 array is known
at compilation time, but we need to make this size "dynamic",
but keeping the object memory layout "flat".
Formally, it can't be done. Practically, see below. (I'm
assuming the Barr is a POD type. Otherwise, you'll run into any
number of problems.)
We are using the "placement" new syntax doing:
class Foo1 : public ShmManager
{
int fData1;
Barr fData2[]; // will be extented using "placement" new
};
ShmManager* ptr = ShmManager::operator new(sizeof(Foo1) + num *
sizeof(Barr));
Foo1* = new(ptr) Foo1();
That's legal C99, but not legal C++. In C++, it would cause
some additional problems (e.g. if you inherit from ShmManager).
To date, no one has done the work necessary to solve them, so it
probably won't be adopted into C.

The way I've worked around this in the (distant) past is to
define something like:

class Foo : public ShmManager
{
int fData1 ;
Barr* fData2() { return this + 1 ; }

public:
void* operator new( size_t n, size_t elementCount )
{
return ShmManager::operator new(
n + elementCount * sizeof( Barr ) ) ;
}
} ;

Note, however, that this may create problems with alignment.
In my case, Barr was in fact char, so the problem didn't
occur. If Barr is something more complicated, you'll have to
take additional steps to ensure that sizeof( Foo ) is a multiple
of the alignment needed for Barr.

Note that this requires the client code to use a somewhat
special syntax:

new ( n ) Foo ;

and that it only really works if Barr is a POD (but my
experience is that it's best to stick with POD's in shared
memory anyway).
So that Foo1 object nows gets a dynamic "num" number of
elements. This seems to work, but is this safe to do that?
It's not legal to specify an empty array specifier here, and if
you specify [1], and extend it, you have undefined behavior when
you attempt to access anything but the first element.
Are we obliged to put the fData2 fied as the *last* element in
the Foo1?
Yes. Otherwise, how would the compiler (not knowing n) know how
to find the other elements.
Is there any better/ safer manner to implement the same
pattern?
Even in shared memory, I'd keep variable length arrays separate.
Perhaps using some sort of smart pointer which only stores the
offset from the beginning of shared memory, and uses a global
variable (in the non-shared memory of each process) to calculate
the real address.

--
James Kanze (GABI Software) email:ja*********@gmail.com
Conseils en informatique orientée objet/
Beratung in objektorientierter Datenverarbeitung
9 place Sémard, 78210 St.-Cyr-l'École, France, +33 (0)1 30 23 00 34
Jun 27 '08 #7
Hi!

Paavo Helde schrieb:
I guess with this you meant to say that e.g. std::vector is outlawed.
Well, boost allows vectors, maps, and other stuff in shared memory:
http://www.boost.org/doc/libs/1_35_0...cess_container

Regards, Frank
Jun 27 '08 #8
Frank Birbacher wrote:
Hi!

Paavo Helde schrieb:
>I guess with this you meant to say that e.g. std::vector is outlawed.

Well, boost allows vectors, maps, and other stuff in shared memory:
http://www.boost.org/doc/libs/1_35_0...cess_container
You don't need boost for that, all you have to do is provide an
appropriate allocator to manage a memory pool in shared memory.

--
Ian Collins.
Jun 27 '08 #9
Hi!

Ian Collins schrieb:
You don't need boost for that, all you have to do is provide an
appropriate allocator to manage a memory pool in shared memory.
Right. My point was: it works, you can use vectors in shared memory. The
OP seems to oppose to this solution. I cannot agree on some fragile
solution using the variable-sized-object-and-int[]-hack (which is
illegal in C++) when there is a cleaner solution (using a vector with
shmem alloc).

Regards, Frank
Jun 27 '08 #10
Frank Birbacher wrote:
Hi!

Ian Collins schrieb:
>You don't need boost for that, all you have to do is provide an
appropriate allocator to manage a memory pool in shared memory.

Right. My point was: it works, you can use vectors in shared memory. The
OP seems to oppose to this solution. I cannot agree on some fragile
solution using the variable-sized-object-and-int[]-hack (which is
illegal in C++) when there is a cleaner solution (using a vector with
shmem alloc).
I agree. The only real taboos with hared memory objects are static data
members and virtual functions.

--
Ian Collins.
Jun 27 '08 #11
On 19 Maj, 01:01, Ian Collins <ian-n...@hotmail.comwrote:
Frank Birbacher wrote:
Hi!
Paavo Helde schrieb:
I guess with this you meant to say that e.g. std::vector is outlawed.
Well, boost allows vectors, maps, and other stuff in shared memory:
http://www.boost.org/doc/libs/1_35_0...ess/quick_guid...

You don't need boost for that, all you have to do is provide an
appropriate allocator to manage a memory pool in shared memory.
This requires the memory pool to have the same adress in every
process. Apparantly this is not the case for the OP.

/Peter
Jun 27 '08 #12

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

10 posts views Thread by Audun Røe | last post: by
8 posts views Thread by Ryan R. Rosario | last post: by
18 posts views Thread by Leslaw Bieniasz | last post: by
5 posts views Thread by Mark P | last post: by
2 posts views Thread by rowe_newsgroups | last post: by
5 posts views Thread by Frederick Gotham | last post: by
reply views Thread by mihailmihai484 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.