STL and shared memory

[Michael Schuler]

The use of STL in shared memory poses a real problem since
(non-smart) pointers are not allowed there.

Is there any solution for containers in shared memory using
smart pointers? Where can I find templates?

[void]

U¿ytkownik Michael Schuler napisa³, On 2004-01-12 15:01:

The use of STL in shared memory poses a real problem since
(non-smart) pointers are not allowed there.

Is there any solution for containers in shared memory using
smart pointers? Where can I find templates?

AFAIR to use shared memory you should use boost library you can find it at:
http://www.boost.org/.
Documentation:
http://www.boost.org/libs/smart_ptr/smart_ptr.htm

Best Regards
Darek Ostolski
--
ERROR 1164 HOW IN THE HELL DID YOU GET HERE

[Philipp Bachmann]

Hello,

take a look to <http://tplusplus.sourceforge.net/> resp. at its source code. It uses special
allocators to place the contents of STL containers into shared memory. It's been presented
at OOPSLA 2003.

Cheers,
Philipp.
"Michael Schuler" <Mi*************@fujitsu-siemens.com> wrote in message news:bt**********@news.fujitsu-siemens.com...

The use of STL in shared memory poses a real problem since
(non-smart) pointers are not allowed there.

Is there any solution for containers in shared memory using
smart pointers? Where can I find templates?

[tom_usenet]

On Mon, 12 Jan 2004 15:01:07 +0100, Michael Schuler
<Mi*************@fujitsu-siemens.com> wrote:

The use of STL in shared memory poses a real problem since
(non-smart) pointers are not allowed there.

Is there any solution for containers in shared memory using
smart pointers? Where can I find templates?

If you can allocate your shared memory at a fixed address in each
process' address space, then you just need to write a custom allocator
to allocate from that memory. But if you can't allocate at a fixed
address, then you have to write a complex allocator that uses "offset"
smart pointers (and references!) that store an offset into the shared
memory rather than a pure pointer, and possibly a "shared memory ID"
or similar if you want to have more than one shared memory area in any
process. This is hard to do in a reliable way since C++ doesn't really
support smart references.

Tom

C++ FAQ: http://www.parashift.com/c++-faq-lite/
C FAQ: http://www.eskimo.com/~scs/C-faq/top.html

[Michael Schuler]

Thanks Tom,

but to my knowledge if I use an allocator, only the items are in shared
memory not the container itself. In my case teh container also must lie
in shared memory to survive a process failure :-(

tom_usenet wrote:

On Mon, 12 Jan 2004 15:01:07 +0100, Michael Schuler
<Mi*************@fujitsu-siemens.com> wrote:

The use of STL in shared memory poses a real problem since
(non-smart) pointers are not allowed there.

Is there any solution for containers in shared memory using
smart pointers? Where can I find templates?

If you can allocate your shared memory at a fixed address in each
process' address space, then you just need to write a custom allocator
to allocate from that memory. But if you can't allocate at a fixed
address, then you have to write a complex allocator that uses "offset"
smart pointers (and references!) that store an offset into the shared
memory rather than a pure pointer, and possibly a "shared memory ID"
or similar if you want to have more than one shared memory area in any
process. This is hard to do in a reliable way since C++ doesn't really
support smart references.

Tom

C++ FAQ: http://www.parashift.com/c++-faq-lite/
C FAQ: http://www.eskimo.com/~scs/C-faq/top.html

[Philipp Bachmann]

Hello,

that's true, but imposes no problem at all, because there's "placement new",
i.e. given we talk about Unix and given there's some magic "shmAllocator<>",
you can write sth like

...
typedef std::map< int,std::less< int >,shmAllocator< int > > shmIntMap;
shm=shmget(shm_key,sizeof(shmIntMap),IPC_CREAT|070 0);
void *const p=(shmIntMap *)shmat(shm,NULL,0);
shmIntMap *m=new(p) shmIntMap;
...

Then, don't forget to manually call the destructor afterwards.

Now, you might ask, o.k., now both the map and the ints contained within the map
go into the shared memory - but what's about the linked structure that establishes
the tree the map internally consists of, i.e. that glues together the container and
its elements? The answer is the "rebind<>" member template class a conforming
allocator has to have.

Cheers,
Philipp.

but to my knowledge if I use an allocator, only the items are in shared
memory not the container itself. In my case teh container also must lie
in shared memory to survive a process failure :-(

Is there any solution for containers in shared memory using
smart pointers? Where can I find templates?

If you can allocate your shared memory at a fixed address in each
process' address space, then you just need to write a custom allocator
to allocate from that memory. But if you can't allocate at a fixed
address, then you have to write a complex allocator that uses "offset"
smart pointers (and references!) that store an offset into the shared
memory rather than a pure pointer, and possibly a "shared memory ID"
or similar if you want to have more than one shared memory area in any
process. This is hard to do in a reliable way since C++ doesn't really
support smart references.

[Jonathan Turkanis]

"void" <chq@nie_spamuj.wp.pl> wrote in message
news:29**************@213.17.192.132...

U¿ytkownik Michael Schuler napisa³, On 2004-01-12 15:01:

The use of STL in shared memory poses a real problem since
(non-smart) pointers are not allowed there.

Is there any solution for containers in shared memory using
smart pointers? Where can I find templates?
AFAIR to use shared memory you should use boost library you can find

it at: http://www.boost.org/.
Documentation:
http://www.boost.org/libs/smart_ptr/smart_ptr.htm

Boost has excellent smart pointers, but does not provide any help with
accessing shared memory.

Jonathan

[Michael Schuler]

Thanks Philipp, good hint,

but I still have the problem with pointers inside the container,
whereas pointers pose problems since the shared memory may have
different addresses in different processes :-(

Philipp Bachmann

Hello,

that's true, but imposes no problem at all, because there's "placement new",
i.e. given we talk about Unix and given there's some magic "shmAllocator<>",
you can write sth like

...
typedef std::map< int,std::less< int >,shmAllocator< int > > shmIntMap;
shm=shmget(shm_key,sizeof(shmIntMap),IPC_CREAT|070 0);
void *const p=(shmIntMap *)shmat(shm,NULL,0);
shmIntMap *m=new(p) shmIntMap;
...

Then, don't forget to manually call the destructor afterwards.

Now, you might ask, o.k., now both the map and the ints contained within the map
go into the shared memory - but what's about the linked structure that establishes
the tree the map internally consists of, i.e. that glues together the container and
its elements? The answer is the "rebind<>" member template class a conforming
allocator has to have.

Cheers,
Philipp.

but to my knowledge if I use an allocator, only the items are in shared
memory not the container itself. In my case teh container also must lie
in shared memory to survive a process failure :-(

Is there any solution for containers in shared memory using
smart pointers? Where can I find templates?

If you can allocate your shared memory at a fixed address in each
process' address space, then you just need to write a custom allocator
to allocate from that memory. But if you can't allocate at a fixed
address, then you have to write a complex allocator that uses "offset"
smart pointers (and references!) that store an offset into the shared
memory rather than a pure pointer, and possibly a "shared memory ID"
or similar if you want to have more than one shared memory area in any
process. This is hard to do in a reliable way since C++ doesn't really
support smart references.

[Philipp Bachmann]

The T++ I've referenced seems to use a shared memory pool of a fixed size,
if I correctly understood its source code. But check this out, please:
<http://allocator.sourceforge.net/>. This looks quite promising, doesn't
it?

Cheers,
Philipp.

but I still have the problem with pointers inside the container,
whereas pointers pose problems since the shared memory may have
different addresses in different processes :-(

that's true, but imposes no problem at all, because there's "placement new",
i.e. given we talk about Unix and given there's some magic "shmAllocator<>",
you can write sth like

...
typedef std::map< int,std::less< int >,shmAllocator< int > > shmIntMap;
shm=shmget(shm_key,sizeof(shmIntMap),IPC_CREAT|070 0);
void *const p=(shmIntMap *)shmat(shm,NULL,0);
shmIntMap *m=new(p) shmIntMap;
...

Then, don't forget to manually call the destructor afterwards.

Now, you might ask, o.k., now both the map and the ints contained within the map
go into the shared memory - but what's about the linked structure that establishes
the tree the map internally consists of, i.e. that glues together the container and
its elements? The answer is the "rebind<>" member template class a conforming
allocator has to have.

but to my knowledge if I use an allocator, only the items are in shared
memory not the container itself. In my case teh container also must lie
in shared memory to survive a process failure :-(
>Is there any solution for containers in shared memory using
>smart pointers? Where can I find templates?

If you can allocate your shared memory at a fixed address in each
process' address space, then you just need to write a custom allocator
to allocate from that memory. But if you can't allocate at a fixed
address, then you have to write a complex allocator that uses "offset"
smart pointers (and references!) that store an offset into the shared
memory rather than a pure pointer, and possibly a "shared memory ID"
or similar if you want to have more than one shared memory area in any
process. This is hard to do in a reliable way since C++ doesn't really
support smart references.

[tom_usenet]

On Tue, 13 Jan 2004 15:05:50 +0100, Michael Schuler
<Mi*************@fujitsu-siemens.com> wrote:

Thanks Philipp, good hint,

but I still have the problem with pointers inside the container,
whereas pointers pose problems since the shared memory may have
different addresses in different processes :-(

Solution 1 (easy if your OS supports it):

Map the shared memory to the same address in every process that
accesses it. Some OSes allow you to at least attempt this, and if it
fails you can bail out.
Solution 2 (a nightmare, not really worth it?):

Assuming your standard library is up to it (the only one I'm aware of
is Dinkumware, Metrowerks might be, libstdc++ and STLport aren't), it
might be possible to write a smart pointer and smart reference that
work using an offset into the shared memory (which will be the same
for every process). You'll obviously have to use those classes for any
other pointers/references you want to put in shared memory. I wrote
some such class templates a while back, with mixed success.
Solution 2 (probably best if you can't map to a fixed address):

Don't use the STL, but your own simpler code. Use explicit offsets,
not pointers.

Tom

C++ FAQ: http://www.parashift.com/c++-faq-lite/
C FAQ: http://www.eskimo.com/~scs/C-faq/top.html

[Michael Schuler]

Thanks. Great answer. I will probably map the shared memory at some
fixed address.

tom_usenet wrote:

On Tue, 13 Jan 2004 15:05:50 +0100, Michael Schuler
<Mi*************@fujitsu-siemens.com> wrote:

Thanks Philipp, good hint,

but I still have the problem with pointers inside the container,
whereas pointers pose problems since the shared memory may have
different addresses in different processes :-(

Solution 1 (easy if your OS supports it):

Map the shared memory to the same address in every process that
accesses it. Some OSes allow you to at least attempt this, and if it
fails you can bail out.
Solution 2 (a nightmare, not really worth it?):

Assuming your standard library is up to it (the only one I'm aware of
is Dinkumware, Metrowerks might be, libstdc++ and STLport aren't), it
might be possible to write a smart pointer and smart reference that
work using an offset into the shared memory (which will be the same
for every process). You'll obviously have to use those classes for any
other pointers/references you want to put in shared memory. I wrote
some such class templates a while back, with mixed success.
Solution 2 (probably best if you can't map to a fixed address):

Don't use the STL, but your own simpler code. Use explicit offsets,
not pointers.

Tom

C++ FAQ: http://www.parashift.com/c++-faq-lite/
C FAQ: http://www.eskimo.com/~scs/C-faq/top.html

[Michael Schuler]

Philipp Bachmann

The T++ I've referenced seems to use a shared memory pool of a fixed size,
if I correctly understood its source code. But check this out, please:
<http://allocator.sourceforge.net/>. This looks quite promising, doesn't
it?

Yes, looks great. Thanks for the hint. Cheers,
Philipp.

but I still have the problem with pointers inside the container,
whereas pointers pose problems since the shared memory may have
different addresses in different processes :-(

that's true, but imposes no problem at all, because there's "placement new",
i.e. given we talk about Unix and given there's some magic "shmAllocator<>",
you can write sth like

...
typedef std::map< int,std::less< int >,shmAllocator< int > > shmIntMap;
shm=shmget(shm_key,sizeof(shmIntMap),IPC_CREAT|070 0);
void *const p=(shmIntMap *)shmat(shm,NULL,0);
shmIntMap *m=new(p) shmIntMap;
...

Then, don't forget to manually call the destructor afterwards.

Now, you might ask, o.k., now both the map and the ints contained within the map
go into the shared memory - but what's about the linked structure that establishes
the tree the map internally consists of, i.e. that glues together the container and
its elements? The answer is the "rebind<>" member template class a conforming
allocator has to have.

but to my knowledge if I use an allocator, only the items are in shared
memory not the container itself. In my case teh container also must lie
in shared memory to survive a process failure :-(

>>Is there any solution for containers in shared memory using
>>smart pointers? Where can I find templates?
>
>If you can allocate your shared memory at a fixed address in each
>process' address space, then you just need to write a custom allocator
>to allocate from that memory. But if you can't allocate at a fixed
>address, then you have to write a complex allocator that uses "offset"
>smart pointers (and references!) that store an offset into the shared
>memory rather than a pure pointer, and possibly a "shared memory ID"
>or similar if you want to have more than one shared memory area in any
>process. This is hard to do in a reliable way since C++ doesn't really
>support smart references.