473,383 Members | 1,846 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,383 software developers and data experts.

shared memory malloc library

Hello,

i need to manage a heap in shared memory.
Does anybody know about a portable (win32+mac+posix) c implementation
for this.

Apr 24 '07 #1
21 8332
In article <11**********************@o40g2000prh.googlegroups .com>,
llothar <ll*****@web.dewrote:
>i need to manage a heap in shared memory.
Does anybody know about a portable (win32+mac+posix) c implementation
for this.
I believe that the win32 interface for shared memory is different
than the posix interface, so as best I know, there is no
implementation that is portable between those three platforms.
There might be an implementation that has conditional coding for
those platforms. But shared memory is not part of standard C
and so is not really appropriate for comp.lang.c; you should ask
in OS-specific newsgroups.

--
"law -- it's a commodity"
-- Andrew Ryan (The Globe and Mail, 2005/11/26)
Apr 24 '07 #2
llothar wrote:
>
i need to manage a heap in shared memory. Does anybody know about
a portable (win32+mac+posix) c implementation for this.
Simply use malloc. It is consistent over all C systems.

--
<http://www.cs.auckland.ac.nz/~pgut001/pubs/vista_cost.txt>
<http://www.securityfocus.com/columnists/423>
<http://www.aaxnet.com/editor/edit043.html>
cbfalconer at maineline.net

--
Posted via a free Usenet account from http://www.teranews.com

Apr 24 '07 #3
CBFalconer <cb********@yahoo.comwrites:
llothar wrote:
>i need to manage a heap in shared memory. Does anybody know about
a portable (win32+mac+posix) c implementation for this.

Simply use malloc. It is consistent over all C systems.
Yes, but it doesn't handle "shared memory", whatever that is.
<OT>Presumably it refers to memory that's shared between processes;
standard C has no such concept.</OT>

--
Keith Thompson (The_Other_Keith) ks***@mib.org <http://www.ghoti.net/~kst>
San Diego Supercomputer Center <* <http://users.sdsc.edu/~kst>
"We must do something. This is something. Therefore, we must do this."
-- Antony Jay and Jonathan Lynn, "Yes Minister"
Apr 24 '07 #4
In article <46***************@yahoo.com>,
CBFalconer <cb********@maineline.netwrote:
>i need to manage a heap in shared memory. Does anybody know about
a portable (win32+mac+posix) c implementation for this.
>Simply use malloc. It is consistent over all C systems.
This is an unhelpful answer. malloc() knows nothing about shared
memory. As another poster pointed out, this can't be done in portable
C, but there may well be libraries with conditional code that work on
the systems in question.

Unix provides shared memory through the system V shm*() functions or
the more natural BSD mmap() function. No doubt win32 provides
something similar. To manage concurrent access to shared memory you
could use any of several synchronisation mechanisms, from atomic
test-and-set in shared memory to heavyweight semaphore system calls.

-- Richard

--
"Consideration shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.
Apr 24 '07 #5
Keith Thompson wrote:
CBFalconer <cb********@yahoo.comwrites:
>llothar wrote:
>>i need to manage a heap in shared memory. Does anybody know
about a portable (win32+mac+posix) c implementation for this.

Simply use malloc. It is consistent over all C systems.

Yes, but it doesn't handle "shared memory", whatever that is.
<OT>Presumably it refers to memory that's shared between
processes; standard C has no such concept.</OT>
However, it may do all that. You can also [OT] arrange a process
to manage the memory.

--
<http://www.cs.auckland.ac.nz/~pgut001/pubs/vista_cost.txt>
<http://www.securityfocus.com/columnists/423>
<http://www.aaxnet.com/editor/edit043.html>
cbfalconer at maineline.net

--
Posted via a free Usenet account from http://www.teranews.com

Apr 24 '07 #6
On 24 Apr 2007 08:00:28 -0700, llothar <ll*****@web.dewrote:
>i need to manage a heap in shared memory.
Does anybody know about a portable (win32+mac+posix) c implementation
for this.
Maybe APR? http://apr.apache.org/docs/apr/1.2/group__apr__shm.html
--
Roland Pibinger
"The best software is simple, elegant, and full of drama" - Grady Booch
Apr 25 '07 #7
I believe that the win32 interface for shared memory is different
than the posix interface, so as best I know, there is no
implementation that is portable between those three platforms.
There might be an implementation that has conditional coding for
those platforms.
Of couse i mean conditional coding.
But shared memory is not part of standard C
and so is not really appropriate for comp.lang.c; you should ask
in OS-specific newsgroups.
Yes. But it is not offtopic here because for me everything that has
to do with plain programming in C belongs here.

And this includes questions about existing libraries for a problem as
in does in
all other comp.lang.xxx newsgroups.

Apr 26 '07 #8
llothar wrote:
>
>>But shared memory is not part of standard C
and so is not really appropriate for comp.lang.c; you should ask
in OS-specific newsgroups.


Yes. But it is not offtopic here because for me everything that has
to do with plain programming in C belongs here.
Shared memory libraries aren't "plain C", they are very platform specific.
And this includes questions about existing libraries for a problem as
in does in
all other comp.lang.xxx newsgroups.
No, it doesn't. You'd get the same response down the hall in the C++ group.

Apart form topicality, you will get better platform specific answers in
a platform specific group, so for your benefit, try those.

--
Ian Collins.
Apr 26 '07 #9
llothar wrote:
>
.... snip ...
>
Yes. But it is not offtopic here because for me everything that
has to do with plain programming in C belongs here.

And this includes questions about existing libraries for a problem
as in does in all other comp.lang.xxx newsgroups.
Yes, it is off-topic here, because the content and implementation
of those other libraries is not presented, or is not in standard C.

--
<http://www.cs.auckland.ac.nz/~pgut001/pubs/vista_cost.txt>
<http://www.securityfocus.com/columnists/423>
<http://www.aaxnet.com/editor/edit043.html>
cbfalconer at maineline.net

--
Posted via a free Usenet account from http://www.teranews.com

Apr 26 '07 #10
llothar wrote:
>But shared memory is not part of standard C
and so is not really appropriate for comp.lang.c; you should ask
in OS-specific newsgroups.

Yes. But it is not offtopic here because for me everything that has
to do with plain programming in C belongs here.
Shared memory isn't "plain programming in C".

We have enough noise already without willfully adding to it.

--
"Who do you serve, and who do you trust?" /Crusade/

Hewlett-Packard Limited Cain Road, Bracknell, registered no:
registered office: Berks RG12 1HN 690597 England

Apr 26 '07 #11
In article <59*************@mid.individual.net>,
Ian Collins <ia******@hotmail.comwrote:
>Shared memory libraries aren't "plain C", they are very platform specific.
They will have to have some system-specific code, but it may be a couple
of lines out of hundreds.

-- Richard
--
"Consideration shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.
Apr 26 '07 #12
Richard Tobin wrote, On 26/04/07 11:59:
In article <59*************@mid.individual.net>,
Ian Collins <ia******@hotmail.comwrote:
>Shared memory libraries aren't "plain C", they are very platform specific.

They will have to have some system-specific code, but it may be a couple
of lines out of hundreds.
However, it was about those "couple of lines" that the OP was asking,
and the correct place to get advice on them is in the system specific
groups where people who know about them hang out.
--
Flash Gordon
Apr 26 '07 #13
In article <qq************@news.flash-gordon.me.uk>,
Flash Gordon <sp**@flash-gordon.me.ukwrote:
>However, it was about those "couple of lines" that the OP was asking,
and the correct place to get advice on them is in the system specific
groups where people who know about them hang out.
He was specifically asking for one that worked on a range of systems;
comp.lang.c seems like a reasonable place for cross-platform questions.

-- Richard

--
"Consideration shall be given to the need for as many as 32 characters
in some alphabets" - X3.4, 1963.
Apr 26 '07 #14
In article <f0***********@pc-news.cogsci.ed.ac.uk>,
Richard Tobin <ri*****@cogsci.ed.ac.ukwrote:
>In article <59*************@mid.individual.net>,
Ian Collins <ia******@hotmail.comwrote:
>>Shared memory libraries aren't "plain C", they are very platform specific.
>They will have to have some system-specific code, but it may be a couple
of lines out of hundreds.
And proper use of shared memory requires noticably different code,
written to take into account sychronization points and the fact
that the memory can become invalidated in mid read or mid write
if mistakes are made in the synchronization. Such code also has to
take into account cache effects, and the near lack of atomic
operations in C.

The code that is -obviously- system specific might turn out to be
only a few lines, but the code that isn't -obviously- system specific
may have noticable effects on how the program is structured.
--
All is vanity. -- Ecclesiastes
Apr 26 '07 #15
On 26 Apr 2007 13:49:50 GMT, in comp.lang.c , ri*****@cogsci.ed.ac.uk
(Richard Tobin) wrote:
>In article <qq************@news.flash-gordon.me.uk>,
Flash Gordon <sp**@flash-gordon.me.ukwrote:
>>However, it was about those "couple of lines" that the OP was asking,
and the correct place to get advice on them is in the system specific
groups where people who know about them hang out.

He was specifically asking for one that worked on a range of systems;
comp.lang.c seems like a reasonable place for cross-platform questions.
And he got an answer "its platform specific, ask again in groups
specialising in the platform of your choice".
So whats the problem ? Remember, being redirected is a valid answer.
--
Mark McIntyre

"Debugging is twice as hard as writing the code in the first place.
Therefore, if you write the code as cleverly as possible, you are,
by definition, not smart enough to debug it."
--Brian Kernighan
Apr 26 '07 #16
On 25 Apr., 17:29, rpbg...@yahoo.com (Roland Pibinger) wrote:
On 24 Apr 2007 08:00:28 -0700, llothar <llot...@web.dewrote:
i need to manage a heap in shared memory.
Does anybody know about a portable (win32+mac+posix) c implementation
for this.

Maybe APR?http://apr.apache.org/docs/apr/1.2/group__apr__shm.html
Thanks but this is just a wrapper around the different OS API's.

I was looking for a malloc/calloc/realloc/free lib that takes an
additional heap parameter and where the heap operates on shared
memory.

The most important difference in the scenario compared to a in-process
heap is that everything needs to be managed relative to a base
directory because the same shared memory in process A is not starting
at the same base adress as in process B. So just taking a usual heap
implementation from somewhere and trying to modify it does not help
very much.
Apr 27 '07 #17
In article <11*********************@t39g2000prd.googlegroups. com>,
llothar <ll*****@web.dewrote:
>I was looking for a malloc/calloc/realloc/free lib that takes an
additional heap parameter and where the heap operates on shared
memory.
>The most important difference in the scenario compared to a in-process
heap is that everything needs to be managed relative to a base
directory because the same shared memory in process A is not starting
at the same base adress as in process B. So just taking a usual heap
implementation from somewhere and trying to modify it does not help
very much.
In that case you are definitely looking for something quite
system specific. POSIX shared memory, for example, has no concept
of a "base directory" (indeed, I don't know of any shared memory
allocator that does use such a concept.)

The shared memory allocation systems that I am familiar with allow
the specification of a virtual address to map to, thus making it
possible for multiple processes to all map the memory to the same
virtual addresses in their address spaces.

I would suggest that you consult system-specific newsgroups to
find out whether they support your "base directory" concept of
shared memory at all.
--
I was very young in those days, but I was also rather dim.
-- Christopher Priest
Apr 27 '07 #18
Sorry i mean of course base address not base directory.

On most systems you can't map virtual address to the same address
range as in another process - this will not work realiable. Thats why
you need to do all pointer operations in the heap code relative to the
base adresse which you get when you map in the shared memory. For the
client side of the library all this is invisible, you just need to
pass an additional parameter to malloc/realloc/free and it will be
slower due to the interprocess mutex.

I thought something like this is a common in the C world.
Apr 27 '07 #19
In article <11**********************@u32g2000prd.googlegroups .com>,
llothar <ll*****@web.dewrote:
>Sorry i mean of course base address not base directory.
>On most systems you can't map virtual address to the same address
range as in another process - this will not work realiable. Thats why
you need to do all pointer operations in the heap code relative to the
base adresse which you get when you map in the shared memory.
[OT]
I don't know about Windows, but Unix systems (POSIX, including
MacOS), you can use mmap(), the first parameter to which is the
virtual address in the current process that you wish the memory to
(if you specify MAP_FIXED.) Since mmap() is system specific, one
usually knows enough about the standard memory layout to be able to
choose addresses that will not conflict with other mappings. This
is the same semantics as shmat() with a non-NULL address.

>For the
client side of the library all this is invisible, you just need to
pass an additional parameter to malloc/realloc/free and it will be
slower due to the interprocess mutex.
>I thought something like this is a common in the C world.
The memory so allocated cannot be used to store pointers that are
meaningful to all shared address spaces. In order to communicate the
location of the allocated objects between processes, you would have to
have the C code find the offset of the allocated object relative to the
beginning of the shared heap -- but in C, taking pointer differences
between different C objects is undefined behaviour. The implementation
would therefore not only need to provide malloc() and free()
equivilents in the heap: it would also need to provide a call that
returned a heap-relative offset given a pointer and a heap identifier,
and another call that converted a heap-relative offset and a heap
identifier into a local pointer.

Things are so much easier when you just map into the same virtual
address spaces using POSIX semantics -- changes to the shared segment
are automatically reflected in all attachments, without anything
equivilent to "shared memory flush" needed to update caches. If the
system is multiprocessor (or multi-core) but -not- cache-coherent
between the processors (or cores), it isn't clear to me that the system
would meet the POSIX semantic requirements.

But you still have the same difficulty about C having essentially no
atomic operations, so your program pretty much has to add in reliance
upon synchronization primitives -- and if this hypothetical
malloc-in-a-heap library does not want to require dependancy upon any
particular OS's synchronization primitives, then the library
is effectively going to have to provide synchronization primitives
itself.

Clearly effective operations with such a library isn't as simple as
just providing an additional parameter to *malloc() and kin.

Anyhow, the trials and tribulations of shared memory are not
considered appropriate topics for comp.lang.c, since proper operation
of shared memory inherently relies upon system operations
which C knows nothing about (and does not want to know anything
about, because they are not portable to a number of the systems
that C is expected to be able to run on.)
--
"It is important to remember that when it comes to law, computers
never make copies, only human beings make copies. Computers are given
commands, not permission. Only people can be given permission."
-- Brad Templeton
Apr 27 '07 #20
I don't know about Windows, but Unix systems (POSIX, including
MacOS), you can use mmap(), the first parameter to which is the
virtual address in the current process that you wish the memory to
(if you specify MAP_FIXED.) Since mmap() is system specific, one
You can't assume any range to be unused if the program runs already.
I'm writting the runtime for an eiffel compiler, so i don't know when
the
user opens a heap. And therefore i can't use this method.
The memory so allocated cannot be used to store pointers that are
meaningful to all shared address spaces. In order to communicate the
location of the allocated objects between processes, you would have to
have the C code find the offset of the allocated object relative to the
beginning of the shared heap -- but in C, taking pointer differences
between different C objects is undefined behaviour.
I don't care about this being theoretical impossible in C, in C you
can't also
not cast a data into a function pointer and i've never seen any system
that
has problem. I'm not aware of any non artifical/academic c compiler
that
can't handle this. C is an high-level assember and you should treat it
like this
for many purposes.

I've seen a library like this before and it worked very well.
Unfortunately i forgot
the name and google is not my friend these days. Looks like i have to
do
it myself.

Apr 27 '07 #21
llothar wrote:
>
.... snip ...
>
I've seen a library like this before and it worked very well.
Unfortunately i forgot
the name and google is not my friend these days. Looks like i have to
do
it myself.
If you limit your input line length to 72 (67 is better) you won't
have this sort of line wrap appearing.

--
<http://www.cs.auckland.ac.nz/~pgut001/pubs/vista_cost.txt>
<http://www.securityfocus.com/columnists/423>
<http://www.aaxnet.com/editor/edit043.html>
<http://kadaitcha.cx/vista/dogsbreakfast/index.html>
cbfalconer at maineline dot net

--
Posted via a free Usenet account from http://www.teranews.com

Apr 27 '07 #22

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

46
by: sbayeta | last post by:
Hi, I'd like to know who is responsible of memory recycling and defragmentation in a C/C++ program, assuming all the memory allocation/deallocation is done using malloc/free or new/delete. ...
12
by: gc | last post by:
I am writing a function that given nx1 vector a and a nx1 b solves a system of equations f(a,c)=b for a nx1 c. While writing the function: 1] Should I allocate the memory for c within the...
2
by: Paulo da Silva | last post by:
Hi. I am writing a shared library to override some libc functions. I have the following questions: Ex. int XXX=0; void f() {++XXX; ...; --XXX;}
10
by: eyh5 | last post by:
Hi, My C code (running on Soalris Unix) has some "segmentation fault" that I wish to use purify to do it. I poked around the web, and found some information about adding some lines in a Makefile...
2
by: Mike | last post by:
Hi, I am new to C and having problems with the following program. Basically I am trying to read some files, loading data structures into memory for latter searching. I am trying to use structres...
9
by: Sundar | last post by:
Hi, i am trying to make an application that will require registering of quite a few dlls and execute. Now one of the first bottlenecks that my mentor refused is allocation of memory or the usage...
2
by: oopsatwork | last post by:
Ok...so, I have been outside of the C world for a _very_ long time...but not so long as to remember how to do multidimensional arrays. So, let me state that I know how to malloc pointers to...
2
by: S S | last post by:
Hi I have my shared (.so) library which uses our own memory manager. 32 bit lib was working fine, but when we switched to 64 bit library, as our own defined small pointer can not handle big...
3
by: S S | last post by:
Hi Are there some known issues using STL with shared library. Recently I got some crash for which the reason I dont see any, and searching on goolge shown some such issues with shared library. ...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...
0
by: ryjfgjl | last post by:
In our work, we often need to import Excel data into databases (such as MySQL, SQL Server, Oracle) for data analysis and processing. Usually, we use database tools like Navicat or the Excel import...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.