By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
445,871 Members | 1,209 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 445,871 IT Pros & Developers. It's quick & easy.

shared memory pointer

P: n/a
Tim
Hello Everyone,

I am getting shared memory in python using the following.

szName = c_char_p(name)
hMapObject = windll.kernel32.CreateFileMappingA(INVALID_HANDLE_ VALUE,
None, PAGE_READONLY, 0, TABLE_SHMEMSIZE, szName)
if (hMapObject == 0):
print "OpenKey: Could not open name file mapping object"
raise WinError()

self.pData = windll.kernel32.MapViewOfFile(hMapObject,
FILE_MAP_ALL_ACCESS, 0, 0, TABLE_SHMEMSIZE)

This seems to work OK. How do I get to the data in self.pData? It is
suppose to be an array of floats. I think MapViewOfFile returns a
point to the data. Here is what I tried.

current_time = 45.55
memcpy( current_time, self.pData, 4 )

but I get an error:

ArgumentError: argument 1: <type 'exceptions.TypeError'>: Don't know
how to convert parameter 1

I need to get my floating point array casted from the self.pData. Any
help would be appreciated!

Thanks
Tim

Sep 10 '07 #1
Share this Question
Share on Google+
5 Replies


P: n/a
Tim wrote:
Hello Everyone,

I am getting shared memory in python using the following.

szName = c_char_p(name)
hMapObject = windll.kernel32.CreateFileMappingA(INVALID_HANDLE_ VALUE,
None, PAGE_READONLY, 0, TABLE_SHMEMSIZE, szName)
if (hMapObject == 0):
print "OpenKey: Could not open name file mapping object"
raise WinError()

self.pData = windll.kernel32.MapViewOfFile(hMapObject,
FILE_MAP_ALL_ACCESS, 0, 0, TABLE_SHMEMSIZE)
Without answering your question directly, why not use
the stdlib mmap module, which does exactly this for
you behind the scenes?

Docs - http://docs.python.org/lib/module-mmap.html
Example - http://aspn.activestate.com/ASPN/Coo.../Recipe/413807

TJG
Sep 10 '07 #2

P: n/a
Tim
On Sep 10, 10:11 am, Tim Golden <m...@timgolden.me.ukwrote:
Tim wrote:
Hello Everyone,
I am getting shared memory in python using the following.
szName = c_char_p(name)
hMapObject = windll.kernel32.CreateFileMappingA(INVALID_HANDLE_ VALUE,
None, PAGE_READONLY, 0, TABLE_SHMEMSIZE, szName)
if (hMapObject == 0):
print "OpenKey: Could not open name file mapping object"
raise WinError()
self.pData = windll.kernel32.MapViewOfFile(hMapObject,
FILE_MAP_ALL_ACCESS, 0, 0, TABLE_SHMEMSIZE)

Without answering your question directly, why not use
the stdlib mmap module, which does exactly this for
you behind the scenes?

Docs -http://docs.python.org/lib/module-mmap.html
Example -http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/413807

TJG
I saw the mmap function in the shared memory example. I had some
concern with my large memory size being written to the disk drive. I
though it might slow down my application. The reason I am writting
this new code is because the existing method using a file. I thought
shared memory would be much faster.

I have tried using the memcpy and addresses, casting, etc. Is there a
way to use memcpy with the pData returned by MapViewOfFile()?

Thanks
Tim
Sep 10 '07 #3

P: n/a
Tim
On Sep 10, 10:11 am, Tim Golden <m...@timgolden.me.ukwrote:
Tim wrote:
Hello Everyone,
I am getting shared memory in python using the following.
szName = c_char_p(name)
hMapObject = windll.kernel32.CreateFileMappingA(INVALID_HANDLE_ VALUE,
None, PAGE_READONLY, 0, TABLE_SHMEMSIZE, szName)
if (hMapObject == 0):
print "OpenKey: Could not open name file mapping object"
raise WinError()
self.pData = windll.kernel32.MapViewOfFile(hMapObject,
FILE_MAP_ALL_ACCESS, 0, 0, TABLE_SHMEMSIZE)

Without answering your question directly, why not use
the stdlib mmap module, which does exactly this for
you behind the scenes?

Docs -http://docs.python.org/lib/module-mmap.html
Example -http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/413807

TJG
I reviewed the mmap function and I have a question. In the example
code below, what is the connection between the data in shared memory
and the mmap function. The fileno is zero. Why is it zero? The size
makes sense because there is 256 bytes in shared memory. The tag is
MyFileMappingObject. This tag does not have anything in common with
the shared memory. How can mmap point to any data in shared memory?
There is no coorelation.
pBuf = windll.kernel32.MapViewOfFile(hMapObject, FILE_MAP_ALL_ACCESS,
0, 0, SHMEMSIZE)
if (pBuf == 0):
print "Could not map view of file"
raise WinError()
else:
memcpy = cdll.msvcrt.memcpy
memcpy(pBuf, szMsg, len(szMsg))

shmem = mmap.mmap(0, 256, "MyFileMappingObject", mmap.ACCESS_WRITE)
shmem.write("Message from Python process")

Sep 10 '07 #4

P: n/a
MC
Hi!

I agree ; on windows mmap use Memory-Mapped-file, who use virtual
memory. And shared memory use physical memory.

The difference is OS an not Python


--
@-salutations

Michel Claveau
Sep 10 '07 #5

P: n/a
Tim wrote:
I saw the mmap function in the shared memory example. I had some
concern with my large memory size being written to the disk drive. I
though it might slow down my application. The reason I am writting
this new code is because the existing method using a file. I thought
shared memory would be much faster.

I have tried using the memcpy and addresses, casting, etc. Is there a
way to use memcpy with the pData returned by MapViewOfFile()?
Tim, let's try to back up on this a moment. As I understand, you're
running under Windows and you want to use shared memory, presumably
between two processes. You know about the CreateFileMapping/MapViewOfFile
API, possibly from this example on MSDN [1] and you want to use that
technique in Windows *without* an explicit file backing.

Am I right so far?

I suggested the mmap module, and you seemed to pick up on it
and be trying to use both your ctypes solution *and* the mmap
module as two halves of the same mechanism. Maybe I misunderstood,
but that's what it looked like.

Then you asked a question about getting hold of a Python object's
memory address to be able to pass it into your ctypes solution.
(Which makes sense, given the nature of that solution).

What you seem to have missed is that your ctypes code is doing
*exactly* what the mmapmodule.c code (which is the implementation
of the mmap module) is doing for you behind the scenes.

Here's your code (very slightly reformatted):

<code>
szName = c_char_p(name)
hMapObject = windll.kernel32.CreateFileMappingA(
INVALID_HANDLE_VALUE,
None,
PAGE_READONLY,
0,
TABLE_SHMEMSIZE,
szName
)
if (hMapObject == 0):
print "OpenKey: Could not open name file mapping object"
raise WinError()

self.pData = windll.kernel32.MapViewOfFile(
hMapObject,
FILE_MAP_ALL_ACCESS,
0,
0,
TABLE_SHMEMSIZE
)

</code>

and here's the code from mmapmodule.c (also reformatted
and snipped about):

<code>
m_obj->map_handle = CreateFileMapping(
m_obj->file_handle,
NULL,
flProtect,
size_hi,
size_lo,
m_obj->tagname
);
if (m_obj->map_handle != NULL) {
m_obj->data = (char *) MapViewOfFile(
m_obj->map_handle,
dwDesiredAccess,
0,
0,
0
);

</code>

I hope you can see that they're basically doing the same
thing. (given the appropriate parameters). The only
clear difference is that the final param to MapViewOfFile
is 0 in the Python code, which the MSDN docs[2] indicate
"If this parameter is 0 (zero), the mapping extends
from the specified offset to the end of the file mapping."
It's not clear from that how it applies to a non-file-backed
FileMapping, but I *assume* that the Python devs have tested
that out.

In short, to have two Python processes talk via shared
memory (not linking to a specific file) the following
works for me:

<code-a>
import mmap

#
# The 0 special file value can be -1 in Python 2.5
#
shmem = mmap.mmap (0, 1000, "TJG", mmap.ACCESS_WRITE)
shmem.write ("blah blah")
</code-a>

<code-b>
import mmap

shmem = mmap.mmap (0, 1000, "TJG", mmap.ACCESS_WRITE)
print shmem.read (9)
</code>

Obviously, I've used dummy codes and values, but since
the .write and .read (and the other helper methods)
use strings, you can always pickle or marshal arbitrary
Python data to move it around.

I hope all that's helpful; if nothing else, it's given
me some exercise in reading the code of the stdlib, which
can't be bad!

TJG

[1] http://msdn2.microsoft.com/en-us/library/aa366551.aspx
[2] http://msdn2.microsoft.com/en-us/library/aa366761.aspx
Sep 11 '07 #6

This discussion thread is closed

Replies have been disabled for this discussion.