473,372 Members | 1,062 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,372 software developers and data experts.

shared memory pointer

Tim
Hello Everyone,

I am getting shared memory in python using the following.

szName = c_char_p(name)
hMapObject = windll.kernel32.CreateFileMappingA(INVALID_HANDLE_ VALUE,
None, PAGE_READONLY, 0, TABLE_SHMEMSIZE, szName)
if (hMapObject == 0):
print "OpenKey: Could not open name file mapping object"
raise WinError()

self.pData = windll.kernel32.MapViewOfFile(hMapObject,
FILE_MAP_ALL_ACCESS, 0, 0, TABLE_SHMEMSIZE)

This seems to work OK. How do I get to the data in self.pData? It is
suppose to be an array of floats. I think MapViewOfFile returns a
point to the data. Here is what I tried.

current_time = 45.55
memcpy( current_time, self.pData, 4 )

but I get an error:

ArgumentError: argument 1: <type 'exceptions.TypeError'>: Don't know
how to convert parameter 1

I need to get my floating point array casted from the self.pData. Any
help would be appreciated!

Thanks
Tim

Sep 10 '07 #1
5 5247
Tim wrote:
Hello Everyone,

I am getting shared memory in python using the following.

szName = c_char_p(name)
hMapObject = windll.kernel32.CreateFileMappingA(INVALID_HANDLE_ VALUE,
None, PAGE_READONLY, 0, TABLE_SHMEMSIZE, szName)
if (hMapObject == 0):
print "OpenKey: Could not open name file mapping object"
raise WinError()

self.pData = windll.kernel32.MapViewOfFile(hMapObject,
FILE_MAP_ALL_ACCESS, 0, 0, TABLE_SHMEMSIZE)
Without answering your question directly, why not use
the stdlib mmap module, which does exactly this for
you behind the scenes?

Docs - http://docs.python.org/lib/module-mmap.html
Example - http://aspn.activestate.com/ASPN/Coo.../Recipe/413807

TJG
Sep 10 '07 #2
Tim
On Sep 10, 10:11 am, Tim Golden <m...@timgolden.me.ukwrote:
Tim wrote:
Hello Everyone,
I am getting shared memory in python using the following.
szName = c_char_p(name)
hMapObject = windll.kernel32.CreateFileMappingA(INVALID_HANDLE_ VALUE,
None, PAGE_READONLY, 0, TABLE_SHMEMSIZE, szName)
if (hMapObject == 0):
print "OpenKey: Could not open name file mapping object"
raise WinError()
self.pData = windll.kernel32.MapViewOfFile(hMapObject,
FILE_MAP_ALL_ACCESS, 0, 0, TABLE_SHMEMSIZE)

Without answering your question directly, why not use
the stdlib mmap module, which does exactly this for
you behind the scenes?

Docs -http://docs.python.org/lib/module-mmap.html
Example -http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/413807

TJG
I saw the mmap function in the shared memory example. I had some
concern with my large memory size being written to the disk drive. I
though it might slow down my application. The reason I am writting
this new code is because the existing method using a file. I thought
shared memory would be much faster.

I have tried using the memcpy and addresses, casting, etc. Is there a
way to use memcpy with the pData returned by MapViewOfFile()?

Thanks
Tim
Sep 10 '07 #3
Tim
On Sep 10, 10:11 am, Tim Golden <m...@timgolden.me.ukwrote:
Tim wrote:
Hello Everyone,
I am getting shared memory in python using the following.
szName = c_char_p(name)
hMapObject = windll.kernel32.CreateFileMappingA(INVALID_HANDLE_ VALUE,
None, PAGE_READONLY, 0, TABLE_SHMEMSIZE, szName)
if (hMapObject == 0):
print "OpenKey: Could not open name file mapping object"
raise WinError()
self.pData = windll.kernel32.MapViewOfFile(hMapObject,
FILE_MAP_ALL_ACCESS, 0, 0, TABLE_SHMEMSIZE)

Without answering your question directly, why not use
the stdlib mmap module, which does exactly this for
you behind the scenes?

Docs -http://docs.python.org/lib/module-mmap.html
Example -http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/413807

TJG
I reviewed the mmap function and I have a question. In the example
code below, what is the connection between the data in shared memory
and the mmap function. The fileno is zero. Why is it zero? The size
makes sense because there is 256 bytes in shared memory. The tag is
MyFileMappingObject. This tag does not have anything in common with
the shared memory. How can mmap point to any data in shared memory?
There is no coorelation.
pBuf = windll.kernel32.MapViewOfFile(hMapObject, FILE_MAP_ALL_ACCESS,
0, 0, SHMEMSIZE)
if (pBuf == 0):
print "Could not map view of file"
raise WinError()
else:
memcpy = cdll.msvcrt.memcpy
memcpy(pBuf, szMsg, len(szMsg))

shmem = mmap.mmap(0, 256, "MyFileMappingObject", mmap.ACCESS_WRITE)
shmem.write("Message from Python process")

Sep 10 '07 #4
MC
Hi!

I agree ; on windows mmap use Memory-Mapped-file, who use virtual
memory. And shared memory use physical memory.

The difference is OS an not Python


--
@-salutations

Michel Claveau
Sep 10 '07 #5
Tim wrote:
I saw the mmap function in the shared memory example. I had some
concern with my large memory size being written to the disk drive. I
though it might slow down my application. The reason I am writting
this new code is because the existing method using a file. I thought
shared memory would be much faster.

I have tried using the memcpy and addresses, casting, etc. Is there a
way to use memcpy with the pData returned by MapViewOfFile()?
Tim, let's try to back up on this a moment. As I understand, you're
running under Windows and you want to use shared memory, presumably
between two processes. You know about the CreateFileMapping/MapViewOfFile
API, possibly from this example on MSDN [1] and you want to use that
technique in Windows *without* an explicit file backing.

Am I right so far?

I suggested the mmap module, and you seemed to pick up on it
and be trying to use both your ctypes solution *and* the mmap
module as two halves of the same mechanism. Maybe I misunderstood,
but that's what it looked like.

Then you asked a question about getting hold of a Python object's
memory address to be able to pass it into your ctypes solution.
(Which makes sense, given the nature of that solution).

What you seem to have missed is that your ctypes code is doing
*exactly* what the mmapmodule.c code (which is the implementation
of the mmap module) is doing for you behind the scenes.

Here's your code (very slightly reformatted):

<code>
szName = c_char_p(name)
hMapObject = windll.kernel32.CreateFileMappingA(
INVALID_HANDLE_VALUE,
None,
PAGE_READONLY,
0,
TABLE_SHMEMSIZE,
szName
)
if (hMapObject == 0):
print "OpenKey: Could not open name file mapping object"
raise WinError()

self.pData = windll.kernel32.MapViewOfFile(
hMapObject,
FILE_MAP_ALL_ACCESS,
0,
0,
TABLE_SHMEMSIZE
)

</code>

and here's the code from mmapmodule.c (also reformatted
and snipped about):

<code>
m_obj->map_handle = CreateFileMapping(
m_obj->file_handle,
NULL,
flProtect,
size_hi,
size_lo,
m_obj->tagname
);
if (m_obj->map_handle != NULL) {
m_obj->data = (char *) MapViewOfFile(
m_obj->map_handle,
dwDesiredAccess,
0,
0,
0
);

</code>

I hope you can see that they're basically doing the same
thing. (given the appropriate parameters). The only
clear difference is that the final param to MapViewOfFile
is 0 in the Python code, which the MSDN docs[2] indicate
"If this parameter is 0 (zero), the mapping extends
from the specified offset to the end of the file mapping."
It's not clear from that how it applies to a non-file-backed
FileMapping, but I *assume* that the Python devs have tested
that out.

In short, to have two Python processes talk via shared
memory (not linking to a specific file) the following
works for me:

<code-a>
import mmap

#
# The 0 special file value can be -1 in Python 2.5
#
shmem = mmap.mmap (0, 1000, "TJG", mmap.ACCESS_WRITE)
shmem.write ("blah blah")
</code-a>

<code-b>
import mmap

shmem = mmap.mmap (0, 1000, "TJG", mmap.ACCESS_WRITE)
print shmem.read (9)
</code>

Obviously, I've used dummy codes and values, but since
the .write and .read (and the other helper methods)
use strings, you can always pickle or marshal arbitrary
Python data to move it around.

I hope all that's helpful; if nothing else, it's given
me some exercise in reading the code of the stdlib, which
can't be bad!

TJG

[1] http://msdn2.microsoft.com/en-us/library/aa366551.aspx
[2] http://msdn2.microsoft.com/en-us/library/aa366761.aspx
Sep 11 '07 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Srijit Kumar Bhadra | last post by:
Hello, Here is some sample code with pywin32 build 203 and ctypes 0.9.6. Best regards, /Srijit File: SharedMemCreate_Mutex_win32all.py # This application should be used with...
5
by: Claudio Grondi | last post by:
Background information: --------------------------------- in order to monitor mainboard sensory data as fan speeds, temperatures, applications like SpeedFan http://www.almico.com/speedfan.php or...
3
by: alanrn | last post by:
I would like to start a dialog on how to implement the equivalent functionality of UNIX shared memory in .NET. I work with a factory automation system. The bulk of the system is written in C/C++....
11
by: Michael Schuler | last post by:
The use of STL in shared memory poses a real problem since (non-smart) pointers are not allowed there. Is there any solution for containers in shared memory using smart pointers? Where can I...
14
by: phil_gg04 | last post by:
Dear C++ Experts, Over the last couple of months I have been writing my first program using shared memory. It has been something of an "in-at-the-deep-end" experience, to say the least. At...
7
by: nass | last post by:
hi all, i am running slackware linux and need to use some function that will will enable me to write and read from a shared mem segment.. i am using open() , to open a file, and then use mmap to...
21
by: llothar | last post by:
Hello, i need to manage a heap in shared memory. Does anybody know about a portable (win32+mac+posix) c implementation for this.
5
by: Sune | last post by:
Hi all, I want to make data stored in-memory (not disk) available to several processes. My concern is that poorly written C applications with dangling pointers may(will) damage the data in this...
2
by: S S | last post by:
Hi I have my shared (.so) library which uses our own memory manager. 32 bit lib was working fine, but when we switched to 64 bit library, as our own defined small pointer can not handle big...
4
by: hugo.arregui | last post by:
Hi! I have two struts like that: struct { int num; int num2; struct b arrayOfB; } a;
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
by: aa123db | last post by:
Variable and constants Use var or let for variables and const fror constants. Var foo ='bar'; Let foo ='bar';const baz ='bar'; Functions function $name$ ($parameters$) { } ...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.