Does any one now if a shared memory module exists, written in python
for a windows platform. I now one exists for unix?
help most appreciated,
S Green 14 24939
S Green wrote: Does any one now if a shared memory module exists, written in python for a windows platform. I now one exists for unix?
Your first message with this same question did in fact make it to
Usenet and the mailing list.
Generally speaking, you should allow several days for answers to questions,
rather than getting anxious just because ten people don't jump up and answer
it in the first few hours.
(And my guess is that one doesn't exist, as Windows shared memory would
probably be a completely unreliable bitch, but that's just my guess. :-)
-Peter st***********@baesystems.com (S Green) writes: Does any one now if a shared memory module exists, written in python for a windows platform. I now one exists for unix?
help most appreciated,
S Green
import mmap
Thomas Heller wrote: st***********@baesystems.com (S Green) writes:
Does any one now if a shared memory module exists, written in python for a windows platform. I now one exists for unix?
help most appreciated,
S Green
import mmap
The docs suggest that mmap.mmap(x, x, mmap.MAP_SHARED) will work under
Unix but not Windows.
Does the Windows version really support shared memory, or is the
multiple-maps-per-file feature only valid within a single process?
-Peter
Peter Hansen <pe***@engcorp.com> writes: Thomas Heller wrote: st***********@baesystems.com (S Green) writes:
> Does any one now if a shared memory module exists, written in python > for a windows platform. I now one exists for unix? > > > help most appreciated, > > S Green
import mmap
The docs suggest that mmap.mmap(x, x, mmap.MAP_SHARED) will work under Unix but not Windows.
Does the Windows version really support shared memory, or is the multiple-maps-per-file feature only valid within a single process?
Yes, it does. You have to give this memory block a name, though:
sharedmem = mmap.mmap(0, 16384, "GlobalSharedMemory")
This allows to access 16384 bytes of memory, shared across processes,
not backed up by a file in the filesystem.
Thomas
Thomas Heller wrote: Peter Hansen <pe***@engcorp.com> writes:
Thomas Heller wrote: st***********@baesystems.com (S Green) writes:
> Does any one now if a shared memory module exists, written in python > for a windows platform. I now one exists for unix? > > > help most appreciated, > > S Green
import mmap
The docs suggest that mmap.mmap(x, x, mmap.MAP_SHARED) will work under Unix but not Windows.
Does the Windows version really support shared memory, or is the multiple-maps-per-file feature only valid within a single process?
Yes, it does. You have to give this memory block a name, though:
sharedmem = mmap.mmap(0, 16384, "GlobalSharedMemory")
This allows to access 16384 bytes of memory, shared across processes, not backed up by a file in the filesystem.
Thanks, Thomas.
In my opinion the documentation on this is entirely unclear. I attach
it for reference, but I can't offer any suggestions for improvement (as
I don't know anything about shared memory) except that even after reading
it a second time, Thomas' answer above is very much news to me:
'''tagname, if specified and not None, is a string giving a tag name
for the mapping. Windows allows you to have many different mappings
against the same file. If you specify the name of an existing tag,
that tag is opened, otherwise a new tag of this name is created. If
this parameter is omitted or None, the mapping is created without a
name. Avoiding the use of the tag parameter will assist in keeping your
code portable between Unix and Windows. '''
(from http://www.python.org/doc/current/lib/module-mmap.html)
-Peter
Thomas Heller wrote: Does the Windows version really support shared memory, or is the multiple-maps-per-file feature only valid within a single process?
Yes, it does. You have to give this memory block a name, though:
sharedmem = mmap.mmap(0, 16384, "GlobalSharedMemory")
This allows to access 16384 bytes of memory, shared across processes, not backed up by a file in the filesystem.
That is très cool, it doesn't tell you this in the docs, does it?
The first argument is 'the file handle' of the file to be mapped,
and it doesn't say that 0 is valid and means 'no file at all'...
However, I've just tried it, and managed to crash Python in mmap.pyd
with an application exception... twice. But trying to reproduce it
now fails-- Python keeps running.
I did this: import mmap mem=mmap.mmap(0,3000000,'irmen') mem[0]='a' mem[2000000]='a'
and initially, it crashed......... Python 2.3.2 on win xp)
--Irmen
Irmen de Jong <irmen@-NOSPAM-REMOVETHIS-xs4all.nl> writes: Thomas Heller wrote:
Does the Windows version really support shared memory, or is the multiple-maps-per-file feature only valid within a single process? Yes, it does. You have to give this memory block a name, though: sharedmem = mmap.mmap(0, 16384, "GlobalSharedMemory") This allows to access 16384 bytes of memory, shared across processes, not backed up by a file in the filesystem.
That is très cool, it doesn't tell you this in the docs, does it? The first argument is 'the file handle' of the file to be mapped, and it doesn't say that 0 is valid and means 'no file at all'...
However, I've just tried it, and managed to crash Python in mmap.pyd with an application exception... twice. But trying to reproduce it now fails-- Python keeps running. I did this: >>> import mmap >>> mem=mmap.mmap(0,3000000,'irmen') >>> mem[0]='a' >>> mem[2000000]='a'
and initially, it crashed......... Python 2.3.2 on win xp)
This works for me even from the beginning (with 2.3 CVS version, XP Pro).
Understading which parameters to pass apparently requires
- reading the mmapmodule.c sources
- and reading about CreateFileMapping and MapViewOfFile in MSDN.
It would be great if someine could submit a patch for the docs <wink>.
I vaguely remember, but am not able to find it anymore: didn't AMK once
had an article about this somewhere?
Thomas
Irmen de Jong wrote: However, I've just tried it, and managed to crash Python in mmap.pyd with an application exception... twice. But trying to reproduce it now fails-- Python keeps running. I did this: >>> import mmap >>> mem=mmap.mmap(0,3000000,'irmen') >>> mem[0]='a' >>> mem[2000000]='a'
and initially, it crashed......... Python 2.3.2 on win xp) --Irmen
Probably not enough actual memory hanging around. Some systems
(and from this I'd guess XP) allocate virtual memory by reserving
address space, not actually allocating the RAM and/or backing
store for that memory. Python has no control over this, and you
have nothing good to do if the memory is over-allocated. When
you create the memory, you could walk across it writing into it
(forcing it to exist), but that would just force the failure to
happen earlier.
-Scott David Daniels Sc***********@Acm.Org
Scott David Daniels wrote: Irmen de Jong wrote:
However, I've just tried it, and managed to crash Python in mmap.pyd with an application exception... twice. But trying to reproduce it now fails-- Python keeps running. I did this: >>> import mmap >>> mem=mmap.mmap(0,3000000,'irmen') >>> mem[0]='a' >>> mem[2000000]='a' and initially, it crashed......... Python 2.3.2 on win xp) --Irmen
Probably not enough actual memory hanging around. Some systems (and from this I'd guess XP) allocate virtual memory by reserving address space, not actually allocating the RAM and/or backing store for that memory.
Over-commit is that called, right?
I'm sorry but that certainly wasn't the case here.
My machine has 512 Mb RAM and about 250 Mb of them
allocated when I tried it. (physical RAM that is)
Very weird, I cannot reproduce the initial crash I experienced.
--Irmen
On Tue, 02 Dec 2003 00:44:15 +0100, Irmen de Jong <irmen@-NOSPAM-REMOVETHIS-xs4all.nl> wrote: Scott David Daniels wrote: Irmen de Jong wrote:
However, I've just tried it, and managed to crash Python in mmap.pyd with an application exception... twice. But trying to reproduce it now fails-- Python keeps running. I did this: >>> import mmap >>> mem=mmap.mmap(0,3000000,'irmen') >>> mem[0]='a' >>> mem[2000000]='a' and initially, it crashed......... Python 2.3.2 on win xp) --Irmen
Probably not enough actual memory hanging around. Some systems (and from this I'd guess XP) allocate virtual memory by reserving address space, not actually allocating the RAM and/or backing store for that memory.
Over-commit is that called, right? I'm sorry but that certainly wasn't the case here. My machine has 512 Mb RAM and about 250 Mb of them allocated when I tried it. (physical RAM that is)
Very weird, I cannot reproduce the initial crash I experienced.
What's that first zero argument?
Snip from mmap docs:
"""
mmap( fileno, length[, tagname[, access]])
(Windows version) Maps length bytes from the file specified by
the file handle fileno, and returns a mmap object. If length is 0,
the maximum length of the map will be the current size of the file
when mmap() is called.
tagname, if specified and not None, is a string giving a tag name
for the mapping. Windows allows you to have many different mappings
against the same file. If you specify the name of an existing tag,
that tag is opened, otherwise a new tag of this name is created.
If this parameter is omitted or None, the mapping is created without a name.
Avoiding the use of the tag parameter will assist in keeping your code portable
between Unix and Windows.
"""
Does that mean you are trying to use stdin as the open file handle?
Regards,
Bengt Richter
Bengt Richter wrote: What's that first zero argument?
[...] Does that mean you are trying to use stdin as the open file handle?
No it doesn't, according to Thomas Heller's post in this thread.
Thomas said 0 means "not associated with a file" on windows...
--Irmen
Irmen de Jong <irmen@-NOSPAM-REMOVETHIS-xs4all.nl> writes: Bengt Richter wrote: What's that first zero argument? [...] Does that mean you are trying to use stdin as the open file handle?
No it doesn't, according to Thomas Heller's post in this thread. Thomas said 0 means "not associated with a file" on windows...
mmapmodule.c internally uses INVALID_HANDLE_VALUE of a file handle of 0
is used.
As I said, read the source (and MSDN). And then submit a documentation
patch <wink>.
Thomas
On Tue, 02 Dec 2003 10:54:19 +0100, Thomas Heller <th*****@python.net> wrote: Irmen de Jong <irmen@-NOSPAM-REMOVETHIS-xs4all.nl> writes:
Bengt Richter wrote: What's that first zero argument? [...] Does that mean you are trying to use stdin as the open file handle?
No it doesn't, according to Thomas Heller's post in this thread. Thomas said 0 means "not associated with a file" on windows...
mmapmodule.c internally uses INVALID_HANDLE_VALUE of a file handle of 0 is used.
Wouldn't passing None be more pythonic for a python interface? As I said, read the source (and MSDN). And then submit a documentation patch <wink>.
How does one do that? (being lazy, I guess I could find out via www.python.org)
Regards,
Bengt Richter bo**@oz.net (Bengt Richter) writes: On Tue, 02 Dec 2003 10:54:19 +0100, Thomas Heller <th*****@python.net> wrote:
Irmen de Jong <irmen@-NOSPAM-REMOVETHIS-xs4all.nl> writes:
Bengt Richter wrote: What's that first zero argument? [...] Does that mean you are trying to use stdin as the open file handle?
No it doesn't, according to Thomas Heller's post in this thread. Thomas said 0 means "not associated with a file" on windows...
mmapmodule.c internally uses INVALID_HANDLE_VALUE of a file handle of 0 is used. Wouldn't passing None be more pythonic for a python interface?
As I said, read the source (and MSDN). And then submit a documentation patch <wink>.
How does one do that? (being lazy, I guess I could find out via www.python.org)
Finding out how to do this is the easier part ;-)
Ok, to get you started:
<http://cvs.sourceforge.net/viewcvs.py/python/python/dist/src/Modules/mmapmodule.c>
<http://msdn.microsoft.com/library/en-us/fileio/base/creating_named_shared_memory.asp>
<http://msdn.microsoft.com/library/en-us/fileio/base/createfilemapping.asp>
<http://msdn.microsoft.com/library/en-us/fileio/base/mapviewoffile.asp>
Thomas This thread has been closed and replies have been disabled. Please start a new discussion. Similar topics
by: Srijit Kumar Bhadra |
last post by:
Hello,
Here is some sample code with pywin32 build 203 and ctypes 0.9.6.
Best regards,
/Srijit
File: SharedMemCreate_Mutex_win32all.py
# This application should be used with...
|
by: alanrn |
last post by:
I would like to start a dialog on how to implement the equivalent
functionality of UNIX shared memory in .NET. I work with a factory
automation system. The bulk of the system is written in C/C++....
|
by: Michael Schuler |
last post by:
The use of STL in shared memory poses a real problem since
(non-smart) pointers are not allowed there.
Is there any solution for containers in shared memory using
smart pointers? Where can I...
|
by: myren, lord |
last post by:
When I first discovered shared memory (between multiple processes) I
immediately started thinking of how to build my own VM subsystem +
locking mechanisms for a large single block of memory. This...
|
by: phil_gg04 |
last post by:
Dear C++ Experts,
Over the last couple of months I have been writing my first program
using shared memory. It has been something of an "in-at-the-deep-end"
experience, to say the least. At...
|
by: Jeremy |
last post by:
Hi all,
I'm getting very confused about how DB2 uses shared memory and I
wonder if someone could clarify matters for me, please ?
We are running 32bit DB2 V7.2 FP9 under AIX 4.3.3 on a machine...
|
by: Jim |
last post by:
Hello,
I have a broken server that we are going to be moving off to a new server
with a new version of DB2 but here is what I have right now:
RedHat 7.0 (2.2.24smp)
DB2 v6.1.0.40
I am...
|
by: Sebastjan Trepca |
last post by:
Hey!
Are there any "live" modules for working with shared memory in Python?
I found some but they are all dead(posh,shm)...
Thanks, Sebastjan
|
by: herbert |
last post by:
I am coding a dozen "background" realtime apps for factory automation in .NET
2.0.
The apps need to share a common memory as there are lots of variables to be
shared (and synchronized of...
|
by: Charles Arthur |
last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
|
by: ryjfgjl |
last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
|
by: ryjfgjl |
last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
|
by: emmanuelkatto |
last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud.
Please let me know.
Thanks!
Emmanuel
|
by: BarryA |
last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
|
by: Sonnysonu |
last post by:
This is the data of csv file
1 2 3
1 2 3
1 2 3
1 2 3
2 3
2 3
3
the lengths should be different i have to store the data by column-wise with in the specific length.
suppose the i have to...
|
by: Oralloy |
last post by:
Hello folks,
I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>".
The problem is that using the GNU compilers,...
|
by: jinu1996 |
last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
|
by: tracyyun |
last post by:
Dear forum friends,
With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each...
| |