473,698 Members | 2,676 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

unable to resize mmap object

Hi folks!

I created an mmap object like so:
--- snip ---
from mmap import mmap,MAP_ANONYM OUS,MAP_PRIVATE
fl = file('/dev/zero','rw')
mm = mmap(fl.fileno( ), 1, MAP_PRIVATE|MAP _ANONYMOUS)
--- snap ---

Now, when I try to resize mm to 10 byte
--- snip ---
mm.resize(10)
--- snap ---
I get an EnvironmentErro r:[Errno 22] Invalid argument.

How can I implement a resizeable anonymous memory mapping?
Thanks for your reply!

F. Sidler
Apr 30 '06 #1
4 3698

Fabiano Sidler wrote:
Hi folks!

I created an mmap object like so:
--- snip ---
from mmap import mmap,MAP_ANONYM OUS,MAP_PRIVATE
fl = file('/dev/zero','rw')
mm = mmap(fl.fileno( ), 1, MAP_PRIVATE|MAP _ANONYMOUS)
--- snap ---

Now, when I try to resize mm to 10 byte
--- snip ---
mm.resize(10)
--- snap ---
I get an EnvironmentErro r:[Errno 22] Invalid argument.


Just a guess: try a new size argument that is multiple of page size.

Apr 30 '06 #2
On Sunday 30 April 2006 21:06, Serge Orlov wrote:
Fabiano Sidler wrote:
Now, when I try to resize mm to 10 byte
--- snip ---
mm.resize(10)
--- snap ---
I get an EnvironmentErro r:[Errno 22] Invalid argument.


Just a guess: try a new size argument that is multiple of page size.


No, doesn't work neitzer. :(

Thank you anyway for the idea!
May 2 '06 #3
Fabiano Sidler wrote:
Hi folks!

I created an mmap object like so:
--- snip ---
from mmap import mmap,MAP_ANONYM OUS,MAP_PRIVATE
fl = file('/dev/zero','rw')
mm = mmap(fl.fileno( ), 1, MAP_PRIVATE|MAP _ANONYMOUS)
--- snap ---

Now, when I try to resize mm to 10 byte
--- snip ---
mm.resize(10)
--- snap ---
I get an EnvironmentErro r:[Errno 22] Invalid argument.

How can I implement a resizeable anonymous memory mapping?
Thanks for your reply!


Bug in Python. Here's the reason (from the changelog):

- Bug #728515: mmap.resize() now resizes the file on Unix as it did
on Windows.

The mmap module resizes the file in question by calling ftruncate.
Only problem is, you can't truncate /dev/zero. I'm going to file this
as a bug; probably on Unix mmap should only resize the file if it's a
regular file (so fstat it first).

Frankly, I'm not so sure matching Windows behavior is a great idea.
mmap module seems to be having an identity crisis. Is it a low-level
system call, or a high-level, OS-independent way to access files as
blocks of memory? The modules is moving towards the latter (what with
Unix mmap accepting ACCESS-style flags, and now this file-resizing
behavior). I actually favor a two-level approach similar to file I/O:
there would low-level system calls in the os module, and high-level
mmap object. (The high-level object would go all the way. It would
accept a filename rather than a file descriptor, anonymous blocks would
be handled OS-independently, rather than mapping /dev/zero, and so on.)

Carl Banks

May 2 '06 #4
Carl Banks wrote:
Frankly, I'm not so sure matching Windows behavior is a great idea.
mmap module seems to be having an identity crisis. Is it a low-level
system call, or a high-level, OS-independent way to access files as
blocks of memory? The modules is moving towards the latter (what with
Unix mmap accepting ACCESS-style flags, and now this file-resizing
behavior). I actually favor a two-level approach similar to file I/O:
there would low-level system calls in the os module, and high-level
mmap object. (The high-level object would go all the way. It would
accept a filename rather than a file descriptor, anonymous blocks would
be handled OS-independently, rather than mapping /dev/zero, and so on.)


I'm sure that we will gladly accept a patch implementing this approach.

Cheers,
Georg
May 4 '06 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

7
3100
by: Michael | last post by:
I'm writing an application that decodes a file containing binary records. Each record is a particular event type. Each record is translated into ASCII and then written to a file. Each file contains the same events. At the moment each record is processed one after the other. It taks about 1m40s to process a large file containing 70,000 records. Would my application benifit from multiple threads and mmap? If so what is the best way to...
26
9313
by: myeates | last post by:
Hi Anyone ever done this? It looks like Python2.4 won't take a length arg Mathew
13
3511
by: George Sakkis | last post by:
I've been trying to track down a memory leak (which I initially attributed erroneously to numpy) and it turns out to be caused by a memory mapped file. It seems that mmap caches without limit the chunks it reads, as the memory usage grows to several hundreds MBs according to the Windows task manager before it dies with a MemoryError. I'm positive that these chunks are not referenced anywhere else; in fact if I change the mmap object to a...
1
2689
by: koara | last post by:
Hello all, i am using the mmap module (python2.4) to access contents of a file. My question regards the relative performance of mmap.seek() vs mmap.tell(). I have a generator that returns stuff from the file, piece by piece. Since other things may happen to the mmap object in between consecutive next() calls (such as another iterator's next()), i have to store the file position before yield and restore it afterwards by means of tell()...
5
6066
by: Matias Surdi | last post by:
Suppose I've a process P1, which generates itself a lot of data , for example 2Mb. Then, I've a process P2 which must access P1 shared memory and, probably, modify this data. To accomplish this, I've been digging around python's mmap module, but I can't figure how to use it without files. Could anybody explain me how could this be accomplished? An example will be very appreciated. Thanks a lot for your help.
2
4629
by: Neal Becker | last post by:
On linux, I don't understand why: f = open ('/dev/eos', 'rw') m = mmap.mmap(f.fileno(), 1000000, prot=mmap.PROT_READ|mmap.PROT_WRITE, flags=mmap.MAP_SHARED) gives 'permission denied', but this c++ code works: #include <sys/mman.h> #include <fcntl.h>
0
1144
by: Kris Kennaway | last post by:
If I do the following: def mmap_search(f, string): fh = file(f) mm = mmap.mmap(fh.fileno(), 0, mmap.MAP_SHARED, mmap.PROT_READ) return mm.find(string) def mmap_is_in(f, string): fh = file(f)
5
4199
by: castironpi | last post by:
Hi all, I have a mmap and a data structure in it. I know the structure's location in the mmap and what structure it is. It has a ctypes definition. I want to initialize a ctypes object to point to a part of the mmap. Here is my attempt:
4
5907
by: keirnus | last post by:
Hello, I've been coding in Access VBA to create an Excel Graph and it was good. Until I got this error: Please check the code below: Private Sub TestGraph3()
0
8685
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9171
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
9032
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
8880
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
0
7743
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
6532
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
4625
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
3053
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
2342
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.