471,081 Members | 906 Online
Bytes | Software Development & Data Engineering Community
Post +

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 471,081 software developers and data experts.

mmap -- memory mapped file


hi, i'm new to this list and new to python as well.

i have a question on the memory mapped file ability python has. when i
use a mmap on a file, will it copy the whole thing to ram or just
whatever part of it i'm working on? basically, i'm wondering if it would
be ok for me to have multiple mmap's open on very large files as i read
or write from them.
Jun 29 '06 #1
1 2018
Carl Mackey <cj******@seas.upenn.edu> wrote:
hi, i'm new to this list and new to python as well.

i have a question on the memory mapped file ability python has. when i
use a mmap on a file, will it copy the whole thing to ram or just
whatever part of it i'm working on? basically, i'm wondering if it would
be ok for me to have multiple mmap's open on very large files as i read
or write from them.


Python relies on the operating system to do the "memory mapping"
sensibly; generally, that means the whole file is mapped in the
*virtual* memory of the process -- the process's address space -- but
only the necessary pages actually get into RAM. This should work fine
with any modern Linux, *BSD, MacOSX, or just about any other Unix that
is still sold, or, also, on Windows (at least in the NT/2000/XP line, I
wouldn't be so sure on some ancient Windows/98 box:-).

Still, assuming that my "very large files" you mean several gigabytes,
mmap will not work well on them with a 32-bit machine, or a 64-bit
machine hobbled by a 32-bit operating system -- the process's address
space being limited to no more 4GB, you may be unable to mmap even just
one "very large file" (this is totally unrelated to how much RAM you may
have: the limitation is with the *address space* of each process).
Alex
Jun 29 '06 #2

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

26 posts views Thread by myeates | last post: by
13 posts views Thread by George Sakkis | last post: by
1 post views Thread by sam_cit | last post: by
1 post views Thread by James T. Dennis | last post: by
5 posts views Thread by Matias Surdi | last post: by
8 posts views Thread by Unknown Soldier | last post: by
6 posts views Thread by osnot | last post: by
6 posts views Thread by castironpi | last post: by
reply views Thread by leo001 | last post: by

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.