memory mapped tar file contents

[Chris Brooks]

Hi,

I would like to read directly from a tar file into memory so I can
manipulate a file (quickly) and write its changes out to another file. I
thought I could do something like:

#!/usr/bin/env python

import tarfile
import mmap

fil = tarfile.open( "out.tar.gz" , "r:gz" )
tarinf = fil.next()
myfils = {}
while tarinf != None:
tarinf = fil.next()
ref = fil.extractfile( tarinf )
myfils[ tarinf.name ] = mmap.mmap( ref.fileno() , 0 )

But the extractfile() function of TarInfo doesn't seem to give me a fileno,
so I can't pass this to mmap.

Thoughts on a way to accomplish this?

Chris
--
View this message in context: http://www.nabble.com/memory-mapped-...p20473925.html
Sent from the Python - python-list mailing list archive at Nabble.com.

[Aaron Brady]

On Nov 12, 8:51*pm, Chris Brooks <[e-mail]:

Hi,

I would like to read directly from a tar file into memory so I can
manipulate a file (quickly) and write its changes out to another file. *I
thought I could do something like:

#!/usr/bin/env python

import tarfile
import mmap

fil = tarfile.open( "out.tar.gz" , "r:gz" )
tarinf = fil.next()
myfils = {}
while tarinf != None:
* * tarinf = fil.next()
* * ref = fil.extractfile( tarinf )
* * myfils[ tarinf.name ] = mmap.mmap( ref.fileno() , 0 )

But the extractfile() function of TarInfo doesn't seem to give me a fileno,
so I can't pass this to mmap.

Thoughts on a way to accomplish this?

It appears you have to read the contents of the file into the mmap.
You can create an anonymous map of size tarinfoobj.size, then set
mapobj[:]= fil.extractfile( tarinf ).read( ) . Untested.