Create TarFile using string buffers

[aurora00]

I have a program that generates a number of files that will be
packaged into a tarball. Can I stream the content into TarFile without
first writing them out to the file system? All add(), addfile() and
gettarinfo() seems to assume there is a file in the disk. But for me I
seems inefficient to write all the content to the disk and then have
it read back by the TarFile module.

Thank you for your help

wy

[Lars =?iso-8859-15?Q?Gust=E4bel?=]

On Mon, Mar 19, 2007 at 12:06:39PM -0700, au******@gmail.com wrote:

I have a program that generates a number of files that will be
packaged into a tarball. Can I stream the content into TarFile without
first writing them out to the file system? All add(), addfile() and
gettarinfo() seems to assume there is a file in the disk. But for me I
seems inefficient to write all the content to the disk and then have
it read back by the TarFile module.

addfile()'s fileobj argument can be anything that has a read()
method. The amount of bytes to read is taken from the
tarinfo.size attribute. You could let your program write its
data to a StringIO object and pass that object to addfile().

http://docs.python.org/lib/tarfile-objects.html

--
Lars Gustäbel
la**@gustaebel.de

Linux is like a wigwam - no Gates, no Windows, Apache inside.

[Gabriel Genellina]

En Mon, 19 Mar 2007 16:06:39 -0300, au******@gmail.com
<au******@gmail.comescribió:

I have a program that generates a number of files that will be
packaged into a tarball. Can I stream the content into TarFile without
first writing them out to the file system? All add(), addfile() and
gettarinfo() seems to assume there is a file in the disk. But for me I
seems inefficient to write all the content to the disk and then have
it read back by the TarFile module.

You can create a TarInfo object directly, and use addfile with a StringIO
object:

import tarfile
from cStringIO import StringIO

data = "Some text, maybe containing\ntwo or more lines\n" + "The quick
brown fox jumps over the lazy dog\n" * 20
fobj = StringIO(data)

tar = tarfile.open("sample.tar", "w")
tarinfo = tarfile.TarInfo("foo.txt")
tarinfo.size = len(data)
tar.addfile(tarinfo, fobj)
tar.close()

tar = tarfile.open("sample.tar", "r")
tar.list()
foo = tar.extractfile("foo.txt")
data_read = foo.read()
print "foo.txt:\n%s" % data_read
tar.close()
assert data == data_read
--
Gabriel Genellina

[=?iso-8859-1?Q?Andr=E9s?= Martinelli]

I'm using the example of the site
http://docs.python.org/lib/module-struct.html :

import struct
pack('hhl', 1, 2, 3)

I should get:

>>'\x00\x01\x00\x02\x00\x00\x00\x03'

I get an empty line, a '\n':

>>>
_

I know it should work. The code is OK.
What could be wrong? Doesnt find the library module?
I repeat that the unpack works ok.
I use debian etch and sarge. Same situation in both of them.

Thanks.
Andrés M.

[skip]

AndrésI'm using the example of the site
Andréshttp://docs.python.org/lib/module-struct.html :

Andrésimport struct
Andréspack('hhl', 1, 2, 3)

AndrésI should get:

>>>'\x00\x01\x00\x02\x00\x00\x00\x03'

AndrésI get an empty line, a '\n':

>>>
_

Try

import struct
struct.pack('hhl', 1, 2, 3)

or

from struct import pack
pack('hhl', 1, 2, 3)

instead.

Skip

[Gabriel Genellina]

En Mon, 19 Mar 2007 19:32:41 -0300, Andrés Martinelli
<an****@lacordilleraandina.com.arescribió:

I'm using the example of the site
http://docs.python.org/lib/module-struct.html :

import struct
pack('hhl', 1, 2, 3)

That code should raise a NameError. Either you are using *another* pack
function, or that is not what you actually typed.
This is what I get:

Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)]
on win32
Type "help", "copyright", "credits" or "license" for more information.
pyimport struct
pystruct.pack('hhl', 1, 2, 3)
'\x01\x00\x02\x00\x03\x00\x00\x00'

I should get:

>>>'\x00\x01\x00\x02\x00\x00\x00\x03'

Maybe on another platform - on i386 I get the byte ordering as above.

--
Gabriel Genellina

[aurora00]

Thanks. It almost works. The problem is I don't know the size of the
file until it has finished streaming. It looks like the tar file
format need the file size written at the beginning :(

[Gabriel Genellina]

En Mon, 19 Mar 2007 21:55:30 -0300, au******@gmail.com
<au******@gmail.comescribió:

Thanks. It almost works. The problem is I don't know the size of the
file until it has finished streaming. It looks like the tar file
format need the file size written at the beginning :(

Yes, maybe because it's originally a tape format. Anyway it could be done;
addfile() could seek back to the header and patch it after the file size
is known...

--
Gabriel Genellina