Version of TAR in tarfile module? TAR 1.14 or 1.15 port to Windows?

[Claudio Grondi]

I need to unpack on a Windows 2000 machine
some Wikipedia media .tar archives which are
compressed with TAR 1.14 (support for long file
names and maybe some other features) .
It seems, that Pythons tarfile module is able to list far
more files inside the archives than WinRAR or 7zip or
TotalCommander, but will it unpack all available files
(largest archive size 17 GByte)?

If tarfile is build on TAR 1.14 or TAR 1.15 it will be
probably ok, so my questions are:

What TAR version is built into the tarfile module?

Is there a TAR 1.14 or 1.15 port to Windows
available in Internet for download (which URL)?

Claudio

[Martin v. Löwis]

Claudio Grondi wrote:

What TAR version is built into the tarfile module?
None: the tarfile module is not built on top of
GNU tar. Instead, it is a complete reimplementation.
Is there a TAR 1.14 or 1.15 port to Windows
available in Internet for download (which URL)?

http://sources.redhat.com/cygwin/

Regards,
Martin

[Claudio Grondi]

Martin, thank you for your response.

I see, that I have to test myself if the tarfile
module can do what I need, so I did and I
have evidence, that the Python tarfile module
is not able to see all the files inside the TAR
archives created on Linux with TAR 1.14 .
The Python tarfile module stops to go through the
TAR archive somewhere already at the beginning,
like the other tools I have used, did.

I have no understanding what Cygwin, MinGW
(do I put MinGW in the right context?) are and do,
but I gave Cygwin a try and installed it selecting
also the tar package version 1.15 for installation.

I tried to run the tar.exe, but in the beginning it was
not able to do anything for me. I haven't given up
and after hours of trying to find the reason for this
I found that:
the tar.exe seems to have no access to any files
not inside the
[%SystemDrive%\Cygwin\usr]
directory on my system drive e: , where Cygwin is
installed.
Is there a way to go around this, because I have
to uncompress a 17 GByte file and my system
drive has only 3 GByte of free storage space.
I have in mind, that it could have something to do
with the mount command, but this is all I can
remember. I work in a Windows command shell
(DOS-box) and mount says:
j: on /cygdrive/j , but I don't know how to write
the entire path
"j:\o\archives\images\dump.tar",
so that the file can be found by tar.exe and
unpacked to "i:\images" .
tar.exe --extract --directory=tmp -f j:/o/archives/images/dump.tar
results in:
/usr/bin/tar: j\:/o/archives/images/dump.tar: Cannot open: Input/Output
error
telling
tar.exe --extract --directory=tmp -f /cygdrive/j/o/archives/images/dump.tar
doesn't work either.

Claudio

"Martin v. Löwis" <ma****@v.loewis.de> schrieb im Newsbeitrag
news:43***********************@news.freenet.de...

Claudio Grondi wrote:

What TAR version is built into the tarfile module?

None: the tarfile module is not built on top of
GNU tar. Instead, it is a complete reimplementation.

Is there a TAR 1.14 or 1.15 port to Windows
available in Internet for download (which URL)?

http://sources.redhat.com/cygwin/

Regards,
Martin

[Diez B. Roggisch]

Claudio Grondi wrote:

remember. I work in a Windows command shell
(DOS-box) and mount says:
j: on /cygdrive/j , but I don't know how to write
the entire path
"j:\o\archives\images\dump.tar",
so that the file can be found by tar.exe and
unpacked to "i:\images" .
tar.exe --extract --directory=tmp -f j:/o/archives/images/dump.tar
results in:
/usr/bin/tar: j\:/o/archives/images/dump.tar: Cannot open: Input/Output
error
telling
tar.exe --extract --directory=tmp -f /cygdrive/j/o/archives/images/dump.tar
doesn't work either.

Try the cygpath-command like this:

echo `cygpath c:\\some\\windows\\path`

That should yield

/cygdrive/c/some/windows/path

Alternatively, do somethink like this

mkdir -p /mnt/j

mount j: /mnt/j

Then /mnt/j should be the root for all files under j:

HTH Diez

[Claudio Grondi]

Thank you both (Martin and Diez) for your help.

The 17 GByte TAR archive was unpacked
without problems the way you suggested.

Let's summarize:

# Python tarfile module can't extract files from
newer TAR archives (archived with tar 1.14 or later)

# The core of my problems was, that I was not aware
how easy it is to install and work with Cygwin and that to
get Cygwins tar.exe to work it is necessary to use the
provided Cygwin bash-3.00 shell an NOT the Windows
command shell (DOS-box) .

The receipt for unpacking Wikipedia media files
provided as TAR archives when using Microsoft
Windows is:

Step 1. download http://sources.redhat.com/cygwin/setup.exe

Step 2a. run the downloaded setup.exe which goes online and
lets you choose which packages should be installed

Step 2b. select for the installation additional to suggested ones
the tar package version 1.15

Step 3. use the Cygwin icon on the Desktop or in Start-Programs-
Cygwin-Cygwin Bash Shell to start a Cygwin shell and type :

bash-3.00$
../bin/tar.exe --extract --directory=/cygdrive/i/wikipedia/en/media
-f
/cygdrive/j/download.wikimedia.org/archives/images/en/20050530_upload.tar

where i: and j: are the drive letters of appropriate Windows drives.

The media files stored in TAR archive
j: \download.wikimedia.org\archives\images\en\2005053 0_upload.tar
will be unpacked to
i:\wikipedia\en\media
directory

Step 4. wait, wait, wait ... (how long depends most on
speed of your harddrives, on my system with USB drives
appr. one hour)

Step 5. BE HAPPY :))
and enjoy it, because you have mastered a step towards
your own static HTML version of Wikipedia.
The problems with extracting the content from the MySQL
database dumps will kill you soon - and if not, especially
for non-english languages (like German, Polish, Russian)
and with math-formulas converted to pictures,
all done on a Windows system __PLEASE__ share your
know-how with me !!!
(the only useful information I found on Internet about it
were postings within the thread I initiated a longer time
ago myself
http://www.pythonforum.org/ftopic194...ta_to_HTM.html
)

Claudio

"Diez B. Roggisch" <de***@nospam.web.de> schrieb im Newsbeitrag
news:3m*************@uni-berlin.de...

Claudio Grondi wrote:

remember. I work in a Windows command shell
(DOS-box) and mount says:
j: on /cygdrive/j , but I don't know how to write
the entire path
"j:\o\archives\images\dump.tar",
so that the file can be found by tar.exe and
unpacked to "i:\images" .
tar.exe --extract --directory=tmp -f j:/o/archives/images/dump.tar
results in:
/usr/bin/tar: j\:/o/archives/images/dump.tar: Cannot open: Input/Output
error
telling
tar.exe --extract --directory=tmp -f /cygdrive/j/o/archives/images/dump.tar doesn't work either.

Try the cygpath-command like this:

echo `cygpath c:\\some\\windows\\path`

That should yield

/cygdrive/c/some/windows/path

Alternatively, do somethink like this

mkdir -p /mnt/j

mount j: /mnt/j

Then /mnt/j should be the root for all files under j:

HTH Diez