473,230 Members | 1,527 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,230 software developers and data experts.

Version of TAR in tarfile module? TAR 1.14 or 1.15 port to Windows?

I need to unpack on a Windows 2000 machine
some Wikipedia media .tar archives which are
compressed with TAR 1.14 (support for long file
names and maybe some other features) .
It seems, that Pythons tarfile module is able to list far
more files inside the archives than WinRAR or 7zip or
TotalCommander, but will it unpack all available files
(largest archive size 17 GByte)?

If tarfile is build on TAR 1.14 or TAR 1.15 it will be
probably ok, so my questions are:

What TAR version is built into the tarfile module?

Is there a TAR 1.14 or 1.15 port to Windows
available in Internet for download (which URL)?

Claudio

Aug 20 '05 #1
4 3337
Claudio Grondi wrote:
What TAR version is built into the tarfile module?
None: the tarfile module is not built on top of
GNU tar. Instead, it is a complete reimplementation.
Is there a TAR 1.14 or 1.15 port to Windows
available in Internet for download (which URL)?


http://sources.redhat.com/cygwin/

Regards,
Martin
Aug 20 '05 #2
Martin, thank you for your response.

I see, that I have to test myself if the tarfile
module can do what I need, so I did and I
have evidence, that the Python tarfile module
is not able to see all the files inside the TAR
archives created on Linux with TAR 1.14 .
The Python tarfile module stops to go through the
TAR archive somewhere already at the beginning,
like the other tools I have used, did.

I have no understanding what Cygwin, MinGW
(do I put MinGW in the right context?) are and do,
but I gave Cygwin a try and installed it selecting
also the tar package version 1.15 for installation.

I tried to run the tar.exe, but in the beginning it was
not able to do anything for me. I haven't given up
and after hours of trying to find the reason for this
I found that:
the tar.exe seems to have no access to any files
not inside the
[%SystemDrive%\Cygwin\usr]
directory on my system drive e: , where Cygwin is
installed.
Is there a way to go around this, because I have
to uncompress a 17 GByte file and my system
drive has only 3 GByte of free storage space.
I have in mind, that it could have something to do
with the mount command, but this is all I can
remember. I work in a Windows command shell
(DOS-box) and mount says:
j: on /cygdrive/j , but I don't know how to write
the entire path
"j:\o\archives\images\dump.tar",
so that the file can be found by tar.exe and
unpacked to "i:\images" .
tar.exe --extract --directory=tmp -f j:/o/archives/images/dump.tar
results in:
/usr/bin/tar: j\:/o/archives/images/dump.tar: Cannot open: Input/Output
error
telling
tar.exe --extract --directory=tmp -f /cygdrive/j/o/archives/images/dump.tar
doesn't work either.

Claudio

"Martin v. Löwis" <ma****@v.loewis.de> schrieb im Newsbeitrag
news:43***********************@news.freenet.de...
Claudio Grondi wrote:
What TAR version is built into the tarfile module?


None: the tarfile module is not built on top of
GNU tar. Instead, it is a complete reimplementation.
Is there a TAR 1.14 or 1.15 port to Windows
available in Internet for download (which URL)?


http://sources.redhat.com/cygwin/

Regards,
Martin



Aug 20 '05 #3
Claudio Grondi wrote:
remember. I work in a Windows command shell
(DOS-box) and mount says:
j: on /cygdrive/j , but I don't know how to write
the entire path
"j:\o\archives\images\dump.tar",
so that the file can be found by tar.exe and
unpacked to "i:\images" .
tar.exe --extract --directory=tmp -f j:/o/archives/images/dump.tar
results in:
/usr/bin/tar: j\:/o/archives/images/dump.tar: Cannot open: Input/Output
error
telling
tar.exe --extract --directory=tmp -f /cygdrive/j/o/archives/images/dump.tar
doesn't work either.

Try the cygpath-command like this:

echo `cygpath c:\\some\\windows\\path`

That should yield

/cygdrive/c/some/windows/path

Alternatively, do somethink like this

mkdir -p /mnt/j

mount j: /mnt/j

Then /mnt/j should be the root for all files under j:

HTH Diez
Aug 20 '05 #4

Thank you both (Martin and Diez) for your help.

The 17 GByte TAR archive was unpacked
without problems the way you suggested.

Let's summarize:

# Python tarfile module can't extract files from
newer TAR archives (archived with tar 1.14 or later)

# The core of my problems was, that I was not aware
how easy it is to install and work with Cygwin and that to
get Cygwins tar.exe to work it is necessary to use the
provided Cygwin bash-3.00 shell an NOT the Windows
command shell (DOS-box) .

The receipt for unpacking Wikipedia media files
provided as TAR archives when using Microsoft
Windows is:

Step 1. download http://sources.redhat.com/cygwin/setup.exe

Step 2a. run the downloaded setup.exe which goes online and
lets you choose which packages should be installed

Step 2b. select for the installation additional to suggested ones
the tar package version 1.15

Step 3. use the Cygwin icon on the Desktop or in Start-Programs-
Cygwin-Cygwin Bash Shell to start a Cygwin shell and type :

bash-3.00$
../bin/tar.exe --extract --directory=/cygdrive/i/wikipedia/en/media
-f
/cygdrive/j/download.wikimedia.org/archives/images/en/20050530_upload.tar

where i: and j: are the drive letters of appropriate Windows drives.

The media files stored in TAR archive
j: \download.wikimedia.org\archives\images\en\2005053 0_upload.tar
will be unpacked to
i:\wikipedia\en\media
directory

Step 4. wait, wait, wait ... (how long depends most on
speed of your harddrives, on my system with USB drives
appr. one hour)

Step 5. BE HAPPY :))
and enjoy it, because you have mastered a step towards
your own static HTML version of Wikipedia.
The problems with extracting the content from the MySQL
database dumps will kill you soon - and if not, especially
for non-english languages (like German, Polish, Russian)
and with math-formulas converted to pictures,
all done on a Windows system __PLEASE__ share your
know-how with me !!!
(the only useful information I found on Internet about it
were postings within the thread I initiated a longer time
ago myself
http://www.pythonforum.org/ftopic194...ta_to_HTM.html
)

Claudio

"Diez B. Roggisch" <de***@nospam.web.de> schrieb im Newsbeitrag
news:3m*************@uni-berlin.de...
Claudio Grondi wrote:
remember. I work in a Windows command shell
(DOS-box) and mount says:
j: on /cygdrive/j , but I don't know how to write
the entire path
"j:\o\archives\images\dump.tar",
so that the file can be found by tar.exe and
unpacked to "i:\images" .
tar.exe --extract --directory=tmp -f j:/o/archives/images/dump.tar
results in:
/usr/bin/tar: j\:/o/archives/images/dump.tar: Cannot open: Input/Output
error
telling
tar.exe --extract --directory=tmp -f /cygdrive/j/o/archives/images/dump.tar doesn't work either.

Try the cygpath-command like this:

echo `cygpath c:\\some\\windows\\path`

That should yield

/cygdrive/c/some/windows/path

Alternatively, do somethink like this

mkdir -p /mnt/j

mount j: /mnt/j

Then /mnt/j should be the root for all files under j:

HTH Diez

Aug 20 '05 #5

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

8
by: Jay Donnell | last post by:
Is there a way to use the tarfile module to recursively compress the contents of a directory and maintain the directory structure in the tar archive? Simply doing os.system('tar -czvf ' +...
5
by: Uwe Mayer | last post by:
Hi, is it possible to delete a file from a tar-archive using the tarfile module? Thanks Uwe
1
by: Beowulf | last post by:
Hello, I'm using Python to automate admin tasks on my job. We use Windoze 2000 as desktop platform. When executing this daily backup scripts I get the following error: Traceback (most recent...
3
by: justin.vanwinkle | last post by:
Hello everyone, I need some tar functionality for my program. Currently the following code properly displays tar archives, and tar.gz archives. However, it chokes on tar.bz2 archives with the...
7
by: aurora00 | last post by:
I have a program that generates a number of files that will be packaged into a tarball. Can I stream the content into TarFile without first writing them out to the file system? All add(), addfile()...
1
by: Antoon Pardon | last post by:
I have to write a little program that has to run on a number of hosts. Versions of python range from 2.2.3. to 2.4.2. The easiest way to implement the necessary behaviour seems to me to use the...
3
by: Anurag | last post by:
Hi, I am trying to use tarfile module to list contents of a 'gz' file but it seems to hang for large files and CPU usage goes 100%. though 'tar -tvf' on same file list contents in couple of...
6
by: Terry Carroll | last post by:
I am trying to do something with a very large tarfile from within Python, and am running into memory constraints. The tarfile in question is a 4-gigabyte datafile from freedb.org,...
1
by: boblatest | last post by:
Hello, I'm trying to catch an "EOFError" exception that occurs when reading truncated tarfile. Here's my routine, and below that the callback trace. Note that although I'm trying to catch all...
0
by: VivesProcSPL | last post by:
Obviously, one of the original purposes of SQL is to make data query processing easy. The language uses many English-like terms and syntax in an effort to make it easy to learn, particularly for...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.