By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
443,271 Members | 1,722 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 443,271 IT Pros & Developers. It's quick & easy.

calling tar commands

P: 12
I think what I知 trying to do is very basic but I知 a rank beginner and I知 stuck. I知 using Python 2.4.1 on WindowsNT. I致e installed UnxUtils, which includes the tar commands for Windows.

Immediate objective:
I have a text file containing a list of tar files, one filename per line. For now I知 just trying to loop through my text file and untar each file in the list.

Background/big picture:
Ultimately I壇 like to
Loop through the list and untar each tar file (~1700 files total),
import some of the un-tarred components of the original tar file into an image processing program (ERDAS Imagine),
zip the output from Imagine, and
delete the unneeded files from the original tar.

So far, I知 able to loop through the list and print filenames. Also, the tar command is working from the command line (and batch file) but I知 missing something when I try to integrate the tar command onto python.

Here痴 what I致e got so far:

#####Loop through and print each line (works fine):
Expand|Select|Wrap|Line Numbers
  1.  
  2. import fileinput
  3.  
  4. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  5. inFile = open(path + "\\tarList7.txt", 'r')
  6.  
  7. for line in inFile:
  8.     print line
  9.  
  10. inFile.close()
  11.  
  12. print "Done!"
  13.  
#####Trying to add tar component:
Expand|Select|Wrap|Line Numbers
  1. import fileinput
  2. import tarfile
  3.  
  4. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  5. inFile = open(path + "\\tarList2.txt", "r")
  6.  
  7. line = inFile.readline()
  8. tar = tarfile.open(line)
  9.  
  10. for line in inFile:
  11.     print line
  12.     tar.extractall()
  13.     tar.close()
  14.  
  15. inFile.close()
  16.  
Thanks in advance...
Jun 6 '07 #1
Share this Question
Share on Google+
18 Replies


ilikepython
Expert 100+
P: 844
I think what I知 trying to do is very basic but I知 a rank beginner and I知 stuck. I知 using Python 2.4.1 on WindowsNT. I致e installed UnxUtils, which includes the tar commands for Windows.

Immediate objective:
I have a text file containing a list of tar files, one filename per line. For now I知 just trying to loop through my text file and untar each file in the list.

Background/big picture:
Ultimately I壇 like to
Loop through the list and untar each tar file (~1700 files total),
import some of the un-tarred components of the original tar file into an image processing program (ERDAS Imagine),
zip the output from Imagine, and
delete the unneeded files from the original tar.

So far, I知 able to loop through the list and print filenames. Also, the tar command is working from the command line (and batch file) but I知 missing something when I try to integrate the tar command onto python.

Here痴 what I致e got so far:

#####Loop through and print each line (works fine):
Expand|Select|Wrap|Line Numbers
  1.  
  2. import fileinput
  3.  
  4. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  5. inFile = open(path + "\\tarList7.txt", 'r')
  6.  
  7. for line in inFile:
  8.     print line
  9.  
  10. inFile.close()
  11.  
  12. print "Done!"
  13.  
#####Trying to add tar component:
Expand|Select|Wrap|Line Numbers
  1. import fileinput
  2. import tarfile
  3.  
  4. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  5. inFile = open(path + "\\tarList2.txt", "r")
  6.  
  7. line = inFile.readline()
  8. tar = tarfile.open(line)
  9.  
  10. for line in inFile:
  11.     print line
  12.     tar.extractall()
  13.     tar.close()
  14.  
  15. inFile.close()
  16.  
Thanks in advance...
What's the error?

I haven't used this module at all but why are you closing the tar file in the for loop?
Jun 6 '07 #2

Expert 100+
P: 511
I think what I知 trying to do is very basic but I知 a rank beginner and I知 stuck. I知 using Python 2.4.1 on WindowsNT. I致e installed UnxUtils, which includes the tar commands for Windows.

Immediate objective:
I have a text file containing a list of tar files, one filename per line. For now I知 just trying to loop through my text file and untar each file in the list.

Background/big picture:
Ultimately I壇 like to
Loop through the list and untar each tar file (~1700 files total),
import some of the un-tarred components of the original tar file into an image processing program (ERDAS Imagine),
zip the output from Imagine, and
delete the unneeded files from the original tar.

So far, I知 able to loop through the list and print filenames. Also, the tar command is working from the command line (and batch file) but I知 missing something when I try to integrate the tar command onto python.

Here痴 what I致e got so far:

#####Loop through and print each line (works fine):
Expand|Select|Wrap|Line Numbers
  1.  
  2. import fileinput
  3.  
  4. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  5. inFile = open(path + "\\tarList7.txt", 'r')
  6.  
  7. for line in inFile:
  8.     print line
  9.  
  10. inFile.close()
  11.  
  12. print "Done!"
  13.  
#####Trying to add tar component:
Expand|Select|Wrap|Line Numbers
  1. import fileinput
  2. import tarfile
  3.  
  4. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  5. inFile = open(path + "\\tarList2.txt", "r")
  6.  
  7. line = inFile.readline()
  8. tar = tarfile.open(line)
  9.  
  10. for line in inFile:
  11.     print line
  12.     tar.extractall()
  13.     tar.close()
  14.  
  15. inFile.close()
  16.  
Thanks in advance...
extractall() takes in arguments...
Jun 7 '07 #3

ilikepython
Expert 100+
P: 844
extractall() takes in arguments...
Yea but they're optional. If a path is not given, it uses the current directory.
Jun 7 '07 #4

Expert 100+
P: 511
Yea but they're optional. If a path is not given, it uses the current directory.
oops, my bad for misreading.
Expand|Select|Wrap|Line Numbers
  1. for line in open("tarlist.txt"):
  2.     line=line.strip()
  3.     tar = tarfile.open(line)
  4.     for t in tar:
  5.         tar.extract(t)
  6.  
Jun 7 '07 #5

P: 12
Sorry... meant to include the error. I included my code in my original posting to show that I've at least been trying, but I'm so green that I suspect there's a better, cleaner way to do this.

Here's the error msg (trailing newline):

Error Message:
Traceback (most recent call last):
File "C:\Python24\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py" , line 310, in RunScript
exec codeObject in __main__.__dict__
File "C:\PythonScripts\MyScripts\Script3.py", line 9, in ?
tar = tarfile.open(line)
File "C:\Python24\lib\tarfile.py", line 916, in open
return func(name, "r", fileobj)
File "C:\Python24\lib\tarfile.py", line 959, in gzopen
fileobj = file(name, mode + "b")
IOError: [Errno 2] No such file or directory: 'NZT050130311116199500.tar\n'
Jun 7 '07 #6

ilikepython
Expert 100+
P: 844
Sorry... meant to include the error. I included my code in my original posting to show that I've at least been trying, but I'm so green that I suspect there's a better, cleaner way to do this.

Here's the error msg (trailing newline):

Error Message:
Traceback (most recent call last):
File "C:\Python24\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py" , line 310, in RunScript
exec codeObject in __main__.__dict__
File "C:\PythonScripts\MyScripts\Script3.py", line 9, in ?
tar = tarfile.open(line)
File "C:\Python24\lib\tarfile.py", line 916, in open
return func(name, "r", fileobj)
File "C:\Python24\lib\tarfile.py", line 959, in gzopen
fileobj = file(name, mode + "b")
IOError: [Errno 2] No such file or directory: 'NZT050130311116199500.tar\n'
Try:
Expand|Select|Wrap|Line Numbers
  1. tar = tarfile.open(line[:-1])
  2.  
That will remove the trailing new line.
Jun 7 '07 #7

bartonc
Expert 5K+
P: 6,596
Try:
Expand|Select|Wrap|Line Numbers
  1. tar = tarfile.open(line[:-1])
  2.  
That will remove the trailing new line.
Expand|Select|Wrap|Line Numbers
  1. As will tar = tarfile.open(line.strip())
In case there is no newline at the end.
Jun 7 '07 #8

P: 12
Thanks much!! I've got the code working (untarring files contained in a txt file) when the python script is located in the same directory as the tar files.

Two more questions:
1) How do I specify different input and output directories so I can run the script from a different location and put my output (untarred files) in yet another directory?
2) Is there redundancy in lines 5 and 8?

Here's the code that's working as described above:
Expand|Select|Wrap|Line Numbers
  1. import fileinput
  2. import tarfile
  3. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  4. tfile = "\\tarList2.txt"
  5. inFile = open(path + tfile)
  6. line = inFile.readline()
  7.  
  8. for line in open(path + tfile):
  9.     line = line.strip()
  10.     print line
  11.     tar = tarfile.open(line)
  12.     for t in tar:
  13.         tar.extract(t)
  14.  
Jun 12 '07 #9

ilikepython
Expert 100+
P: 844
Thanks much!! I've got the code working (untarring files contained in a txt file) when the python script is located in the same directory as the tar files.

Two more questions:
1) How do I specify different input and output directories so I can run the script from a different location and put my output (untarred files) in yet another directory?
2) Is there redundancy in lines 5 and 8?

Here's the code that's working as described above:
Expand|Select|Wrap|Line Numbers
  1. import fileinput
  2. import tarfile
  3. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  4. tfile = "\\tarList2.txt"
  5. inFile = open(path + tfile)
  6. line = inFile.readline()
  7.  
  8. for line in open(path + tfile):
  9.     line = line.strip()
  10.     print line
  11.     tar = tarfile.open(line)
  12.     for t in tar:
  13.         tar.extract(t)
  14.  
You don't need to open the file every line. Also it is a good idea to use the os module for files and directories for portability:
Expand|Select|Wrap|Line Numbers
  1. import os, os.path
  2. import tarfile
  3. import fileinput
  4.  
  5. path = raw_input("Enter a directory: ")
  6. tfile = raw_input("Enter the file: ")
  7.  
  8. for line in open(os.path.join(path, tfile)):
  9.     line = line.strip()
  10.     print line
  11.     tar = tarfile.open(line)
  12.     for t in tar:
  13.         tar.extract(t)
  14.  
I replaced your path with raw_input() calls, is that you need?

Also, it might be a good idea to check if the path exists so your program doesn't raise an exception.
Jun 12 '07 #10

P: 12
Nope, I don't really want interactive raw_input, but thanks anyway.

As i've got it set up now, the textfile (tarList2.txt) just has a list of file names w/out paths:
NZT070830130627200100.tar
NZT070830140710200000.tar
etc...

When I have the txt file and the script in the same directory as the tar files, it works fine and extracts the files in the same directory.

What I'd like to do is
1) specify a different output directory for the extracted tar files
and
2) (of less importance) be able to store the script in a different location from the text and tar files and reference the directory where the tar files reside. I added the full path to the entries in the text file:
C:\WorkSpace\tmp\NZT070830130627200100.tar
C:\WorkSpace\tmp\NZT070830140710200000.tar

but I'm having trouble managing my strings. I get a "no such file" error with double backslashes in the path when it tries to read the txt file. I also tried using the r"path\file.tar" format in the text file:
r"C:\WorkSpace\tmp\NZT070830130627200100.tar"
r"C:\WorkSpace\tmp\NZT070830140710200000.tar"

but get the same "double-slash" error:
IOError: [Errno 2] No such file or directory: 'r"C:\\WorkSpace\\tmp\\NZT070830130627200100.tar "

How do I get rid of the extra back-slash? I appreciate your help.
Jun 13 '07 #11

ilikepython
Expert 100+
P: 844
Nope, I don't really want interactive raw_input, but thanks anyway.

As i've got it set up now, the textfile (tarList2.txt) just has a list of file names w/out paths:
NZT070830130627200100.tar
NZT070830140710200000.tar
etc...

When I have the txt file and the script in the same directory as the tar files, it works fine and extracts the files in the same directory.

What I'd like to do is
1) specify a different output directory for the extracted tar files
and
2) (of less importance) be able to store the script in a different location from the text and tar files and reference the directory where the tar files reside. I added the full path to the entries in the text file:
C:\WorkSpace\tmp\NZT070830130627200100.tar
C:\WorkSpace\tmp\NZT070830140710200000.tar

but I'm having trouble managing my strings. I get a "no such file" error with double backslashes in the path when it tries to read the txt file. I also tried using the r"path\file.tar" format in the text file:
r"C:\WorkSpace\tmp\NZT070830130627200100.tar"
r"C:\WorkSpace\tmp\NZT070830140710200000.tar"

but get the same "double-slash" error:
IOError: [Errno 2] No such file or directory: 'r"C:\\WorkSpace\\tmp\\NZT070830130627200100.tar "

How do I get rid of the extra back-slash? I appreciate your help.
1. The tar functions extract() and extractall() take an optional parameter that determines the path. If you don't give one it extracts to the same path as the script.
Expand|Select|Wrap|Line Numbers
  1. for t in tar:
  2.     tar.extract(t, path)
  3.  
2. There are several ways you can fix this. Either you can have the path in the text file like:
Expand|Select|Wrap|Line Numbers
  1. C:\\WorkSpace\\tmp\\NZT070830140710200000.tar
  2.  
(Remember that text files aren't python files, You don't need to inclose it in quotes.)
Then you will just open it normally.
Another way you, is you can call the open function like this:
Expand|Select|Wrap|Line Numbers
  1. tar = tarfile.open(`line`)
  2.  
That makes it a raw string.
For that way the lines in the text file should be:
Expand|Select|Wrap|Line Numbers
  1. C:\WorkSpace\tmp\NZT070830140710200000.tar
  2.  
Hope that helps.
Jun 13 '07 #12

P: 12
Dear ilikepython,

Thanks for hanging in there w/ me. I'm still screwing up the strings; now I'm getting quadruple back-slashes!

I tried to make your changes and here's the code followed by the error msg and my text file example:

Expand|Select|Wrap|Line Numbers
  1. import os, os.path
  2. import fileinput
  3. import tarfile
  4.  
  5. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  6. tfile = r"tarList2p.txt"
  7.  
  8. for line in open(os.path.join(path, tfile)):
  9.     line = line.strip()
  10.     print line
  11.     tar = tarfile.open(`line`)
  12.     for t in tar:
  13.         tar.extract(t, r"C:\WorkSpace\LTC\MRLC_test\NZTunzipped")
  14.  
IOError: [Errno 2] No such file or directory: "'C:\\\\WorkSpace\\\\LTC\\\\MRLC_test\\\\NZT\\\\tm p\\\\NZT070830130627200100.tar'"

Example of C:\WorkSpace\LTC\MRLC_test\NZT\tmp\tarList2p.txt:
C:\WorkSpace\LTC\MRLC_test\NZT\tmp\NZT070830130627 200100.tar
C:\WorkSpace\LTC\MRLC_test\NZT\tmp\NZT070830140710 200000.tar

What am I overlooking??
Jun 13 '07 #13

ilikepython
Expert 100+
P: 844
Dear ilikepython,

Thanks for hanging in there w/ me. I'm still screwing up the strings; now I'm getting quadruple back-slashes!

I tried to make your changes and here's the code followed by the error msg and my text file example:

Expand|Select|Wrap|Line Numbers
  1. import os, os.path
  2. import fileinput
  3. import tarfile
  4.  
  5. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  6. tfile = r"tarList2p.txt"
  7.  
  8. for line in open(os.path.join(path, tfile)):
  9.     line = line.strip()
  10.     print line
  11.     tar = tarfile.open(`line`)
  12.     for t in tar:
  13.         tar.extract(t, r"C:\WorkSpace\LTC\MRLC_test\NZTunzipped")
  14.  
IOError: [Errno 2] No such file or directory: "'C:\\\\WorkSpace\\\\LTC\\\\MRLC_test\\\\NZT\\\\tm p\\\\NZT070830130627200100.tar'"

Example of C:\WorkSpace\LTC\MRLC_test\NZT\tmp\tarList2p.txt:
C:\WorkSpace\LTC\MRLC_test\NZT\tmp\NZT070830130627 200100.tar
C:\WorkSpace\LTC\MRLC_test\NZT\tmp\NZT070830140710 200000.tar

What am I overlooking??
Oops, I'm really sorry, I gave you bad information. When you read from a file, the strings stays the same as it is from the file. Try it without the apostrophies:
Expand|Select|Wrap|Line Numbers
  1. tar = tarfile.open(line)
  2.  
If it doesn't work, please post back here, I'm interested.
Jun 14 '07 #14

ilikepython
Expert 100+
P: 844
Oops, I'm really sorry, I gave you bad information. When you read from a file, the strings stays the same as it is from the file. Try it without the apostrophies:
Expand|Select|Wrap|Line Numbers
  1. tar = tarfile.open(line)
  2.  
If it doesn't work, please post back here, I'm interested.
Ok, I did some tests and it works, so I'm pretty sure it will work for you.
Jun 14 '07 #15

P: 12
Nope. The only thing I changed was getting rid of the apostrophes and now I'm back to double back-slashes in the error msg:

IOError: [Errno 2] No such file or directory: 'C:\\WorkSpace\\LTC\\MRLC_test\\NZT\\tmp\\NZT07083 0130627200100.tar'

The path strings in the text file are just as you'd see them in Windows:

C:\WorkSpace\LTC\MRLC_test\NZT\tmp\NZT070830130627 200100.tar
Jun 14 '07 #16

bvdet
Expert Mod 2.5K+
P: 2,851
Nope. The only thing I changed was getting rid of the apostrophes and now I'm back to double back-slashes in the error msg:

IOError: [Errno 2] No such file or directory: 'C:\\WorkSpace\\LTC\\MRLC_test\\NZT\\tmp\\NZT07083 0130627200100.tar'

The path strings in the text file are just as you'd see them in Windows:

C:\WorkSpace\LTC\MRLC_test\NZT\tmp\NZT070830130627 200100.tar
That appears to be a valid file name and error message. I get the same message when attempting to open a file for reading that does not exist:
IOError: [Errno 2] No such file or directory: 'H:\\TEMP\\temsys\\xdata9992.txt'

Try this on your system:
Expand|Select|Wrap|Line Numbers
  1. >>> import os
  2. >>> os.path.isfile(r'H:\TEMP\temsys\xdata9992.txt')
  3. False
  4. >>> os.path.isfile(r'H:\TEMP\temsys\xdata999.txt')
  5. True
  6. >>>
Jun 14 '07 #17

P: 12
Sooo sorry... I had the paths wrong in the textfile; the files I was trying to hit were one level up. doh!!

It's working beautifully. I have more development ahead but will start a new thread if/when I get stuck.

Thanks again for all your help.

with tail between legs,
Thom
Jun 14 '07 #18

P: 4
Just coming from a slightly minimalist view, you could also opt to run this command directly through the shell/command prompt. But since you're on Windows, chances are you don't have the tar utility and if you did it wouldn't be in the Path. =P But a Linux equivalent:

Expand|Select|Wrap|Line Numbers
  1. import os
  2.  
  3. f = open("path_to_list.txt")
  4. for line in f:
  5.   os.system("tar -zxvf " + line)
  6.  
Note this would untar to the current directory that the script is in, and you'd have to add a path to the line unless it's already in the file.
Jun 14 '07 #19

Post your reply

Sign in to post your reply or Sign up for a free account.