473,232 Members | 1,635 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,232 software developers and data experts.

calling tar commands

12
I think what I知 trying to do is very basic but I知 a rank beginner and I知 stuck. I知 using Python 2.4.1 on WindowsNT. I致e installed UnxUtils, which includes the tar commands for Windows.

Immediate objective:
I have a text file containing a list of tar files, one filename per line. For now I知 just trying to loop through my text file and untar each file in the list.

Background/big picture:
Ultimately I壇 like to
Loop through the list and untar each tar file (~1700 files total),
import some of the un-tarred components of the original tar file into an image processing program (ERDAS Imagine),
zip the output from Imagine, and
delete the unneeded files from the original tar.

So far, I知 able to loop through the list and print filenames. Also, the tar command is working from the command line (and batch file) but I知 missing something when I try to integrate the tar command onto python.

Here痴 what I致e got so far:

#####Loop through and print each line (works fine):
Expand|Select|Wrap|Line Numbers
  1.  
  2. import fileinput
  3.  
  4. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  5. inFile = open(path + "\\tarList7.txt", 'r')
  6.  
  7. for line in inFile:
  8.     print line
  9.  
  10. inFile.close()
  11.  
  12. print "Done!"
  13.  
#####Trying to add tar component:
Expand|Select|Wrap|Line Numbers
  1. import fileinput
  2. import tarfile
  3.  
  4. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  5. inFile = open(path + "\\tarList2.txt", "r")
  6.  
  7. line = inFile.readline()
  8. tar = tarfile.open(line)
  9.  
  10. for line in inFile:
  11.     print line
  12.     tar.extractall()
  13.     tar.close()
  14.  
  15. inFile.close()
  16.  
Thanks in advance...
Jun 6 '07 #1
18 3297
ilikepython
844 Expert 512MB
I think what I知 trying to do is very basic but I知 a rank beginner and I知 stuck. I知 using Python 2.4.1 on WindowsNT. I致e installed UnxUtils, which includes the tar commands for Windows.

Immediate objective:
I have a text file containing a list of tar files, one filename per line. For now I知 just trying to loop through my text file and untar each file in the list.

Background/big picture:
Ultimately I壇 like to
Loop through the list and untar each tar file (~1700 files total),
import some of the un-tarred components of the original tar file into an image processing program (ERDAS Imagine),
zip the output from Imagine, and
delete the unneeded files from the original tar.

So far, I知 able to loop through the list and print filenames. Also, the tar command is working from the command line (and batch file) but I知 missing something when I try to integrate the tar command onto python.

Here痴 what I致e got so far:

#####Loop through and print each line (works fine):
Expand|Select|Wrap|Line Numbers
  1.  
  2. import fileinput
  3.  
  4. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  5. inFile = open(path + "\\tarList7.txt", 'r')
  6.  
  7. for line in inFile:
  8.     print line
  9.  
  10. inFile.close()
  11.  
  12. print "Done!"
  13.  
#####Trying to add tar component:
Expand|Select|Wrap|Line Numbers
  1. import fileinput
  2. import tarfile
  3.  
  4. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  5. inFile = open(path + "\\tarList2.txt", "r")
  6.  
  7. line = inFile.readline()
  8. tar = tarfile.open(line)
  9.  
  10. for line in inFile:
  11.     print line
  12.     tar.extractall()
  13.     tar.close()
  14.  
  15. inFile.close()
  16.  
Thanks in advance...
What's the error?

I haven't used this module at all but why are you closing the tar file in the for loop?
Jun 6 '07 #2
ghostdog74
511 Expert 256MB
I think what I知 trying to do is very basic but I知 a rank beginner and I知 stuck. I知 using Python 2.4.1 on WindowsNT. I致e installed UnxUtils, which includes the tar commands for Windows.

Immediate objective:
I have a text file containing a list of tar files, one filename per line. For now I知 just trying to loop through my text file and untar each file in the list.

Background/big picture:
Ultimately I壇 like to
Loop through the list and untar each tar file (~1700 files total),
import some of the un-tarred components of the original tar file into an image processing program (ERDAS Imagine),
zip the output from Imagine, and
delete the unneeded files from the original tar.

So far, I知 able to loop through the list and print filenames. Also, the tar command is working from the command line (and batch file) but I知 missing something when I try to integrate the tar command onto python.

Here痴 what I致e got so far:

#####Loop through and print each line (works fine):
Expand|Select|Wrap|Line Numbers
  1.  
  2. import fileinput
  3.  
  4. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  5. inFile = open(path + "\\tarList7.txt", 'r')
  6.  
  7. for line in inFile:
  8.     print line
  9.  
  10. inFile.close()
  11.  
  12. print "Done!"
  13.  
#####Trying to add tar component:
Expand|Select|Wrap|Line Numbers
  1. import fileinput
  2. import tarfile
  3.  
  4. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  5. inFile = open(path + "\\tarList2.txt", "r")
  6.  
  7. line = inFile.readline()
  8. tar = tarfile.open(line)
  9.  
  10. for line in inFile:
  11.     print line
  12.     tar.extractall()
  13.     tar.close()
  14.  
  15. inFile.close()
  16.  
Thanks in advance...
extractall() takes in arguments...
Jun 7 '07 #3
ilikepython
844 Expert 512MB
extractall() takes in arguments...
Yea but they're optional. If a path is not given, it uses the current directory.
Jun 7 '07 #4
ghostdog74
511 Expert 256MB
Yea but they're optional. If a path is not given, it uses the current directory.
oops, my bad for misreading.
Expand|Select|Wrap|Line Numbers
  1. for line in open("tarlist.txt"):
  2.     line=line.strip()
  3.     tar = tarfile.open(line)
  4.     for t in tar:
  5.         tar.extract(t)
  6.  
Jun 7 '07 #5
tcurdts
12
Sorry... meant to include the error. I included my code in my original posting to show that I've at least been trying, but I'm so green that I suspect there's a better, cleaner way to do this.

Here's the error msg (trailing newline):

Error Message:
Traceback (most recent call last):
File "C:\Python24\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py" , line 310, in RunScript
exec codeObject in __main__.__dict__
File "C:\PythonScripts\MyScripts\Script3.py", line 9, in ?
tar = tarfile.open(line)
File "C:\Python24\lib\tarfile.py", line 916, in open
return func(name, "r", fileobj)
File "C:\Python24\lib\tarfile.py", line 959, in gzopen
fileobj = file(name, mode + "b")
IOError: [Errno 2] No such file or directory: 'NZT050130311116199500.tar\n'
Jun 7 '07 #6
ilikepython
844 Expert 512MB
Sorry... meant to include the error. I included my code in my original posting to show that I've at least been trying, but I'm so green that I suspect there's a better, cleaner way to do this.

Here's the error msg (trailing newline):

Error Message:
Traceback (most recent call last):
File "C:\Python24\Lib\site-packages\pythonwin\pywin\framework\scriptutils.py" , line 310, in RunScript
exec codeObject in __main__.__dict__
File "C:\PythonScripts\MyScripts\Script3.py", line 9, in ?
tar = tarfile.open(line)
File "C:\Python24\lib\tarfile.py", line 916, in open
return func(name, "r", fileobj)
File "C:\Python24\lib\tarfile.py", line 959, in gzopen
fileobj = file(name, mode + "b")
IOError: [Errno 2] No such file or directory: 'NZT050130311116199500.tar\n'
Try:
Expand|Select|Wrap|Line Numbers
  1. tar = tarfile.open(line[:-1])
  2.  
That will remove the trailing new line.
Jun 7 '07 #7
bartonc
6,596 Expert 4TB
Try:
Expand|Select|Wrap|Line Numbers
  1. tar = tarfile.open(line[:-1])
  2.  
That will remove the trailing new line.
Expand|Select|Wrap|Line Numbers
  1. As will tar = tarfile.open(line.strip())
In case there is no newline at the end.
Jun 7 '07 #8
tcurdts
12
Thanks much!! I've got the code working (untarring files contained in a txt file) when the python script is located in the same directory as the tar files.

Two more questions:
1) How do I specify different input and output directories so I can run the script from a different location and put my output (untarred files) in yet another directory?
2) Is there redundancy in lines 5 and 8?

Here's the code that's working as described above:
Expand|Select|Wrap|Line Numbers
  1. import fileinput
  2. import tarfile
  3. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  4. tfile = "\\tarList2.txt"
  5. inFile = open(path + tfile)
  6. line = inFile.readline()
  7.  
  8. for line in open(path + tfile):
  9.     line = line.strip()
  10.     print line
  11.     tar = tarfile.open(line)
  12.     for t in tar:
  13.         tar.extract(t)
  14.  
Jun 12 '07 #9
ilikepython
844 Expert 512MB
Thanks much!! I've got the code working (untarring files contained in a txt file) when the python script is located in the same directory as the tar files.

Two more questions:
1) How do I specify different input and output directories so I can run the script from a different location and put my output (untarred files) in yet another directory?
2) Is there redundancy in lines 5 and 8?

Here's the code that's working as described above:
Expand|Select|Wrap|Line Numbers
  1. import fileinput
  2. import tarfile
  3. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  4. tfile = "\\tarList2.txt"
  5. inFile = open(path + tfile)
  6. line = inFile.readline()
  7.  
  8. for line in open(path + tfile):
  9.     line = line.strip()
  10.     print line
  11.     tar = tarfile.open(line)
  12.     for t in tar:
  13.         tar.extract(t)
  14.  
You don't need to open the file every line. Also it is a good idea to use the os module for files and directories for portability:
Expand|Select|Wrap|Line Numbers
  1. import os, os.path
  2. import tarfile
  3. import fileinput
  4.  
  5. path = raw_input("Enter a directory: ")
  6. tfile = raw_input("Enter the file: ")
  7.  
  8. for line in open(os.path.join(path, tfile)):
  9.     line = line.strip()
  10.     print line
  11.     tar = tarfile.open(line)
  12.     for t in tar:
  13.         tar.extract(t)
  14.  
I replaced your path with raw_input() calls, is that you need?

Also, it might be a good idea to check if the path exists so your program doesn't raise an exception.
Jun 12 '07 #10
tcurdts
12
Nope, I don't really want interactive raw_input, but thanks anyway.

As i've got it set up now, the textfile (tarList2.txt) just has a list of file names w/out paths:
NZT070830130627200100.tar
NZT070830140710200000.tar
etc...

When I have the txt file and the script in the same directory as the tar files, it works fine and extracts the files in the same directory.

What I'd like to do is
1) specify a different output directory for the extracted tar files
and
2) (of less importance) be able to store the script in a different location from the text and tar files and reference the directory where the tar files reside. I added the full path to the entries in the text file:
C:\WorkSpace\tmp\NZT070830130627200100.tar
C:\WorkSpace\tmp\NZT070830140710200000.tar

but I'm having trouble managing my strings. I get a "no such file" error with double backslashes in the path when it tries to read the txt file. I also tried using the r"path\file.tar" format in the text file:
r"C:\WorkSpace\tmp\NZT070830130627200100.tar"
r"C:\WorkSpace\tmp\NZT070830140710200000.tar"

but get the same "double-slash" error:
IOError: [Errno 2] No such file or directory: 'r"C:\\WorkSpace\\tmp\\NZT070830130627200100.tar "

How do I get rid of the extra back-slash? I appreciate your help.
Jun 13 '07 #11
ilikepython
844 Expert 512MB
Nope, I don't really want interactive raw_input, but thanks anyway.

As i've got it set up now, the textfile (tarList2.txt) just has a list of file names w/out paths:
NZT070830130627200100.tar
NZT070830140710200000.tar
etc...

When I have the txt file and the script in the same directory as the tar files, it works fine and extracts the files in the same directory.

What I'd like to do is
1) specify a different output directory for the extracted tar files
and
2) (of less importance) be able to store the script in a different location from the text and tar files and reference the directory where the tar files reside. I added the full path to the entries in the text file:
C:\WorkSpace\tmp\NZT070830130627200100.tar
C:\WorkSpace\tmp\NZT070830140710200000.tar

but I'm having trouble managing my strings. I get a "no such file" error with double backslashes in the path when it tries to read the txt file. I also tried using the r"path\file.tar" format in the text file:
r"C:\WorkSpace\tmp\NZT070830130627200100.tar"
r"C:\WorkSpace\tmp\NZT070830140710200000.tar"

but get the same "double-slash" error:
IOError: [Errno 2] No such file or directory: 'r"C:\\WorkSpace\\tmp\\NZT070830130627200100.tar "

How do I get rid of the extra back-slash? I appreciate your help.
1. The tar functions extract() and extractall() take an optional parameter that determines the path. If you don't give one it extracts to the same path as the script.
Expand|Select|Wrap|Line Numbers
  1. for t in tar:
  2.     tar.extract(t, path)
  3.  
2. There are several ways you can fix this. Either you can have the path in the text file like:
Expand|Select|Wrap|Line Numbers
  1. C:\\WorkSpace\\tmp\\NZT070830140710200000.tar
  2.  
(Remember that text files aren't python files, You don't need to inclose it in quotes.)
Then you will just open it normally.
Another way you, is you can call the open function like this:
Expand|Select|Wrap|Line Numbers
  1. tar = tarfile.open(`line`)
  2.  
That makes it a raw string.
For that way the lines in the text file should be:
Expand|Select|Wrap|Line Numbers
  1. C:\WorkSpace\tmp\NZT070830140710200000.tar
  2.  
Hope that helps.
Jun 13 '07 #12
tcurdts
12
Dear ilikepython,

Thanks for hanging in there w/ me. I'm still screwing up the strings; now I'm getting quadruple back-slashes!

I tried to make your changes and here's the code followed by the error msg and my text file example:

Expand|Select|Wrap|Line Numbers
  1. import os, os.path
  2. import fileinput
  3. import tarfile
  4.  
  5. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  6. tfile = r"tarList2p.txt"
  7.  
  8. for line in open(os.path.join(path, tfile)):
  9.     line = line.strip()
  10.     print line
  11.     tar = tarfile.open(`line`)
  12.     for t in tar:
  13.         tar.extract(t, r"C:\WorkSpace\LTC\MRLC_test\NZTunzipped")
  14.  
IOError: [Errno 2] No such file or directory: "'C:\\\\WorkSpace\\\\LTC\\\\MRLC_test\\\\NZT\\\\tm p\\\\NZT070830130627200100.tar'"

Example of C:\WorkSpace\LTC\MRLC_test\NZT\tmp\tarList2p.txt:
C:\WorkSpace\LTC\MRLC_test\NZT\tmp\NZT070830130627 200100.tar
C:\WorkSpace\LTC\MRLC_test\NZT\tmp\NZT070830140710 200000.tar

What am I overlooking??
Jun 13 '07 #13
ilikepython
844 Expert 512MB
Dear ilikepython,

Thanks for hanging in there w/ me. I'm still screwing up the strings; now I'm getting quadruple back-slashes!

I tried to make your changes and here's the code followed by the error msg and my text file example:

Expand|Select|Wrap|Line Numbers
  1. import os, os.path
  2. import fileinput
  3. import tarfile
  4.  
  5. path = r"C:\WorkSpace\LTC\MRLC_test\NZT\tmp"
  6. tfile = r"tarList2p.txt"
  7.  
  8. for line in open(os.path.join(path, tfile)):
  9.     line = line.strip()
  10.     print line
  11.     tar = tarfile.open(`line`)
  12.     for t in tar:
  13.         tar.extract(t, r"C:\WorkSpace\LTC\MRLC_test\NZTunzipped")
  14.  
IOError: [Errno 2] No such file or directory: "'C:\\\\WorkSpace\\\\LTC\\\\MRLC_test\\\\NZT\\\\tm p\\\\NZT070830130627200100.tar'"

Example of C:\WorkSpace\LTC\MRLC_test\NZT\tmp\tarList2p.txt:
C:\WorkSpace\LTC\MRLC_test\NZT\tmp\NZT070830130627 200100.tar
C:\WorkSpace\LTC\MRLC_test\NZT\tmp\NZT070830140710 200000.tar

What am I overlooking??
Oops, I'm really sorry, I gave you bad information. When you read from a file, the strings stays the same as it is from the file. Try it without the apostrophies:
Expand|Select|Wrap|Line Numbers
  1. tar = tarfile.open(line)
  2.  
If it doesn't work, please post back here, I'm interested.
Jun 14 '07 #14
ilikepython
844 Expert 512MB
Oops, I'm really sorry, I gave you bad information. When you read from a file, the strings stays the same as it is from the file. Try it without the apostrophies:
Expand|Select|Wrap|Line Numbers
  1. tar = tarfile.open(line)
  2.  
If it doesn't work, please post back here, I'm interested.
Ok, I did some tests and it works, so I'm pretty sure it will work for you.
Jun 14 '07 #15
tcurdts
12
Nope. The only thing I changed was getting rid of the apostrophes and now I'm back to double back-slashes in the error msg:

IOError: [Errno 2] No such file or directory: 'C:\\WorkSpace\\LTC\\MRLC_test\\NZT\\tmp\\NZT07083 0130627200100.tar'

The path strings in the text file are just as you'd see them in Windows:

C:\WorkSpace\LTC\MRLC_test\NZT\tmp\NZT070830130627 200100.tar
Jun 14 '07 #16
bvdet
2,851 Expert Mod 2GB
Nope. The only thing I changed was getting rid of the apostrophes and now I'm back to double back-slashes in the error msg:

IOError: [Errno 2] No such file or directory: 'C:\\WorkSpace\\LTC\\MRLC_test\\NZT\\tmp\\NZT07083 0130627200100.tar'

The path strings in the text file are just as you'd see them in Windows:

C:\WorkSpace\LTC\MRLC_test\NZT\tmp\NZT070830130627 200100.tar
That appears to be a valid file name and error message. I get the same message when attempting to open a file for reading that does not exist:
IOError: [Errno 2] No such file or directory: 'H:\\TEMP\\temsys\\xdata9992.txt'

Try this on your system:
Expand|Select|Wrap|Line Numbers
  1. >>> import os
  2. >>> os.path.isfile(r'H:\TEMP\temsys\xdata9992.txt')
  3. False
  4. >>> os.path.isfile(r'H:\TEMP\temsys\xdata999.txt')
  5. True
  6. >>>
Jun 14 '07 #17
tcurdts
12
Sooo sorry... I had the paths wrong in the textfile; the files I was trying to hit were one level up. doh!!

It's working beautifully. I have more development ahead but will start a new thread if/when I get stuck.

Thanks again for all your help.

with tail between legs,
Thom
Jun 14 '07 #18
Jach
4
Just coming from a slightly minimalist view, you could also opt to run this command directly through the shell/command prompt. But since you're on Windows, chances are you don't have the tar utility and if you did it wouldn't be in the Path. =P But a Linux equivalent:

Expand|Select|Wrap|Line Numbers
  1. import os
  2.  
  3. f = open("path_to_list.txt")
  4. for line in f:
  5.   os.system("tar -zxvf " + line)
  6.  
Note this would untar to the current directory that the script is in, and you'd have to add a path to the line unless it's already in the file.
Jun 14 '07 #19

Sign in to post your reply or Sign up for a free account.

Similar topics

12
by: Moosebumps | last post by:
So, after reading some messages about os.system, and looking at the popen stuff and trying it a bit, I still have not found a way to keep a command window open for several commands (on Windows...
11
by: ronan_boisard | last post by:
hi all, I'm trying to call ksh script from python. I have a file (toto.env) with a scirpt looking like: -- begin --- #!/bin/ksh # export TOTO_ENV=/home/toto -- end ---
8
by: Barry Sanderson | last post by:
I'm having a problem calling a .dll file from asp. Here is the situation: I'm creating a web enabled mp3 app that uses windows messaging to control winamp. To this end I've made a dll that...
18
by: Simula | last post by:
I am developing an HTML javascript application and I want to preserve state in a way that can be book-marked. I chose HTML anchors as a means of preserving state. When the application changes...
1
by: Marina | last post by:
The commands that are generated by the SqlCommandBuilder are not quite right. I am calling GetUpdateCommand and GetDeleteCommand, modifying the results, and assigning them to the UpdateCommand and...
2
by: Rajat Katyal | last post by:
Hi: I prepare the statement for execution as follows: PREPARE query(text) as SELECT count(*) FROM transform_customer_billing where inv_no = $1; The problem is Iam not able to execute this...
12
by: creo | last post by:
Hi all! this is a (relatively) newbie question I am writing a shell in Python and I am facing a problem The problem is, after taking the input from user, i have to execute the command...
15
by: tmp123 | last post by:
Hello, Thanks for your time. We have very big files with python commands (more or less, 500000 commands each file). It is possible to execute them command by command, like if the commands...
9
by: Ratfish | last post by:
I'm getting a "2014:: Commands out of sync; you can't run this command now" error on a php page when I try to call a second stored procedure against a MySQL db. Does anyone know why I might be...
0
by: xahlee | last post by:
Here's a little tutorial that lets you write emacs commands for processing the current text selection in emacs in your favorite lang. Elisp Wrapper For Perl Scripts...
0
by: jianzs | last post by:
Introduction Cloud-native applications are conventionally identified as those designed and nurtured on cloud infrastructure. Such applications, rooted in cloud technologies, skillfully benefit from...
0
by: abbasky | last post by:
### Vandf component communication method one: data sharing ​ Vandf components can achieve data exchange through data sharing, state sharing, events, and other methods. Vandf's data exchange method...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: fareedcanada | last post by:
Hello I am trying to split number on their count. suppose i have 121314151617 (12cnt) then number should be split like 12,13,14,15,16,17 and if 11314151617 (11cnt) then should be split like...
0
by: stefan129 | last post by:
Hey forum members, I'm exploring options for SSL certificates for multiple domains. Has anyone had experience with multi-domain SSL certificates? Any recommendations on reliable providers or specific...
0
Git
by: egorbl4 | last post by:
Скачал я git, хотел начать настройку, а там вылезло вот это Что это? Что мне с этим делать? ...
1
by: davi5007 | last post by:
Hi, Basically, I am trying to automate a field named TraceabilityNo into a web page from an access form. I've got the serial held in the variable strSearchString. How can I get this into the...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: Aftab Ahmad | last post by:
Hello Experts! I have written a code in MS Access for a cmd called "WhatsApp Message" to open WhatsApp using that very code but the problem is that it gives a popup message everytime I clicked on...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.