473,836 Members | 1,563 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

Walk thru each subdirectory from a top directory


i am trying to use python to walk thru each subdirectory from a top
directory. Here is my script:

savedPagesDirec tory = "/home/meryl/saved_pages/data"

dir=open(savedP agesDirectory, 'r')

for file in dir:
if (isdir(file)):
# get the full path of the file
fileName = savedPagesDirec tory + file + 'index.html'
print fileName

$ ./scripts/regressionTest. py
Traceback (most recent call last):
File "./scripts/regressionTest. py", line 12, in ?
dir=open(savedP agesDirectory, 'r')
IOError: [Errno 21] Is a directory

But I get the above error:

Can you please tell me what did I do wrong?

Thank you.

Feb 26 '07 #1
9 2882
si************* **@gmail.com schrieb:
i am trying to use python to walk thru each subdirectory from a top
directory. Here is my script:

savedPagesDirec tory = "/home/meryl/saved_pages/data"

dir=open(savedP agesDirectory, 'r')

for file in dir:
if (isdir(file)):
# get the full path of the file
fileName = savedPagesDirec tory + file + 'index.html'
print fileName

$ ./scripts/regressionTest. py
Traceback (most recent call last):
File "./scripts/regressionTest. py", line 12, in ?
dir=open(savedP agesDirectory, 'r')
IOError: [Errno 21] Is a directory

But I get the above error:

Can you please tell me what did I do wrong?
You can't open a directory.

Use the function os.walk to do what you want.

Diez
Feb 26 '07 #2
On Feb 26, 9:28 pm, silverburgh.me. ..@gmail.com wrote:
i am trying to use python to walk thru each subdirectory from a top
directory. Here is my script:

savedPagesDirec tory = "/home/meryl/saved_pages/data"

dir=open(savedP agesDirectory, 'r')

for file in dir:
if (isdir(file)):
# get the full path of the file
fileName = savedPagesDirec tory + file + 'index.html'
print fileName

$ ./scripts/regressionTest. py
Traceback (most recent call last):
File "./scripts/regressionTest. py", line 12, in ?
dir=open(savedP agesDirectory, 'r')
IOError: [Errno 21] Is a directory

But I get the above error:

Can you please tell me what did I do wrong?

Thank you.
>From Alan Gaulds Tut.
>>for t in os.walk('Root') :
.... print t
....
('Root', ['D1', 'D2', 'D3'], ['FA.txt', 'FB.txt'])
('Root/D1', ['D1-1'], ['FC.txt'])
('Root/D1/D1-1', [], ['FF.txt'])
('Root/D2', [], ['FD.txt'])
('Root/D3', ['D3-1'], ['FE.txt'])
('Root/D3/D3-1', [], ['target.txt'])
>>>

This bit below is from one of my first programs as I'm currently
learning. It is designed to go form the root down and return the full
paths of everything it finds into a list. (I then check the reults for
files paths that exceed a certain length - but you don't need to know
that.)
def findallfiles(se lf, base):
self.results = []
for root,dirs,files in os.walk(base):
os.chdir(root)
self.scan = glob.glob("*")
for r in self.scan:
if root[-1] == "\\":
self.results.ap pend(root + r)
else:
self.results.ap pend(root + "\\" + r)
return self.results
Feb 26 '07 #3
Whoops, the first bit of my reply ended up in the quoted text. See
above.

Adam

Feb 26 '07 #4
Adam wrote:
On Feb 26, 9:28 pm, silverburgh.me. ..@gmail.com wrote:
>i am trying to use python to walk thru each subdirectory from a top
directory. Here is my script: ....

This bit below is from one of my first programs as I'm currently
learning. It is designed to go form the root down and return the full
paths of everything it finds into a list. (I then check the reults for
files paths that exceed a certain length - but you don't need to know
that.)
def findallfiles(se lf, base):
self.results = []
for root,dirs,files in os.walk(base):
os.chdir(root)
^^^ Mistake here, don't change directories during os.walk ^^^
self.scan = glob.glob("*")
for r in self.scan:
if root[-1] == "\\":
self.results.ap pend(root + r)
else:
self.results.ap pend(root + "\\" + r)
return self.results
def produce_all_fil es(base):
for root, dirs, files in os.walk(base):
for r in files:
yield os.path.join(ro ot, r)
### possibly also (but I'd only go for files)
#for r in dirs:
# yield os.path.join(ro ot, r)

def findallfiles(ba se):
return list(produce_al l_files(base))

--
--Scott David Daniels
sc***********@a cm.org
Feb 28 '07 #5
On Tue, 27 Feb 2007 20:31:43 -0800, Scott David Daniels wrote:
> def findallfiles(se lf, base):
self.results = []
for root,dirs,files in os.walk(base):
os.chdir(root)
^^^ Mistake here, don't change directories during os.walk ^^^
I came across this problem some time ago. I had to walk a directory tree,
calling an external program on each file. Unfortunately, that external
program wrote directly to the current working directory, which caused all
sorts of havoc. This is how I dealt with it:
def unbin(where):
"""Walk through a directory tree, calling macunpack to extract the
contents of MacBinary files.
"""
def _unbin(data, dirname, files):
for oldname in files:
fullname = os.path.normpat h(os.path.join( dirname, oldname))
if os.path.isfile( fullname):
# Dammit, macunpack writes directly to the current
# working directory. Changing the cwd breaks the file
# tree walker, so we have to remember the current
# directory, change it to where we want to be, then
# change it back.
wd = os.getcwd()
os.chdir(dirnam e)
result = os.system('macu npack -f "%s"' % oldname)
if result == 0:
# Unpacking worked, so delete the original.
os.remove(oldna me)
os.chdir(wd) # sigh...

os.path.walk(wh ere, _unbin, None)
Is there another (better) way of dealing with this sort of situation?
--
Steven D'Aprano

Feb 28 '07 #6
On Tue, 27 Feb 2007 06:22:51 +0000, Dennis Lee Bieber wrote:

[snip 350-odd file names]
Just a sample of the start of a very long listing...
Thanks for sharing. What's with the nude ferret?

--
Steven D'Aprano

Feb 28 '07 #7
Steven D'Aprano wrote:
On Tue, 27 Feb 2007 20:31:43 -0800, Scott David Daniels wrote:
>> def findallfiles(se lf, base):
self.results = []
for root,dirs,files in os.walk(base):
os.chdir(root)
^^^ Mistake here, don't change directories during os.walk ^^^

I came across this problem some time ago. I had to walk a directory tree,
calling an external program on each file. Unfortunately, that external
program wrote directly to the current working directory, which caused all
sorts of havoc. This is how I dealt with it:
def unbin(where):
"""Walk through a directory tree, calling macunpack to extract the
contents of MacBinary files.
"""
def _unbin(data, dirname, files):
for oldname in files:
fullname = os.path.normpat h(os.path.join( dirname, oldname))
if os.path.isfile( fullname):
# Dammit, macunpack writes directly to the current
# working directory. Changing the cwd breaks the file
# tree walker, so we have to remember the current
# directory, change it to where we want to be, then
# change it back.
wd = os.getcwd()
os.chdir(dirnam e)
result = os.system('macu npack -f "%s"' % oldname)
if result == 0:
# Unpacking worked, so delete the original.
os.remove(oldna me)
os.chdir(wd) # sigh...

os.path.walk(wh ere, _unbin, None)
Is there another (better) way of dealing with this sort of situation?
Does the problem occur if you pass an absolute path to
os.walk()/os.path.walk()?

Peter
Feb 28 '07 #8
On Wed, 28 Feb 2007 08:38:41 +0100, Peter Otten wrote:
Does the problem occur if you pass an absolute path to
os.walk()/os.path.walk()?
Well, arguably the problem is that macunpack insists on writing to the
current working directory.

Or the problem is that os.walk sets the working directory to the initial
argument when you start, and then keeps it until it finishes. (Arguably
that's the right behaviour.)

You can play around with this to check:

def walker(where):
"""Walk through a directory tree, printing the current working directory."
def _walk(data, dirname, files):
print "dirname = '%s'; wd = '%s'" % (dirname, os.getcwd())
os.path.walk(wh ere, _walk, None)
For those times when os.walk's behaviour doesn't mesh well with that of
the external program you are calling (like macunpack) is there an
alternative to:

- save the cwd;
- change directories;
- call the program;
- return to the saved directory

?

--
Steven D'Aprano

Mar 1 '07 #9
Steven D'Aprano wrote:
For those times when os.walk's behaviour doesn't mesh well with that of
the external program you are calling (like macunpack) is there an
alternative to:

- save the cwd;
- change directories;
- call the program;
- return to the saved directory

?
os.walk() creates paths from the root you provide and the files/direcotories
it finds via os.listdir(), and uses that to determine. If that root is an
absolute path any such paths are absolute, too, and undoing the directory
change is not necessary.

cwd = os.getcwd()
for path, dirs, files in os.walk(os.path .abspath(root)) :
os.chdir(path)
for file in files:
invoke external program
os.chdir(cwd)

That said, undoing the change immediately is better style, and probably a
good usecase for a with statement like

with current_working _directory(path ):
invoke external program

Last minute idea:

for path, dirs, files in os.walk(root):
for file in files:
os.system("cd '%s' && macunpack '%s'" % (path, file))

Peter

Mar 2 '07 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
1327
by: Frans Englich | last post by:
Hello, Have a look at this recursive function: def walkDirectory( directory, element ): element = element.newChild( None, "directory", None ) # automatically appends to parent element.setProp( "name", os.path.basename(directory))
4
1906
by: Bill | last post by:
Will IIS6.0 recognize a global.asa file located in a subdirectory on the webserver? I'd like a file that will execute whenever a file in the subdirectory is browsed - or, at least it is executed the first time a file in the subdirectory is browsed. Thanks!
7
4571
by: KraftDiner | last post by:
The os.walk function walks the operating systems directory tree. This seems to work, but I don't quite understand the tupple that is returned... Can someone explain please? for root, dirs, files in os.walk('/directory/'): print root # print dirs # print files
2
4025
by: gregpinero | last post by:
In the example from help(os.walk) it lists this: from os.path import join, getsize for root, dirs, files in walk('python/Lib/email'): print root, "consumes", print sum(), print "bytes in", len(files), "non-directory files" if 'CVS' in dirs: dirs.remove('CVS') # don't visit CVS directories
2
3024
by: Martin Marcher | last post by:
Hello, I'm playing around with os.walk and I made up del_tree(path) which I think is correct (in terms of the algorithm, but not as python wants it :)). As soon as some directory is deleted the iterator of os.walk chokes. OK that is somehow clear to me as it can't be valid anymore since it can't go to the just deleted directory but it is in the iterator.
2
3803
by: dj | last post by:
Hello All, I am attempting to us os.walk to populate two lists with values from a directory. The first list contains all the files in the directory and subdirectories. The second list contains only the files in the subdirectories. Here is the code: import os
0
2046
by: Jeff McNeil | last post by:
Your args are fine, that's just the way os.path.walk works. If you just need the absolute pathname of a directory when given a relative path, you can always use os.path.abspath, too. A couple more examples that may help, using os.walk: .... for j in i + i: .... print os.path.join(i, j) .... /var/log/apache2
4
2473
by: Jeff Nyman | last post by:
Greetings all. I did some searching on this but I can't seem to find a specific solution. I have code like this: ========================================= def walker1(arg, dirname, names): DC_List.append((dirname,'')) os.path.walk('\\\\vcdcflx006\\Flex\\Sites', walker1, 0)
6
21172
by: D | last post by:
Hello, How can one exclude a directory (and all its subdirectories) when running os.walk()? Thanks, Doug
0
9812
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
10534
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10579
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
1
7775
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5644
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
0
5814
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?
1
4444
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
2
4004
muto222
by: muto222 | last post by:
How can i add a mobile payment intergratation into php mysql website.
3
3103
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.