473,387 Members | 1,687 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,387 software developers and data experts.

os.walk and recursive deletion

Hello,

I'm playing around with os.walk and I made up del_tree(path) which I
think is correct (in terms of the algorithm, but not as python wants
it :)).

As soon as some directory is deleted the iterator of os.walk chokes.
OK that is somehow clear to me as it can't be valid anymore since it
can't go to the just deleted directory but it is in the iterator.

I generated the test data with (bash, don't know of other shell
expansion features)

# root="test"
# mkdir $root
# mkdir -p
# $root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}
# touch $root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{0,1,2,3,4,5,6,7,8,9}
# touch $root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{0,1,2,3,4,5,6,7,8,9}

Here goes the script, as far as I tested it it runs fine in test mode.
Now that is just use the print statements like attached below. as soon
as os.rmdir and os.unlink are involved the iterator chokes. How do I
get around that, or what should I use to be able to recursively delete
everything under a certain subdir?

#--snip
import sys
import os
import unittest

def del_tree(path):
walker = os.walk(path)
for entry in walker:
cwd = entry[0]
subdirs = entry[1]
files = entry[2]
print "Checking directory: %s" %(cwd, )
if files:
for file in files:
file = os.path.join(cwd, file)
print "The file is now: %s" % (file, )
#os.unlink(file)
print "OK directory %s has no more files, checking for
subdirs..." % (cwd, )
else:
print "OK directory %s NEVER HAD FILES, checking for
subdirs..." % (cwd, )
if not subdirs:
print "We can delete: %s" % (cwd, )
#os.rmdir(cwd)
else:
for subdir in subdirs:
subdir = os.path.join(cwd, subdir)
print "We need to recurse into: %s" % (subdir, )
del_tree(subdir)
#os.rmdir(path)
print "Removing: %s" % (path, )
#--snap
--
http://noneisyours.marcher.name
http://feeds.feedburner.com/NoneIsYours
Oct 27 '07 #1
2 2989
Martin Marcher wrote:
Hello,

I'm playing around with os.walk and I made up del_tree(path) which I
think is correct (in terms of the algorithm, but not as python wants
it :)).

As soon as some directory is deleted the iterator of os.walk chokes.
OK that is somehow clear to me as it can't be valid anymore since it
can't go to the just deleted directory but it is in the iterator.

I generated the test data with (bash, don't know of other shell
expansion features)

# root="test"
# mkdir $root
# mkdir -p
# $root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}
# touch $root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{0,1,2,3,4,5,6,7,8,9}
# touch $root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{0,1,2,3,4,5,6,7,8,9}

Here goes the script, as far as I tested it it runs fine in test mode.
Now that is just use the print statements like attached below. as soon
as os.rmdir and os.unlink are involved the iterator chokes. How do I
get around that, or what should I use to be able to recursively delete
everything under a certain subdir?

#--snip
import sys
import os
import unittest

def del_tree(path):
walker = os.walk(path)
for entry in walker:
cwd = entry[0]
subdirs = entry[1]
files = entry[2]
print "Checking directory: %s" %(cwd, )
if files:
for file in files:
file = os.path.join(cwd, file)
print "The file is now: %s" % (file, )
#os.unlink(file)
print "OK directory %s has no more files, checking for
subdirs..." % (cwd, )
else:
print "OK directory %s NEVER HAD FILES, checking for
subdirs..." % (cwd, )
if not subdirs:
print "We can delete: %s" % (cwd, )
#os.rmdir(cwd)
else:
for subdir in subdirs:
subdir = os.path.join(cwd, subdir)
print "We need to recurse into: %s" % (subdir, )
del_tree(subdir)
#os.rmdir(path)
print "Removing: %s" % (path, )
#--snap
While it is possible to mix recursion and os.walk()...

def del_tree(path):
for cwd, subdirs, files in os.walk(path):
print "Checking directory: %s" %(cwd, )
if files:
for file in files:
file = os.path.join(cwd, file)
print "The file is now: %s" % (file, )
os.unlink(file)
print "OK directory %s has no more files, checking for subdirs..." % (cwd, )
else:
print "OK directory %s NEVER HAD FILES, checking for subdirs..." % (cwd, )
for subdir in subdirs:
subdir = os.path.join(cwd, subdir)
print "We need to recurse into: %s" % (subdir, )
del_tree(subdir)
break # ugly but necessary
print "Removing: %s" % (path, )
os.rmdir(path)

....the idea behind it is that you use it instead of recursion:

def del_tree(root):
for path, dirs, files in os.walk(root, False):
for fn in files:
os.unlink(os.path.join(path, fn))
for dn in dirs:
os.rmdir(os.path.join(path, dn))
os.rmdir(root)

The second parameter to os.walk() ensures that the tree is walked
depth-first, i. e. the contents of a directory are seen before the
directory itself. Personally, I would prefer a recursive implementation
based on os.listdir(),

def del_tree(root):
for name in os.listdir(root):
path = os.path.join(root, name)
if os.path.isdir(path):
del_tree(path)
else:
os.unlink(path)
os.rmdir(root)

an approach that is also taken by shutils.rmtree().

Peter
Oct 27 '07 #2
On Sat, 27 Oct 2007 18:07:44 +0200, Martin Marcher wrote:
Hello,

I'm playing around with os.walk and I made up del_tree(path) which I
think is correct (in terms of the algorithm, but not as python wants
it :)).
It's not correct in terms of the algorithm if you take the algorithm of
`os.walk()` into the equation.
Here goes the script, as far as I tested it it runs fine in test mode.
Now that is just use the print statements like attached below. as soon
as os.rmdir and os.unlink are involved the iterator chokes. How do I
get around that, or what should I use to be able to recursively delete
everything under a certain subdir?
#--snip
import sys
import os
import unittest

def del_tree(path):
walker = os.walk(path)
`os.walk()` is itself diving recursivly into the subdirectories…
for entry in walker:
cwd = entry[0]
subdirs = entry[1]
files = entry[2]
print "Checking directory: %s" %(cwd, )
if files:
for file in files:
file = os.path.join(cwd, file)
print "The file is now: %s" % (file, )
#os.unlink(file)
print "OK directory %s has no more files, checking for
subdirs..." % (cwd, )
else:
print "OK directory %s NEVER HAD FILES, checking for
subdirs..." % (cwd, )
if not subdirs:
print "We can delete: %s" % (cwd, )
#os.rmdir(cwd)
else:
for subdir in subdirs:
subdir = os.path.join(cwd, subdir)
print "We need to recurse into: %s" % (subdir, )
del_tree(subdir)
…and here you are calling the your function recursively which then calls
again `os.walk()` on that subdirectory. That's a little bit too much.

Just use `os.listdir()` (and `os.path.isdir()`) in your recursive function.
#os.rmdir(path)
print "Removing: %s" % (path, )
#--snap
Or `shutil.rmtree()`. :-)

Ciao,
Marc 'BlackJack' Rintsch
Oct 27 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

9
by: hokieghal99 | last post by:
This script is not recursive... in order to make it recursive, I have to call it several times (my kludge... hey, it works). I thought os.walk's sole purpose was to recursively walk a directory...
9
by: Marcello Pietrobon | last post by:
Hello, I am using Pyton 2.3 I desire to walk a directory without recursion this only partly works: def walk_files() : for root, dirs, files in os.walk(top, topdown=True): for filename in...
0
by: Frans Englich | last post by:
Hello, Have a look at this recursive function: def walkDirectory( directory, element ): element = element.newChild( None, "directory", None ) # automatically appends to parent...
1
by: gusmeister | last post by:
Is there a Perl mod that has a recursive file copy function (similar to `cp -r` in Unix or `xcopy` in Windows)? File::Path does have a recursive file deletion function (rmtree) but no recursive...
3
by: SB | last post by:
Hi, I have a super-simple need to just walk the files in a single directory. I thought this would do it, but "permanentFilelist" ends up containing all folders in all subdirectories. Could...
22
by: delraydog | last post by:
It's quite simple to walk to the DOM tree going forward however I can't figure out a nice clean way to walk the DOM tree in reverse. Checking previousSibling is not sufficient as the...
5
by: Terry Olsen | last post by:
How can I walk through the InnerExceptions? Would the following code be correct? Private Sub ShowException(ByVal ex As Exception) MsgBox(ex.Message) If ex.InnerException Is Nothing = False Then...
5
by: Gordon | last post by:
I'm trying to remove a directory and all its contents from within a script. I wrote a recursive function to take care of it, but when I run it I get random "Directory not empty" error messages. ...
0
by: Fredrik Lundh | last post by:
A. Joseph wrote: os.walk traverses the directory tree, so I'm not sure why you think that your program needs to use recursion? wouldn't a plain loop work? import os, shutil for dirpath,...
0
by: taylorcarr | last post by:
A Canon printer is a smart device known for being advanced, efficient, and reliable. It is designed for home, office, and hybrid workspace use and can also be used for a variety of purposes. However,...
0
by: ryjfgjl | last post by:
If we have dozens or hundreds of excel to import into the database, if we use the excel import function provided by database editors such as navicat, it will be extremely tedious and time-consuming...
0
by: emmanuelkatto | last post by:
Hi All, I am Emmanuel katto from Uganda. I want to ask what challenges you've faced while migrating a website to cloud. Please let me know. Thanks! Emmanuel
0
BarryA
by: BarryA | last post by:
What are the essential steps and strategies outlined in the Data Structures and Algorithms (DSA) roadmap for aspiring data scientists? How can individuals effectively utilize this roadmap to progress...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can...
0
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers,...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.