473,287 Members | 1,565 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,287 software developers and data experts.

os.walk and recursive deletion

Hello,

I'm playing around with os.walk and I made up del_tree(path) which I
think is correct (in terms of the algorithm, but not as python wants
it :)).

As soon as some directory is deleted the iterator of os.walk chokes.
OK that is somehow clear to me as it can't be valid anymore since it
can't go to the just deleted directory but it is in the iterator.

I generated the test data with (bash, don't know of other shell
expansion features)

# root="test"
# mkdir $root
# mkdir -p
# $root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}
# touch $root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{0,1,2,3,4,5,6,7,8,9}
# touch $root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{0,1,2,3,4,5,6,7,8,9}

Here goes the script, as far as I tested it it runs fine in test mode.
Now that is just use the print statements like attached below. as soon
as os.rmdir and os.unlink are involved the iterator chokes. How do I
get around that, or what should I use to be able to recursively delete
everything under a certain subdir?

#--snip
import sys
import os
import unittest

def del_tree(path):
walker = os.walk(path)
for entry in walker:
cwd = entry[0]
subdirs = entry[1]
files = entry[2]
print "Checking directory: %s" %(cwd, )
if files:
for file in files:
file = os.path.join(cwd, file)
print "The file is now: %s" % (file, )
#os.unlink(file)
print "OK directory %s has no more files, checking for
subdirs..." % (cwd, )
else:
print "OK directory %s NEVER HAD FILES, checking for
subdirs..." % (cwd, )
if not subdirs:
print "We can delete: %s" % (cwd, )
#os.rmdir(cwd)
else:
for subdir in subdirs:
subdir = os.path.join(cwd, subdir)
print "We need to recurse into: %s" % (subdir, )
del_tree(subdir)
#os.rmdir(path)
print "Removing: %s" % (path, )
#--snap
--
http://noneisyours.marcher.name
http://feeds.feedburner.com/NoneIsYours
Oct 27 '07 #1
2 2987
Martin Marcher wrote:
Hello,

I'm playing around with os.walk and I made up del_tree(path) which I
think is correct (in terms of the algorithm, but not as python wants
it :)).

As soon as some directory is deleted the iterator of os.walk chokes.
OK that is somehow clear to me as it can't be valid anymore since it
can't go to the just deleted directory but it is in the iterator.

I generated the test data with (bash, don't know of other shell
expansion features)

# root="test"
# mkdir $root
# mkdir -p
# $root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}
# touch $root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{0,1,2,3,4,5,6,7,8,9}
# touch $root/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{a,b,c,d,e,f,g,h,i,j,k,l,m,n,o,p,q,r,s,t,u,v,w,x,y ,z}/{0,1,2,3,4,5,6,7,8,9}

Here goes the script, as far as I tested it it runs fine in test mode.
Now that is just use the print statements like attached below. as soon
as os.rmdir and os.unlink are involved the iterator chokes. How do I
get around that, or what should I use to be able to recursively delete
everything under a certain subdir?

#--snip
import sys
import os
import unittest

def del_tree(path):
walker = os.walk(path)
for entry in walker:
cwd = entry[0]
subdirs = entry[1]
files = entry[2]
print "Checking directory: %s" %(cwd, )
if files:
for file in files:
file = os.path.join(cwd, file)
print "The file is now: %s" % (file, )
#os.unlink(file)
print "OK directory %s has no more files, checking for
subdirs..." % (cwd, )
else:
print "OK directory %s NEVER HAD FILES, checking for
subdirs..." % (cwd, )
if not subdirs:
print "We can delete: %s" % (cwd, )
#os.rmdir(cwd)
else:
for subdir in subdirs:
subdir = os.path.join(cwd, subdir)
print "We need to recurse into: %s" % (subdir, )
del_tree(subdir)
#os.rmdir(path)
print "Removing: %s" % (path, )
#--snap
While it is possible to mix recursion and os.walk()...

def del_tree(path):
for cwd, subdirs, files in os.walk(path):
print "Checking directory: %s" %(cwd, )
if files:
for file in files:
file = os.path.join(cwd, file)
print "The file is now: %s" % (file, )
os.unlink(file)
print "OK directory %s has no more files, checking for subdirs..." % (cwd, )
else:
print "OK directory %s NEVER HAD FILES, checking for subdirs..." % (cwd, )
for subdir in subdirs:
subdir = os.path.join(cwd, subdir)
print "We need to recurse into: %s" % (subdir, )
del_tree(subdir)
break # ugly but necessary
print "Removing: %s" % (path, )
os.rmdir(path)

....the idea behind it is that you use it instead of recursion:

def del_tree(root):
for path, dirs, files in os.walk(root, False):
for fn in files:
os.unlink(os.path.join(path, fn))
for dn in dirs:
os.rmdir(os.path.join(path, dn))
os.rmdir(root)

The second parameter to os.walk() ensures that the tree is walked
depth-first, i. e. the contents of a directory are seen before the
directory itself. Personally, I would prefer a recursive implementation
based on os.listdir(),

def del_tree(root):
for name in os.listdir(root):
path = os.path.join(root, name)
if os.path.isdir(path):
del_tree(path)
else:
os.unlink(path)
os.rmdir(root)

an approach that is also taken by shutils.rmtree().

Peter
Oct 27 '07 #2
On Sat, 27 Oct 2007 18:07:44 +0200, Martin Marcher wrote:
Hello,

I'm playing around with os.walk and I made up del_tree(path) which I
think is correct (in terms of the algorithm, but not as python wants
it :)).
It's not correct in terms of the algorithm if you take the algorithm of
`os.walk()` into the equation.
Here goes the script, as far as I tested it it runs fine in test mode.
Now that is just use the print statements like attached below. as soon
as os.rmdir and os.unlink are involved the iterator chokes. How do I
get around that, or what should I use to be able to recursively delete
everything under a certain subdir?
#--snip
import sys
import os
import unittest

def del_tree(path):
walker = os.walk(path)
`os.walk()` is itself diving recursivly into the subdirectories…
for entry in walker:
cwd = entry[0]
subdirs = entry[1]
files = entry[2]
print "Checking directory: %s" %(cwd, )
if files:
for file in files:
file = os.path.join(cwd, file)
print "The file is now: %s" % (file, )
#os.unlink(file)
print "OK directory %s has no more files, checking for
subdirs..." % (cwd, )
else:
print "OK directory %s NEVER HAD FILES, checking for
subdirs..." % (cwd, )
if not subdirs:
print "We can delete: %s" % (cwd, )
#os.rmdir(cwd)
else:
for subdir in subdirs:
subdir = os.path.join(cwd, subdir)
print "We need to recurse into: %s" % (subdir, )
del_tree(subdir)
…and here you are calling the your function recursively which then calls
again `os.walk()` on that subdirectory. That's a little bit too much.

Just use `os.listdir()` (and `os.path.isdir()`) in your recursive function.
#os.rmdir(path)
print "Removing: %s" % (path, )
#--snap
Or `shutil.rmtree()`. :-)

Ciao,
Marc 'BlackJack' Rintsch
Oct 27 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

9
by: hokieghal99 | last post by:
This script is not recursive... in order to make it recursive, I have to call it several times (my kludge... hey, it works). I thought os.walk's sole purpose was to recursively walk a directory...
9
by: Marcello Pietrobon | last post by:
Hello, I am using Pyton 2.3 I desire to walk a directory without recursion this only partly works: def walk_files() : for root, dirs, files in os.walk(top, topdown=True): for filename in...
0
by: Frans Englich | last post by:
Hello, Have a look at this recursive function: def walkDirectory( directory, element ): element = element.newChild( None, "directory", None ) # automatically appends to parent...
1
by: gusmeister | last post by:
Is there a Perl mod that has a recursive file copy function (similar to `cp -r` in Unix or `xcopy` in Windows)? File::Path does have a recursive file deletion function (rmtree) but no recursive...
3
by: SB | last post by:
Hi, I have a super-simple need to just walk the files in a single directory. I thought this would do it, but "permanentFilelist" ends up containing all folders in all subdirectories. Could...
22
by: delraydog | last post by:
It's quite simple to walk to the DOM tree going forward however I can't figure out a nice clean way to walk the DOM tree in reverse. Checking previousSibling is not sufficient as the...
5
by: Terry Olsen | last post by:
How can I walk through the InnerExceptions? Would the following code be correct? Private Sub ShowException(ByVal ex As Exception) MsgBox(ex.Message) If ex.InnerException Is Nothing = False Then...
5
by: Gordon | last post by:
I'm trying to remove a directory and all its contents from within a script. I wrote a recursive function to take care of it, but when I run it I get random "Directory not empty" error messages. ...
0
by: Fredrik Lundh | last post by:
A. Joseph wrote: os.walk traverses the directory tree, so I'm not sure why you think that your program needs to use recursion? wouldn't a plain loop work? import os, shutil for dirpath,...
2
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 7 Feb 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:30 (7.30PM). In this month's session, the creator of the excellent VBE...
0
by: DolphinDB | last post by:
The formulas of 101 quantitative trading alphas used by WorldQuant were presented in the paper 101 Formulaic Alphas. However, some formulas are complex, leading to challenges in calculation. Take...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: Aftab Ahmad | last post by:
So, I have written a code for a cmd called "Send WhatsApp Message" to open and send WhatsApp messaage. The code is given below. Dim IE As Object Set IE =...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: marcoviolo | last post by:
Dear all, I would like to implement on my worksheet an vlookup dynamic , that consider a change of pivot excel via win32com, from an external excel (without open it) and save the new file into a...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: jfyes | last post by:
As a hardware engineer, after seeing that CEIWEI recently released a new tool for Modbus RTU Over TCP/UDP filtering and monitoring, I actively went to its official website to take a look. It turned...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.