By using this site, you agree to our updated Privacy Policy and our Terms of Use. Manage your Cookies Settings.
443,846 Members | 1,857 Online
Bytes IT Community
+ Ask a Question
Need help? Post your question and get tips & solutions from a community of 443,846 IT Pros & Developers. It's quick & easy.

Unexpected output while walking dirs

P: n/a
hi,

i am creating a program to go through a directory structure recursively
(including directories below it) and move all files that end in .msf to
a directory above the current level.

the code i have so far is as follows:

<code>

#!/usr/bin/python
#Author: Evan Carmi
#Date: 20061101
#Purpose: To uncorrupt Mozilla Thunderbird mail index files.
#Version: 2.04
top = 'f:\\test\\mail'
import os, time, itertools

#walk and find all files
allfiles = []
for root, dirs, files in os.walk(top, topdown=False):
for name in files:
allfiles.append(os.path.join(root, name))
#remove all non .msf files
index = []
for match in allfiles:
if match.endswith('.msf'):
index.append(match)
print index
#need to fix the problem with adding folders to thunderbird
indexdest = []
indexdest = ['%s\\..\\..\\%s\\%s\\%s' % (x , time.strftime('%Y%m%d%H%M%S'),
os.path.basename(os.path.normpath(x+'\\..')), os.path.basename(x)) for
x in index]
#\\.. to remove the ending and than basename to add it back on
indexdest = [os.path.normpath(i) for i in indexdest]
indexdest = [i.replace('Mail', 'backups-msf') for i in indexdest]

for a, b in itertools.izip(index, indexdest):
os.renames(a, b)

</code>

if the directory structure is:

-test
-Mail
-Local Folders
-mail.binarymanipulations.com

and there are no additional directories inside of Local Folders than everything
works and the script moves the .msf files to a directory on the same level as
-Mail named backups-msf.

the problem occurs when the directory structure:
-test
-Mail
-Local Folders
-mail.binarymanipulations.com

has additional directories inside of Local Folders. This results in very odd
behavior and the directory structure looks like:

-test
-Mail
-Local Folders
-mail.binarymanipulations.com
-20061228005643
-Local Folders
-mail.binarymanipulations.com

after running the script.
i hope that my problem is clear now, if you have any other information that
would be helpful please tell me.
I am testing this on a w2k machine with python 2.4

thanks,
Evan

Dec 28 '06 #1
Share this Question
Share on Google+
2 Replies


P: n/a


On Dec 28, 10:25 am, Evan Carmi <e...@binarymanipulations.comwrote:
hi,

i am creating a program to go through a directory structure recursively
(including directories below it) and move all files that end in .msf to
a directory above the current level.

the code i have so far is as follows:

<code>

#!/usr/bin/python
#Author: Evan Carmi
#Date: 20061101
#Purpose: To uncorrupt Mozilla Thunderbird mail index files.
#Version: 2.04
top = 'f:\\test\\mail'

import os, time, itertools

#walk and find all files
allfiles = []
for root, dirs, files in os.walk(top, topdown=False):
for name in files:
allfiles.append(os.path.join(root, name))
#remove all non .msf files
index = []
for match in allfiles:
if match.endswith('.msf'):
index.append(match)
print index
#need to fix the problem with adding folders to thunderbird
indexdest = []
indexdest = ['%s\\..\\..\\%s\\%s\\%s' % (x , time.strftime('%Y%m%d%H%M%S'),
os.path.basename(os.path.normpath(x+'\\..')), os.path.basename(x)) for
x in index]
#\\.. to remove the ending and than basename to add it back on
indexdest = [os.path.normpath(i) for i in indexdest]
indexdest = [i.replace('Mail', 'backups-msf') for i in indexdest]

for a, b in itertools.izip(index, indexdest):
os.renames(a, b)

</code>

if the directory structure is:

-test
-Mail
-Local Folders
-mail.binarymanipulations.com

and there are no additional directories inside of Local Folders than everything
works and the script moves the .msf files to a directory on the same level as
-Mail named backups-msf.

the problem occurs when the directory structure:
-test
-Mail
-Local Folders
-mail.binarymanipulations.com

has additional directories inside of Local Folders. This results in very odd
behavior and the directory structure looks like:

-test
-Mail
-Local Folders
-mail.binarymanipulations.com
-20061228005643
-Local Folders
-mail.binarymanipulations.com

after running the script.
i hope that my problem is clear now, if you have any other information that
would be helpful please tell me.
I am testing this on a w2k machine with python 2.4

thanks,
Evan
If I do get this correct - you have files like \test\Mail\somename.msf
and \test\Mail\somedirectory\someothername.msf, these files you want to
move to \test\backup\timestamp\somename.msf and
\test\backup\timestamp\somedirectory\someothername .msf.

In your code you start with collecting allfiles, and then you remove
all except *,msf files. You can do this in one step to, by collecting
only the wanted files in a listby applying the glob command from module
glob :

allfiles = []
for root, dirs, files in os.walk(top, topdown=False):
targetfiles = glob.glob(os.path.join(root,'*.msf'))
allfiles += targetfiles

Now allfiles contains the list of .msf files down from directory
specified in variable top. From here I would create a list of 2-tuples,
with as first element the full original filename, and as second element
the desired backupname. Note that in your code you call the clock in a
for loop, this might result in more directories.

backuproot = os.path.join(os.path.dirname(top), 'backup-msf',
time.strftime('%Y%m%d%H%M%S'))
backupfilenames = []
for f in allfiles:
backupfilenames.append((f, os.path.join(backuproot,
os.path.basename(f))))
>From here it is easy to do the move.
Dec 28 '06 #2

P: n/a
wittempj <athotmail.com <martin.witte <atgmail.comwrites:
>
If I do get this correct - you have files like \test\Mail\somename.msf
and \test\Mail\somedirectory\someothername.msf, these files you want to
move to \test\backup\timestamp\somename.msf and
\test\backup\timestamp\somedirectory\someothername .msf.

In your code you start with collecting allfiles, and then you remove
all except *,msf files. You can do this in one step to, by collecting
only the wanted files in a listby applying the glob command from module
glob :

allfiles = []
for root, dirs, files in os.walk(top, topdown=False):
targetfiles = glob.glob(os.path.join(root,'*.msf'))
allfiles += targetfiles

Now allfiles contains the list of .msf files down from directory
specified in variable top. From here I would create a list of 2-tuples,
with as first element the full original filename, and as second element
the desired backupname. Note that in your code you call the clock in a
for loop, this might result in more directories.

backuproot = os.path.join(os.path.dirname(top), 'backup-msf',
time.strftime('%Y%m%d%H%M%S'))
backupfilenames = []
for f in allfiles:
backupfilenames.append((f, os.path.join(backuproot,
os.path.basename(f))))
From here it is easy to do the move.
Thanks so much for your help, you got me started and I finished it with some
help from others as well. For anyone interested the working code is:

--
#!/usr/bin/python
#Author: Evan Carmi
#Date: 20060102
#Purpose: To uncorrupt Mozilla Thunderbird mail index files.
#Version: 1.00
import os, time, glob

srcRoot = 'f:\\test\\mail'
backupRoot = os.path.join(os.path.dirname(srcRoot), 'backup-msf',
time.strftime('%Y%m%d%H%M%S'))

for root, dirs, files in os.walk(srcRoot, topdown=False):
sources = glob.glob(os.path.join(root,'*.msf'))
pairs = []

for source in sources:
# If srcRoot is /foo/bar and source is /foo/bar/baz, let relativeSource
equal /baz
#let relativeSource be equal to the remainder of source when you take
away len(srcRoot)
idx = len(srcRoot)
relativeSource = source[idx:]

# Then let destination equal /quux/baz if backupRoot is /quux
destination = backupRoot + relativeSource
# relativeSource begins with a path separator, so os.path.join will
misinterpret it.

pair = (source, destination)
pairs.append(pair)

for pair in pairs:
os.renames(*pair)
# This is functionally equivalent to os.renames(pair[0], pair[1])

--

Thanks, Evan

Jan 2 '07 #3

This discussion thread is closed

Replies have been disabled for this discussion.