473,320 Members | 1,810 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,320 software developers and data experts.

Unexpected output while walking dirs

hi,

i am creating a program to go through a directory structure recursively
(including directories below it) and move all files that end in .msf to
a directory above the current level.

the code i have so far is as follows:

<code>

#!/usr/bin/python
#Author: Evan Carmi
#Date: 20061101
#Purpose: To uncorrupt Mozilla Thunderbird mail index files.
#Version: 2.04
top = 'f:\\test\\mail'
import os, time, itertools

#walk and find all files
allfiles = []
for root, dirs, files in os.walk(top, topdown=False):
for name in files:
allfiles.append(os.path.join(root, name))
#remove all non .msf files
index = []
for match in allfiles:
if match.endswith('.msf'):
index.append(match)
print index
#need to fix the problem with adding folders to thunderbird
indexdest = []
indexdest = ['%s\\..\\..\\%s\\%s\\%s' % (x , time.strftime('%Y%m%d%H%M%S'),
os.path.basename(os.path.normpath(x+'\\..')), os.path.basename(x)) for
x in index]
#\\.. to remove the ending and than basename to add it back on
indexdest = [os.path.normpath(i) for i in indexdest]
indexdest = [i.replace('Mail', 'backups-msf') for i in indexdest]

for a, b in itertools.izip(index, indexdest):
os.renames(a, b)

</code>

if the directory structure is:

-test
-Mail
-Local Folders
-mail.binarymanipulations.com

and there are no additional directories inside of Local Folders than everything
works and the script moves the .msf files to a directory on the same level as
-Mail named backups-msf.

the problem occurs when the directory structure:
-test
-Mail
-Local Folders
-mail.binarymanipulations.com

has additional directories inside of Local Folders. This results in very odd
behavior and the directory structure looks like:

-test
-Mail
-Local Folders
-mail.binarymanipulations.com
-20061228005643
-Local Folders
-mail.binarymanipulations.com

after running the script.
i hope that my problem is clear now, if you have any other information that
would be helpful please tell me.
I am testing this on a w2k machine with python 2.4

thanks,
Evan

Dec 28 '06 #1
2 1304


On Dec 28, 10:25 am, Evan Carmi <e...@binarymanipulations.comwrote:
hi,

i am creating a program to go through a directory structure recursively
(including directories below it) and move all files that end in .msf to
a directory above the current level.

the code i have so far is as follows:

<code>

#!/usr/bin/python
#Author: Evan Carmi
#Date: 20061101
#Purpose: To uncorrupt Mozilla Thunderbird mail index files.
#Version: 2.04
top = 'f:\\test\\mail'

import os, time, itertools

#walk and find all files
allfiles = []
for root, dirs, files in os.walk(top, topdown=False):
for name in files:
allfiles.append(os.path.join(root, name))
#remove all non .msf files
index = []
for match in allfiles:
if match.endswith('.msf'):
index.append(match)
print index
#need to fix the problem with adding folders to thunderbird
indexdest = []
indexdest = ['%s\\..\\..\\%s\\%s\\%s' % (x , time.strftime('%Y%m%d%H%M%S'),
os.path.basename(os.path.normpath(x+'\\..')), os.path.basename(x)) for
x in index]
#\\.. to remove the ending and than basename to add it back on
indexdest = [os.path.normpath(i) for i in indexdest]
indexdest = [i.replace('Mail', 'backups-msf') for i in indexdest]

for a, b in itertools.izip(index, indexdest):
os.renames(a, b)

</code>

if the directory structure is:

-test
-Mail
-Local Folders
-mail.binarymanipulations.com

and there are no additional directories inside of Local Folders than everything
works and the script moves the .msf files to a directory on the same level as
-Mail named backups-msf.

the problem occurs when the directory structure:
-test
-Mail
-Local Folders
-mail.binarymanipulations.com

has additional directories inside of Local Folders. This results in very odd
behavior and the directory structure looks like:

-test
-Mail
-Local Folders
-mail.binarymanipulations.com
-20061228005643
-Local Folders
-mail.binarymanipulations.com

after running the script.
i hope that my problem is clear now, if you have any other information that
would be helpful please tell me.
I am testing this on a w2k machine with python 2.4

thanks,
Evan
If I do get this correct - you have files like \test\Mail\somename.msf
and \test\Mail\somedirectory\someothername.msf, these files you want to
move to \test\backup\timestamp\somename.msf and
\test\backup\timestamp\somedirectory\someothername .msf.

In your code you start with collecting allfiles, and then you remove
all except *,msf files. You can do this in one step to, by collecting
only the wanted files in a listby applying the glob command from module
glob :

allfiles = []
for root, dirs, files in os.walk(top, topdown=False):
targetfiles = glob.glob(os.path.join(root,'*.msf'))
allfiles += targetfiles

Now allfiles contains the list of .msf files down from directory
specified in variable top. From here I would create a list of 2-tuples,
with as first element the full original filename, and as second element
the desired backupname. Note that in your code you call the clock in a
for loop, this might result in more directories.

backuproot = os.path.join(os.path.dirname(top), 'backup-msf',
time.strftime('%Y%m%d%H%M%S'))
backupfilenames = []
for f in allfiles:
backupfilenames.append((f, os.path.join(backuproot,
os.path.basename(f))))
>From here it is easy to do the move.
Dec 28 '06 #2
wittempj <athotmail.com <martin.witte <atgmail.comwrites:
>
If I do get this correct - you have files like \test\Mail\somename.msf
and \test\Mail\somedirectory\someothername.msf, these files you want to
move to \test\backup\timestamp\somename.msf and
\test\backup\timestamp\somedirectory\someothername .msf.

In your code you start with collecting allfiles, and then you remove
all except *,msf files. You can do this in one step to, by collecting
only the wanted files in a listby applying the glob command from module
glob :

allfiles = []
for root, dirs, files in os.walk(top, topdown=False):
targetfiles = glob.glob(os.path.join(root,'*.msf'))
allfiles += targetfiles

Now allfiles contains the list of .msf files down from directory
specified in variable top. From here I would create a list of 2-tuples,
with as first element the full original filename, and as second element
the desired backupname. Note that in your code you call the clock in a
for loop, this might result in more directories.

backuproot = os.path.join(os.path.dirname(top), 'backup-msf',
time.strftime('%Y%m%d%H%M%S'))
backupfilenames = []
for f in allfiles:
backupfilenames.append((f, os.path.join(backuproot,
os.path.basename(f))))
From here it is easy to do the move.
Thanks so much for your help, you got me started and I finished it with some
help from others as well. For anyone interested the working code is:

--
#!/usr/bin/python
#Author: Evan Carmi
#Date: 20060102
#Purpose: To uncorrupt Mozilla Thunderbird mail index files.
#Version: 1.00
import os, time, glob

srcRoot = 'f:\\test\\mail'
backupRoot = os.path.join(os.path.dirname(srcRoot), 'backup-msf',
time.strftime('%Y%m%d%H%M%S'))

for root, dirs, files in os.walk(srcRoot, topdown=False):
sources = glob.glob(os.path.join(root,'*.msf'))
pairs = []

for source in sources:
# If srcRoot is /foo/bar and source is /foo/bar/baz, let relativeSource
equal /baz
#let relativeSource be equal to the remainder of source when you take
away len(srcRoot)
idx = len(srcRoot)
relativeSource = source[idx:]

# Then let destination equal /quux/baz if backupRoot is /quux
destination = backupRoot + relativeSource
# relativeSource begins with a path separator, so os.path.join will
misinterpret it.

pair = (source, destination)
pairs.append(pair)

for pair in pairs:
os.renames(*pair)
# This is functionally equivalent to os.renames(pair[0], pair[1])

--

Thanks, Evan

Jan 2 '07 #3

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

2
by: Tony Gahlinger | last post by:
I'm learning/experimenting with some simple JS/html markup, running an apache daemon and mozilla firefox browser in RH 9. Let's say I run the following markup with one or more of lines 6-10...
2
by: Gerhard Esterhuizen | last post by:
Hi, I am observing unexpected behaviour, in the form of a corrupted class member access, from a simple C++ program that accesses an attribute declared in a virtual base class via a chain of...
5
by: Tom Lam lemontea | last post by:
Hi all, This is my very first post here, I've seriously tried some programming on C, and shown below is my very first program(So you can expect it to be very messy) that I wrote after I've learned...
2
by: Darko Bazulj | last post by:
Hi, How to make output in .txt file in next situation Dim folderspec1 As String = "e:\a" Dim n As String Dim di As DirectoryInfo = New DirectoryInfo(folderspec1) Dim dirs As...
6
by: rtilley | last post by:
Hello, When working with file and dir info recursively on Windows XP. I'm going about it like this: for root, dirs, files in os.walk(path): for f in files: ADD F to dictionary for d in...
8
by: J. Frank Parnell | last post by:
Hi there, I got this directory tree listing function somewhere, it works great, but I want it to only list directories that do have one or more .jpgs inside. I tried getting a list of each dir's...
2
by: mahesh.kanakaraj | last post by:
Hi All, I am new to C++ programming. I encounter an unexpected behavior of 'setprecision'. The code snippet is : #include <iostream.h> #include <stdio.h> #include <iomanip.h>
14
by: indu_shreenath | last post by:
Hey, I want to get the output of "DIR /AD /B" command to a varriable using python. How can I do this? Thanks, Indu
13
by: bintom | last post by:
I ran the following simple code in C++ and got unexpected results: float f = 139.4; cout << f; Output: 139.399994;
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
0
by: ryjfgjl | last post by:
ExcelToDatabase: batch import excel into database automatically...
0
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
1
by: PapaRatzi | last post by:
Hello, I am teaching myself MS Access forms design and Visual Basic. I've created a table to capture a list of Top 30 singles and forms to capture new entries. The final step is a form (unbound)...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.