473,322 Members | 1,241 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,322 software developers and data experts.

using functions and file renaming problem

A few questions about the following code. How would I "wrap" this in a
function, and do I need to?

Also, how can I make the code smart enough to realize that when a file
has 2 or more bad charcters in it, that the code needs to run until all
bad characters are gone? For example, if a file has the name
"<bad*mac\file" the program has to run 3 times to get all three bad
chars out of the file name.

The passes look like this:

1. <bad*mac\file becomes -bad*mac\file
2. -bad*mac\file becomes -bad-mac\file
3. -bad-mac\file becomes -bad-mac-file

I think the problem is that once the program finds a bad char in a
filename it replaces it with a dash, which creates a new filename that
wasn't present when the program first ran, thus it has to run again to
see the new filename.

import os, re, string
bad = re.compile(r'%2f|%25|[*?<>/\|\\]') #search for these.
print " "
setpath = raw_input("Path to the dir that you would like to clean: ")
print " "
print "--- Remove Bad Charaters From Filenames ---"
print " "
for root, dirs, files in os.walk(setpath):
for file in files:
badchars = bad.findall(file)
newfile = ''
for badchar in badchars:
newfile = file.replace(badchar,'-') #replace bad chars.
if newfile:
newpath = os.path.join(root,newfile)
oldpath = os.path.join(root,file)
os.rename(oldpath,newpath)
print oldpath
print newpath
print " "
print "--- Done ---"
print " "

Jul 18 '05 #1
5 3309
bo**@oz.net (Bengt Richter) wrote in message
This looks like an old post that ignores some responses you got to your original post like this.
Did some mail get lost? Or was this an accidental repost of something old? I still see
indentation misalignments, probably due to mixing tabs and spaces (bad news in python ;-)

Regards,
Bengt Richter


Sorry Bengt, I overlooked some responses to an earlier, similar
question. I have too many computers in too many places, so forgive me.
Thanks for taking the time to tell me again!!! I'll clean up the
indentation... I promise.
Jul 18 '05 #2
On Friday 18 Jul 2003 2:33 am, hokiegal99 wrote:
A few questions about the following code. How would I "wrap" this in a
function, and do I need to?

Also, how can I make the code smart enough to realize that when a file
has 2 or more bad charcters in it, that the code needs to run until all
bad characters are gone?
It almost is ;-)
For example, if a file has the name
"<bad*mac\file" the program has to run 3 times to get all three bad
chars out of the file name.

The passes look like this:

1. <bad*mac\file becomes -bad*mac\file
2. -bad*mac\file becomes -bad-mac\file
3. -bad-mac\file becomes -bad-mac-file

I think the problem is that once the program finds a bad char in a
filename it replaces it with a dash, which creates a new filename that
wasn't present when the program first ran, thus it has to run again to
see the new filename.


No, the problem is that you're throwing away all but the last correction.
Read my comments below:

import os, re, string
bad = re.compile(r'%2f|%25|[*?<>/\|\\]') #search for these.
print " "
setpath = raw_input("Path to the dir that you would like to clean: ")
print " "
print "--- Remove Bad Charaters From Filenames ---"
print " "
for root, dirs, files in os.walk(setpath):
for file in files:
badchars = bad.findall(file) # find any bad characters
newfile = file
for badchar in badchars: # loop through each character in badchars
# note that if badchars is empty, this loop is not entered
# show whats happening
print "replacing",badchar,"in",newfile,":",
# replace all occurrences of this badchar with '-' and remember
# it for next iteration of loop:
newfile = newfile.replace(badchar,'-') #replace bad chars.
print newfile
if badchars: # were there any bad characters in the name?
newpath = os.path.join(root,newfile)
oldpath = os.path.join(root,file)
os.rename(oldpath,newpath)
print oldpath
print newpath
print " "
print "--- Done ---"
print " "
wrt wrapping it up in a function, here's a starter for you... "fill in the
blanks":
-------8<-----------
def cleanup(setpath):
# compile regex for finding bad characters

# walk directory tree...

# find any bad characters

# loop through each character in badchars

# replace all occurrences of this badchar with '-' and remember

# were there any bad characters in the name?

-------8<-----------

To call this you could do (on linux - mac paths are different):

-------8<-----------
cleanup("/home/andy/Python")
-------8<-----------

hope that helps
-andyj
Jul 18 '05 #3
On Friday 18 Jul 2003 11:16 pm, hokiegal99 wrote:
Thanks again for the help Andy! One last question: What is the advantage
of placing code in a function? I don't see how having this bit of code
in a function improves it any. Could someone explain this?

Thanks!

8<--- (old quotes)

The 'benefit' of functions is only really reaped when you have a specificneed
for them! You don't *have* to use them if you don't *need* to (but they can
still improve the readability of your code).

Consider the following contrived example:

----------8<------------
# somewhere in the dark recesses of a large project...
. . .
for filename in os.listdir(cfg.userdir):
newname = filename
for ch in cfg.badchars:
newname.replace(ch,"-")
if newname != filename:
os.rename(os.path.join(cfg.userdir,filename),
os.path.join(cfg.userdir,newname)
. . .
. . .
# in another dark corner...

. . .
for filename in os.listdir(cfg.tempdir):
newname = filename
for ch in cfg.badchars:
newname.replace(ch,"-")
if newname != filename:
os.rename(os.path.join(cfg.userdir,filename),
os.path.join(cfg.userdir,newname)
. . .
# somewhere else...

. . .
for filename in os.listdir(cfg.extradir):
newname = filename
for ch in cfg.badchars:
newname.replace(ch,"-")
if newname != filename:
os.rename(os.path.join(cfg.userdir,filename),
os.path.join(cfg.userdir,newname)
. . .
----------8<------------

See the repetition? ;-)

Imagine a situation where you need to do something far more complicated over,
and over again... It's not very programmer efficient, and it makes the code
longer, too - thus costing more to write (time) and more to store (disks).

Imagine having to change the behaviour of this 'hard-coded' routine, and what
would happen if you missed one... however, if it is in a function, you only
have *one* place to change it.

When we generalise the algorithm and put it into a function we can do:

----------8<------------

. . .
. . .

# somewhere near the top of the project code...
def cleanup_filenames(dir):

""" renames any files within dir that contain bad characters
(ie. ones in cfg.badchars). Does not walk the directory tree.
"""

for filename in os.listdir(dir):
newname = filename
for ch in cfg.badchars:
newname.replace(ch,"-")
if newname != filename:
os.rename(os.path.join(cfg.userdir,filename),
os.path.join(cfg.userdir,newname)

. . .
. . .

# somewhere in the dark recesses of a large project...
. . .
cleanup_filenames(cfg.userdir)
. . .
. . .
# in another dark corner...
. . .
cleanup_filenames(cfg.tempdir)
. . .
# somewhere else...
. . .
cleanup_filenames(cfg.extradir)
. . .

----------8<------------

Even in this small, contrived example, we've saved about 13 lines of code(ok,
that's notwithstanding the blank lines and the """ docstring """ at the top
of the function).

There's another twist, too. In the docstring for cleanup_filenames it says
"Does not walk the directory tree." because we didn't code it to deal with
subdirectories. But we could, without using os.walk...

Directories form a tree structure, and the easiest way to process trees is by
using /recursion/, which means functions that call themselves. An old
programmer's joke is this:

Recursion, defn. [if not understood] see Recursion.

Each time you call a function, it gets a brand new environment, called the
'local scope'. All variables inside this scope are private; they may have
the same names, but they refer to different objects. This can be really
handy...

----------8<------------

def cleanup_filenames(dir):

""" renames any files within dir that contain bad characters
(ie. ones in cfg.badchars). Walks the directory tree to process
subdirectories.
"""

for filename in os.listdir(dir):
newname = filename
for ch in cfg.badchars:
newname.replace(ch,"-")
if newname != filename:
os.rename(os.path.join(cfg.userdir,filename),
os.path.join(cfg.userdir,newname)
# recurse if subdirectory...
if os.path.isdir(os.path.join(cfg.userdir,newname)):
cleanup_filenames(os.path.join(cfg.userdir,newname ))

----------8<------------

This version *DOES* deal with subdirectories... with only two extra lines,
too! Trying to write this without recursion would be a nightmare (even in
Python).

A very important thing to note, however, is that there is a HARD LIMIT onthe
number of times a function can call itself, called the RecursionLimit:

----------8<------------
n=1
def rec(): n=n+1
rec()
rec() . . .
(huge traceback list)
. . .
RuntimeError: maximum recursion limit reached.n

991
----------8<------------

Another very important thing about recursion is that a recursive function
should *ALWAYS* have a 'get-out-clause', a condition that stops the
recursion. Guess what happens if you don't have one ... ;-)

Finally (at least for now), functions also provide a way to break down your
code into logical sections. Many programmers will write the higher level
functions first, delegating 'complicated bits' to further sub-functions as
they go, and worry about implementing them once they've got the overall
algorithm finished. This allows one to concentrate on the right level of
detail, rather than getting bogged down in the finer points: you just make up
names for functions that you're *going* to implement later. Sometimes, you
might make a 'stub' like:

def doofer(dooby, doo):
pass

so that your program is /syntactically/ correct, and will run (to a certain
degree). This allows debugging to proceed before you have written
everything. You'd do this for functions which aren't *essential* to the
program, but maybe add 'special features', for example, additonal
error-checking or output formatting.

A sort of extension of the function idea is 'modules', which make functions
and other objects available to other 'client' programs. When you say:

import os

you are effectively adding all the functions and objects of the os moduleinto
your own program, without having to re-write them. This enables programmers
to share their functions and other code as convenient 'black boxes'. Modules,
however, are a slightly more advanced topic.
Hope that helps.

-andyj

Jul 18 '05 #4
My scripts aren't long and complex, so I don't really *need* to use
functions. But the idea of using them is appealing to me because it
seems the right thing to do from a design point of view. I can see how
larger, more complex programs would get out of hand if the programmer
did not use functions so they'd be absolutely necessary there. But if
they allow larger programs to have a better overall design that's more
compact and readable (like your examples showed) then one could argue
that they would do the same for smaller, simplier programs too.

Thanks for the indepth explanation. It was very helpful. I'm going to
try using functions within my fix_files.py script.
Andy Jewell wrote:
On Friday 18 Jul 2003 11:16 pm, hokiegal99 wrote:
Thanks again for the help Andy! One last question: What is the advantage
of placing code in a function? I don't see how having this bit of code
in a function improves it any. Could someone explain this?

Thanks!


8<--- (old quotes)

The 'benefit' of functions is only really reaped when you have a specific need
for them! You don't *have* to use them if you don't *need* to (but they can
still improve the readability of your code).

Consider the following contrived example:

----------8<------------
# somewhere in the dark recesses of a large project...
. . .
for filename in os.listdir(cfg.userdir):
newname = filename
for ch in cfg.badchars:
newname.replace(ch,"-")
if newname != filename:
os.rename(os.path.join(cfg.userdir,filename),
os.path.join(cfg.userdir,newname)
. . .
. . .
# in another dark corner...

. . .
for filename in os.listdir(cfg.tempdir):
newname = filename
for ch in cfg.badchars:
newname.replace(ch,"-")
if newname != filename:
os.rename(os.path.join(cfg.userdir,filename),
os.path.join(cfg.userdir,newname)
. . .
# somewhere else...

. . .
for filename in os.listdir(cfg.extradir):
newname = filename
for ch in cfg.badchars:
newname.replace(ch,"-")
if newname != filename:
os.rename(os.path.join(cfg.userdir,filename),
os.path.join(cfg.userdir,newname)
. . .
----------8<------------

See the repetition? ;-)

Imagine a situation where you need to do something far more complicated over,
and over again... It's not very programmer efficient, and it makes the code
longer, too - thus costing more to write (time) and more to store (disks).

Imagine having to change the behaviour of this 'hard-coded' routine, and what
would happen if you missed one... however, if it is in a function, you only
have *one* place to change it.

When we generalise the algorithm and put it into a function we can do:

----------8<------------

. . .
. . .

# somewhere near the top of the project code...
def cleanup_filenames(dir):

""" renames any files within dir that contain bad characters
(ie. ones in cfg.badchars). Does not walk the directory tree.
"""

for filename in os.listdir(dir):
newname = filename
for ch in cfg.badchars:
newname.replace(ch,"-")
if newname != filename:
os.rename(os.path.join(cfg.userdir,filename),
os.path.join(cfg.userdir,newname)

. . .
. . .

# somewhere in the dark recesses of a large project...
. . .
cleanup_filenames(cfg.userdir)
. . .
. . .
# in another dark corner...
. . .
cleanup_filenames(cfg.tempdir)
. . .
# somewhere else...
. . .
cleanup_filenames(cfg.extradir)
. . .

----------8<------------

Even in this small, contrived example, we've saved about 13 lines of code (ok,
that's notwithstanding the blank lines and the """ docstring """ at the top
of the function).

There's another twist, too. In the docstring for cleanup_filenames it says
"Does not walk the directory tree." because we didn't code it to deal with
subdirectories. But we could, without using os.walk...

Directories form a tree structure, and the easiest way to process trees is by
using /recursion/, which means functions that call themselves. An old
programmer's joke is this:

Recursion, defn. [if not understood] see Recursion.

Each time you call a function, it gets a brand new environment, called the
'local scope'. All variables inside this scope are private; they may have
the same names, but they refer to different objects. This can be really
handy...

----------8<------------

def cleanup_filenames(dir):

""" renames any files within dir that contain bad characters
(ie. ones in cfg.badchars). Walks the directory tree to process
subdirectories.
"""

for filename in os.listdir(dir):
newname = filename
for ch in cfg.badchars:
newname.replace(ch,"-")
if newname != filename:
os.rename(os.path.join(cfg.userdir,filename),
os.path.join(cfg.userdir,newname)
# recurse if subdirectory...
if os.path.isdir(os.path.join(cfg.userdir,newname)):
cleanup_filenames(os.path.join(cfg.userdir,newname ))

----------8<------------

This version *DOES* deal with subdirectories... with only two extra lines,
too! Trying to write this without recursion would be a nightmare (even in
Python).

A very important thing to note, however, is that there is a HARD LIMIT on the
number of times a function can call itself, called the RecursionLimit:

----------8<------------
n=1
def rec():
n=n+1
rec()

rec()
. . .
(huge traceback list)
. . .
RuntimeError: maximum recursion limit reached.
n

991
----------8<------------

Another very important thing about recursion is that a recursive function
should *ALWAYS* have a 'get-out-clause', a condition that stops the
recursion. Guess what happens if you don't have one ... ;-)

Finally (at least for now), functions also provide a way to break down your
code into logical sections. Many programmers will write the higher level
functions first, delegating 'complicated bits' to further sub-functions as
they go, and worry about implementing them once they've got the overall
algorithm finished. This allows one to concentrate on the right level of
detail, rather than getting bogged down in the finer points: you just make up
names for functions that you're *going* to implement later. Sometimes, you
might make a 'stub' like:

def doofer(dooby, doo):
pass

so that your program is /syntactically/ correct, and will run (to a certain
degree). This allows debugging to proceed before you have written
everything. You'd do this for functions which aren't *essential* to the
program, but maybe add 'special features', for example, additonal
error-checking or output formatting.

A sort of extension of the function idea is 'modules', which make functions
and other objects available to other 'client' programs. When you say:

import os

you are effectively adding all the functions and objects of the os module into
your own program, without having to re-write them. This enables programmers
to share their functions and other code as convenient 'black boxes'. Modules,
however, are a slightly more advanced topic.
Hope that helps.

-andyj

Jul 18 '05 #5
hokiegal99 wrote:

My scripts aren't long and complex, so I don't really *need* to use
functions. But the idea of using them is appealing to me because it
seems the right thing to do from a design point of view. I can see how
larger, more complex programs would get out of hand if the programmer
did not use functions so they'd be absolutely necessary there. But if
they allow larger programs to have a better overall design that's more
compact and readable (like your examples showed) then one could argue
that they would do the same for smaller, simplier programs too.


Uh oh! You're being poisoned with the meme of "right design".

Don't use functions because they "seem the right thing to do", use
them because *they reduce repetition*, or simplify code by *making
it more readable*.

If you aren't reducing repetition with them you are losing the
primary benefit. If you also aren't making your code more readable,
don't use functions.

Whatever you do, don't use them just because somebody has convinced
you "they're the right thing".

-Peter
Jul 18 '05 #6

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

0
by: Saradhi | last post by:
Hi All, Here I am facing a performance problem with the TreeView Node renaming. I am displaying a hierarchy Data in a treeview in my Windows C# Application. My tree view represents an...
7
by: Lalasa | last post by:
Hi, Can anybody tell me how many cpu cycles File.copy would take and how many cpu cycles File.Move would take? CFile::Rename in C++ takes just one cpu cycle. As there is no File.Rename in C#,...
1
by: MikeY | last post by:
Hopefully someone can help, I have a listview box where I display my desired files. I single click on the desired file to be renamed and I rename it with a new name. My problem arises when the...
6
by: Divya | last post by:
Hi, I have a web page which generates a CSV file based on some user input. When this file is downloaded by the user, the file is being automatically converted to .xls. Any idea how I can prevent...
1
by: oldgent | last post by:
I am having a problem installing the starter kits. I have reinstalled VS 2005, think that might be the problem. I then installed both 'Personal Website" and the "Club Website" starter kits. I...
30
by: Pep | last post by:
Is it best to include the code "using namespace std;" in the source or should each keyword in the std namespace be qualified by the namespace tag, such as std::cout << "using std namespace" <<...
4
by: samjnaa | last post by:
Please check for sanity and approve for posting at python-dev. Currently file-directory-related functionality in the Python standard library is scattered among various modules such as shutil, os,...
18
by: Angus | last post by:
Hello We have a lot of C++ code. And we need to now create a library which can be used from C and C++. Given that we have a lot of C++ code using classes how can we 'hide' the fact that it is...
36
by: Don | last post by:
I wrote an app that alerts a user who attempts to open a file that the file is currently in use. It works fine except when the file is opened by Notepad. If a text file is opened, most computers...
0
by: DolphinDB | last post by:
Tired of spending countless mintues downsampling your data? Look no further! In this article, you’ll learn how to efficiently downsample 6.48 billion high-frequency records to 61 million...
1
isladogs
by: isladogs | last post by:
The next Access Europe meeting will be on Wednesday 6 Mar 2024 starting at 18:00 UK time (6PM UTC) and finishing at about 19:15 (7.15PM). In this month's session, we are pleased to welcome back...
0
by: Vimpel783 | last post by:
Hello! Guys, I found this code on the Internet, but I need to modify it a little. It works well, the problem is this: Data is sent from only one cell, in this case B5, but it is necessary that data...
0
by: ArrayDB | last post by:
The error message I've encountered is; ERROR:root:Error generating model response: exception: access violation writing 0x0000000000005140, which seems to be indicative of an access violation...
1
by: CloudSolutions | last post by:
Introduction: For many beginners and individual users, requiring a credit card and email registration may pose a barrier when starting to use cloud servers. However, some cloud server providers now...
1
by: Defcon1945 | last post by:
I'm trying to learn Python using Pycharm but import shutil doesn't work
0
by: af34tf | last post by:
Hi Guys, I have a domain whose name is BytesLimited.com, and I want to sell it. Does anyone know about platforms that allow me to list my domain in auction for free. Thank you
0
by: Faith0G | last post by:
I am starting a new it consulting business and it's been a while since I setup a new website. Is wordpress still the best web based software for hosting a 5 page website? The webpages will be...
0
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 3 Apr 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome former...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.