473,847 Members | 1,525 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

recursive file editing

I'm a python newbie; here are a few questions relative to a
problem I'm trying to solve; I'm wandering if python is the best
instrument or if awk or a mix of bash and sed would be better:

1) how would I get recursively descend
through all files in all subdirectories of
the one in which my script is called ?

2) each file examined should be edited; IF a string of this type is found

foo.asp=dev?bar (where bar can be a digit or an empty space)

it should always be substituted with this string

foo-bar.html (if bar is an empty space the new string is foo-.html)

3) the names of files read may themselves be of the sort foo.asp=dev?bar ;
the edited output file should also be renamed according to the same rule
as above ... or would this be better handled by a bash script ?

Any hints appreciated

--
Michele Alzetta

Jul 18 '05 #1
15 2730
This is a Perl one-liner:

perl -p -i -e 's/foo/bar/gi' `find ./`

regards,
--
Leif Biberg Kristensen
http://solumslekt.org/
Validare necesse est
Jul 18 '05 #2
> I'm a python newbie; here are a few questions relative to a
problem I'm trying to solve; I'm wandering if python is the best
instrument or if awk or a mix of bash and sed would be better:

1) how would I get recursively descend
through all files in all subdirectories of
the one in which my script is called ?
Check out os.path.walk.

2) each file examined should be edited; IF a string of this type is found

foo.asp=dev?bar (where bar can be a digit or an empty space)

it should always be substituted with this string

foo-bar.html (if bar is an empty space the new string is foo-.html)
Check out the re module.

3) the names of files read may themselves be of the sort foo.asp=dev?bar ;
the edited output file should also be renamed according to the same rule
as above ... or would this be better handled by a bash script ?


Do it however you feel more comfortable, Python can do it.
- Josiah
Jul 18 '05 #3
Il Sat, 03 Apr 2004 22:35:30 +0200, Leif B. Kristensen ha scritto:
This is a Perl one-liner:

perl -p -i -e 's/foo/bar/gi' `find ./`


Didn't work; however I realized I could just repeatedly run variations of

sed -i s/foo/bar/g *

and then rename the files with a bash script called renna.

I'm sure python could do it but I got stuck -

for file in os.walk('mydir' ):
print file[2]

gives me the names of all files

but how do I open each with r+ mode ?

for thing in os.walk('mydir' ):
file(thing,mode =r+)

is invalid syntax

--
Michele Alzetta

Jul 18 '05 #4
> for thing in os.walk('mydir' ):
file(thing,mode =r+)


for thing in os.walk('mydir' ):
filehandle = file(thing, 'r+')
- Josiah
Jul 18 '05 #5
TaeKyon wrote:
I'm a python newbie; here are a few questions relative to a
problem I'm trying to solve; I'm wandering if python is the best
instrument or if awk or a mix of bash and sed would be better:

1) how would I get recursively descend
through all files in all subdirectories of
the one in which my script is called ?

2) each file examined should be edited; IF a string of this type is found

foo.asp=dev?bar (where bar can be a digit or an empty space)

it should always be substituted with this string

foo-bar.html (if bar is an empty space the new string is foo-.html)

3) the names of files read may themselves be of the sort foo.asp=dev?bar ;
the edited output file should also be renamed according to the same rule
as above ... or would this be better handled by a bash script ?

Any hints appreciated


The following code comes with no warranties. Be sure to backup valuable data
before trying it. You may need to edit the regular expressions. Call the
script with the directory you want to process.

Peter

import os, re, sys

class Path(object):
def __init__(self, folder, name):
self.folder = folder
self.name = name

def _get_path(self) :
return os.path.join(se lf.folder, self.name)
path = property(_get_p ath)

def rename(self, newname):
if self.name != newname:
os.rename(self. path, os.path.join(se lf.folder, newname))
self.name = newname

def processContents (self, operation):
data = file(self.path) .read()
newdata = operation(data)
if data != newdata:
file(self.path, "w").write(newd ata)

def __str__(self):
return self.path

def files(rootfolde r):
for folder, folders, files in os.walk(rootfol der):
for name in files:
yield Path(folder, name)

fileExpr = re.compile(r"^( .+?)\.asp\=dev\ ?(.*)$")
filePattern = r"\1-\2.html"

textExpr = re.compile(r"([/\'\"])(.+?)\.asp\=de v\?(.*?)([\'\"])")
textPattern = r"\1\2-\3.html\4"

if __name__ == "__main__":
for f in files(sys.argv[1]):
f.rename(fileEx pr.sub(filePatt ern, f.name))
f.processConten ts(lambda s: textExpr.sub(te xtPattern, s))

Jul 18 '05 #6
Il Sat, 03 Apr 2004 17:22:04 -0800, Josiah Carlson ha scritto:
for thing in os.walk('mydir' ):
filehandle = file(thing, 'r+')


I'm such a newbe I can't get it to work. Here is an example:

in empty directory foo I touch a b c d;
suppose I want to write "This works !" in each of these files.

I run python
import os
for thing in os.walk('foo'): .... thingopen = file(thing,'r+' )
.... thingopen.write ("This works !")
.... thingopen.close ()
....
Traceback (most recent call last):
File "<stdin>", line 2, in ?
TypeError: coercing to Unicode: need string or buffer, tuple found

And in fact:
for thing in os.walk('foo'):

.... print thing
....
('foo', [], ['a', 'b', 'c', 'd'])

which is a tuple, I suppose.

Selecting thing[2] doesn't help, because it now complains of it being a
list.

In the end I get this to work:

for filetuple in os.walk('foo'):
.... for filename in filetuple[2]:
.... fileopen = file(filename, 'r+')
fileopen.write( "This works !")
fileopen.close( )

which seems a bit of a clumsy way to do it.
And besides it only works if I run python from directory foo,
otherwise it tells me "no such file or directory".

--
Michele Alzetta

Jul 18 '05 #7
TaeKyon wrote:
Il Sat, 03 Apr 2004 17:22:04 -0800, Josiah Carlson ha scritto:
for thing in os.walk('mydir' ):
filehandle = file(thing, 'r+')


I'm such a newbe I can't get it to work. Here is an example:

in empty directory foo I touch a b c d;
suppose I want to write "This works !" in each of these files.

I run python
import os
for thing in os.walk('foo'): ... thingopen = file(thing,'r+' )
... thingopen.write ("This works !")
... thingopen.close ()
...
Traceback (most recent call last):
File "<stdin>", line 2, in ?
TypeError: coercing to Unicode: need string or buffer, tuple found

And in fact:
for thing in os.walk('foo'):

... print thing
...
('foo', [], ['a', 'b', 'c', 'd'])

which is a tuple, I suppose.

Selecting thing[2] doesn't help, because it now complains of it being a
list.

In the end I get this to work:

for filetuple in os.walk('foo'):
... for filename in filetuple[2]:
... fileopen = file(filename, 'r+')
fileopen.write( "This works !")
fileopen.close( )

which seems a bit of a clumsy way to do it.
And besides it only works if I run python from directory foo,
otherwise it tells me "no such file or directory".


A minimal working example is:

import os
for path, folders, files in os.walk("/path/to/folder"):
for name in files:
filepath = os.path.join(pa th, name)
fileopen = file(filepath, 'r+')
fileopen.write( "This works !")
fileopen.close( )

You need to compose the filepath, and, yes, it's a bit clumsy.
I've written a little generator function to hide some of the clumsiness:

def files(folder):
for path, folders, files in os.walk(folder) :
for name in files:
yield os.path.join(pa th, name)

With that the code is simplified to:

for filepath in files("/path/to/folder"):
fileopen = file(filepath, 'r+')
fileopen.write( "This works !")
fileopen.close( )

HTH,
Peter
Jul 18 '05 #8
> for filetuple in os.walk('foo'):
... for filename in filetuple[2]:
... fileopen = file(filename, 'r+')
fileopen.write( "This works !")
fileopen.close( )

which seems a bit of a clumsy way to do it.
You have the right idea, although this would be a bit cleaner:

for root, dirs, files in os.walk( 'foo' ):
for name in files:
etc...

You might want to take a look at the documentation for os.walk. It explains
all this and has a couple of good code samples.
And besides it only works if I run python from directory foo,
otherwise it tells me "no such file or directory".


You mean if you run Python from the *parent* directory of foo, right?

'foo' is a relative path, not an absolute one, so it gets appended to the
current directory.

-Mike
Jul 18 '05 #9
Il Mon, 05 Apr 2004 19:15:01 +0200, Peter Otten ha scritto:
You need to compose the filepath, and, yes, it's a bit clumsy.
I've written a little generator function to hide some of the clumsiness:

def files(folder):
for path, folders, files in os.walk(folder) :
for name in files:
yield os.path.join(pa th, name)

With that the code is simplified to:

for filepath in files("/path/to/folder"):
fileopen = file(filepath, 'r+')
fileopen.write( "This works !")
fileopen.close( )


Great !

--
Michele Alzetta

Jul 18 '05 #10

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

5
2026
by: Magnus Lyck? | last post by:
Something really strange is happening to me (sometimes). I'm using Python 2.3.2 on NT 4.0 as well as win32all-157, adodbapi and db_row. During a recursive call to a method, it seems Python messes up its variable bindings once in a while. Suddenly, one of several local variables gets rebound to the object it was bound to one step up in the recursion. The relevant part of the code looks like this with print statements
1
7112
by: gusmeister | last post by:
Is there a Perl mod that has a recursive file copy function (similar to `cp -r` in Unix or `xcopy` in Windows)? File::Path does have a recursive file deletion function (rmtree) but no recursive copy function (e.g. cptree).
2
3435
by: Oxmard | last post by:
Armed with my new O'Reilly book Optimizing Oracle Performance I have been trying to get a better understanding of how Oracle works. The book makes the statement, " A database cal with dep=n + 1 is the recursive child of the first subsequent dep=n database call listed in the SQL data stream. The book gives a few examples, and in trying it out it seemed to work until I tried the following SQL. My question are why does this not keep with...
5
20109
by: betterdie | last post by:
Dear guru I want to delete all file and folder recursivly under php code, can anyone give me commend for this. Thank very much
4
3717
by: Elmo Watson | last post by:
Is there a way, with the System.IO class, to do a recursive list of a directory structure? For instance, in DirectoryInfo, you have GetDirectories and GetFiles .... In Directory, you have Directory.GetFileSystemEntries(path), but I would like to know how to put this together, knowing which entry is a Subdirectory and which entry is a file, and make a recursive list of the Directory structure below a specific path - - -
16
1861
by: Indy | last post by:
Hi, I have a XHTML input file with custom tag which specifies html fragments to include For example: <html> .... <include frag1="frag1.html" frag2="frag2.html"> More html here </include> ....html...
3
1335
by: mizrandir | last post by:
Can someone tell me why python doesn't crash when I do the following: ] ] How does python handle this internally? Will it crash or use up lot's of memory in similar but more complicated cases?
10
2574
by: AsheeG87 | last post by:
Hello Everyone! I have a linked list and am trying to include a recursive search. However, I am having trouble understanding how I would go about that. I don't quite understand a recursive search....would any of you be so kind to explain it to me...maybe include some examples. I would GREATLY appreciate it!!!
0
9735
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10996
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10658
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
0
9497
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then launch it, all on its own.... Now, this would greatly impact the work of software developers. The idea...
1
7889
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
7062
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert into image. Globals.ThisAddIn.Application.ActiveDocument.Select();...
0
5730
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
4541
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system
3
3171
bsmnconsultancy
by: bsmnconsultancy | last post by:
In today's digital era, a well-designed website is crucial for businesses looking to succeed. Whether you're a small business owner or a large corporation in Toronto, having a strong online presence can significantly impact your brand's success. BSMN Consultancy, a leader in Website Development in Toronto offers valuable insights into creating effective websites that not only look great but also perform exceptionally well. In this comprehensive...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.