473,549 Members | 2,777 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

[perl-python] find & replace strings for all files in a dir

suppose you want to do find & replace of string of all files in a
directory.
here's the code:

©# -*- coding: utf-8 -*-
©# Python
©
©import os,sys
©
©mydir= '/Users/t/web'
©
©findStr='<!DOC TYPE HTML PUBLIC "-//W3C//DTD HTML 3.2 FINAL//EN">'
©repStr='<!DOCT YPE HTML PUBLIC "-//W3C//DTD HTML 4.01
Transitional//EN">'
©
©def replaceStringIn File(findStr,re pStr,filePath):
© "replaces all findStr by repStr in file filePath"
© tempName=filePa th+'~~~'
© input = open(filePath)
© output = open(tempName,' w')
©
© for s in input:
© output.write(s. replace(findStr ,repStr))
© output.close()
© input.close()
© os.rename(tempN ame,filePath)
© print filePath
©
©def myfun(dummy, dirr, filess):
© for child in filess:
© if '.html' == os.path.splitex t(child)[1] and
©os.path.isfile (dirr+'/'+child):
© replaceStringIn File(findStr,re pStr,dirr+'/'+child)
©os.path.walk(m ydir, myfun, 3)
note that files will be overwritten.
be sure to backup the folder before you run it.

try to edit the code to suite your needs.

previous tips can be found at:
http://xahlee.org/perl-python/python.html

---------------------------------------
the following is a Perl version i wrote few years ago.
Note: if regex is turned on, correctness is not guranteed.
it is very difficult if not impossible in Perl to move regex pattern
around and preserve their meanings.
#!/usr/local/bin/perl

=pod

Description:
This script does find and replace on a given foler recursively.

Features:
* multiple Find and Replace string pairs can be given.
* The find/replace strings can be set to regex or literal.
* Files can be filtered according to file name suffix matching or other

criterions.
* Backup copies of original files will be made at a user specified
folder that preserves all folder structures of original folder.
* A report will be generated that indicates which files has been
changed, how many changes, and total number of files changed.
* files will retain their own/group/permissions settings.

usage:
1. edit the parts under the section '#-- arguments --'.
2. edit the subroutine fileFilterQ to set which file will be checked or

skipped.

to do:
* in the report, print the strings that are changed, possibly with
surrounding lines.
* allow just find without replace.
* add the GNU syntax for unix command prompt.
* Report if backup directory exists already, or provide toggle to
overwrite, or some other smarties.

Date created: 2000/02
Author: Xah

=cut

#-- modules --

use strict;
use File::Find;
use File::Path;
use File::Copy;
use Data::Dumper;

#-- arguments --

# the folder to be search on.
my $folderPath = q[/Users/t/web/UnixResource_di r];

# this is the backup folder path.
my $backupFolderPa th = q[/Users/t/xxxb];

my %findReplaceH = (
q[<pre><a href="freebooks .html">back to Unix
Pestilence</a><pre>]=>q[<pre>? Back to <a href="freebooks .html">Unix
Pestilence</a></pre>],
);

# $useRegexQ has values 1 or 0. If 1, inteprets the pairs in
%findReplaceH
# to be regex.
my $useRegexQ = 0;

# in bytes. larger files will be skipped
my $fileSizeLimit = 500 * 1000;
#-- globals --

$folderPath =~ s[/$][]; # e.g. '/home/joe/public_html'
$backupFolderPa th =~ s[/$][]; # e.g. '/tmp/joe_back';

$folderPath =~ m[/(\w+)$];
my $previousDir = $`; # e.g. '/home/joe'
my $lastDir = $1; # e.g. 'public_html'
my $backupRoot = $backupFolderPa th . '/' . $1; # e.g.
'/tmp/joe_back/public_html'

my $refLargeFiles = [];
my $totalFileChang edCount = 0;

#-- subroutines --

# fileFilterQ($fu llFilePath) return true if file is desired.
sub fileFilterQ ($) {
my $fileName = $_[0];

if ((-s $fileName) > $fileSizeLimit) {
push (@$refLargeFile s, $fileName);
return 0;
};
if ($fileName =~ m{\.html$}) {
print "processing : $fileName\n";
return 1;};

## if (-d $fileName) {return 0;}; # directory
## if (not (-T $fileName)) {return 0;}; # not text file

return 0;
};

# go through each file, accumulate a hash.
sub processFile {
my $currentFile = $File::Find::na me; # full path spect
my $currentDir = $File::Find::di r;
my $currentFileNam e = $_;

if (not fileFilterQ($cu rrentFile)) {
return 1;
}

# open file. Read in the whole file.
if (not(open FILE, "<$currentFile" )) {die("Error opening file:
$!");};
my $wholeFileStrin g;
{local $/ = undef; $wholeFileStrin g = <FILE>;};
if (not(close(FILE ))) {die("Error closing file: $!");};

# do the replacement.
my $replaceCount = 0;

foreach my $key1 (keys %findReplaceH) {
my $pattern = ($useRegexQ ? $key1 : quotemeta($key1 ));
$replaceCount = $replaceCount + ($wholeFileStri ng =~
s/$pattern/$findReplaceH{$ key1}/g);
};

if ($replaceCount > 0) { # replacement has happened
$totalFileChang edCount++;
# do backup
# make a directory in the backup path, make a backup
copy.
my $pathAdd = $currentDir; $pathAdd =~
s[$folderPath][];
mkpath("$backup Root/$pathAdd", 0, 0777);
copy($currentFi le,
"$backupRoo t/$pathAdd/$currentFileNam e") or
die "error: file copying file failed on
$currentFile\n$ !";

# write to the original
# get the file mode.
my ($mode, $uid, $gid) = (stat($currentF ile))[2,4,5];

# write out a new file.
if (not(open OUTFILE, ">$currentFile" )) {die("Error
opening file: $!");};
print OUTFILE $wholeFileStrin g;
if (not(close(OUTF ILE))) {die("Error closing file:
$!");};

# set the file mode.
chmod($mode, $currentFile);
chown($uid, $gid, $currentFile);

print "-----^$*%$@#-------------------------------\n";
print "$replaceCo unt replacements made at\n";
print "$currentFile\n ";
}

};
#-- main body --

find(\&processF ile, $folderPath);

print "--------------------------------------------\n\n\n";
print "Total of $totalFileChang edCount files changed.\n";

if (scalar @$refLargeFiles > 0) {
print "The following large files are skipped:\n";
print Dumper($refLarg eFiles);
}
__END__
Xah
xa*@xahlee.org
http://xahlee.org/PageTwo_dir/more.html

Jul 18 '05 #1
1 3711
Xah Lee wrote:
suppose you want to do find & replace of string of all files in a
directory.
here's the code: [snip] Xah
xa*@xahlee.org
http://xahlee.org/PageTwo_dir/more.html


When are you going to take the hint (from everybody in
comp.lang.perl. misc and comp.lang.pytho n) to stop posting! Your posts
do not help anybody and will only hurt a beginner. *PLEASE STOP POSTING*!

--
k g a b e r t (@at@) x m i s s i o n (.dot.) c o m

* After "extensive" research, I noticed
* that yy******@yahoo. com received 12
* spam e-mail messages after just two
* posts on usenet groups. If you want
* to email me, use the "encrypted"
* email address at the beggining of my
* signature.
Jul 18 '05 #2

This thread has been closed and replies have been disabled. Please start a new discussion.

Similar topics

3
3443
by: David F. Skoll | last post by:
Hi, I'm tearing my hair out on this one. I'm trying to embed a Perl interpreter into a C program. I need to be able to create and destroy the interpreter periodically, but will never actually have two interpreters at the same time. On Red Hat Linux 7.3 with Perl 5.6.1, the attached program segfaults. On Red Hat 9 with Perl 5.8.0, it...
1
4669
by: Julia Bell | last post by:
I would like to run the same script on two different platforms. The directory in which the script(s) will be stored is common to the two platforms. (I see the same directory contents regardless of which platform I use to access the directory.) Platform 1: perl is installed in /tps/bin/perl. CPAN modules are available Perl is also...
1
668
by: sm00thcrimnl13 | last post by:
if i have windows 2000 and know how to write perl scripts, how to i actuvate the script through perl?
1
3615
by: smsabu2002 | last post by:
Hi, I am facing the build problem while installing the DBD-MySql perl module (ver 2.9008) using both GCC and CC compilers in HP-UX machine. For the Build using GCC, the compiler error is produced due to the unknown GCC compiler option "+DAportable". For the Build using CC, the preprocessor error is produced due to the recursive...
13
3237
by: Otto J. Makela | last post by:
I'm trying to install to php the Perl-1.0.0.tgz package (from http://pecl.php.net/package/perl, enabling one to call perl libraries) to a pre-existing Solaris system. Unfortunately, the attempt fails in a rather dramatic way, spewing out thousands of "relocation remains"... I'm somewhat lost on what to do next, the documentation that came...
6
2988
by: surfivor | last post by:
I may be involved in a data migration project involving databases and creating XML feeds. Our site is PHP based, so I imagine the team might suggest PHP, but I had a look at the PHP documentation for one of the Pear modules for creating XML and it didn't look like much. I've used Perl XML:Twig and I have the impression that there is more Perl...
4
3682
by: billb | last post by:
I installed a perl extension for PHP to use some perl inside my php primarily because I have perl working with oracle and not php and oracle. So I want to use my old perl scripts, and use the functionality of php. The extension uses the '$perl->eval' function. i am kind of stuck with the syntax or how to put the php variable into the perl...
223
7126
by: Pilcrow | last post by:
Given that UNIX, including networking, is almost entirely coded in C, how come so many things are almost impossible in ordinary C? Examples: Network and internet access, access to UNIX interprocess controls and communication, locale determination, EBCDIC/ASCII discrimination, etc. Almost all of these are easy in Perl. Why isn't there a...
4
8575
by: vijayarl | last post by:
Hi All, i have the following software installed in my system : 1.OS: Win2k 2.Eclipse Version used :3.4.0 & even the perl too... 1. I have imported the my own perl project in Eclipse, when i tried to run the External Tools --> Perl -w am getting the popup saying then it says : " Variable references empty selection:
0
7477
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language...
0
7750
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. ...
0
7991
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that...
1
7509
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For...
0
7838
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the...
0
6084
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing, and deployment—without human intervention. Imagine an AI that can take a project description, break it down, write the code, debug it, and then...
1
5396
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes...
0
5118
by: conductexam | last post by:
I have .net C# application in which I am extracting data from word file and save it in database particularly. To store word all data as it is I am converting the whole word file firstly in HTML and then checking html paragraph one by one. At the time of converting from word file to html my equations which are in the word document file was convert...
0
3503
by: adsilva | last post by:
A Windows Forms form does not have the event Unload, like VB6. What one acts like?

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.