Connecting Tech Pros Worldwide Help | Site Map

grep text file line by line and grep a directory for a match of each line

Newbie
 
Join Date: May 2008
Posts: 16
#1: Apr 3 '09
Hi everyone,

This is my first time posting to the UNIX form and it might be a strange request, but I believe I have most of the pieces.

Here's the overall goal -- I am trying to find the links in a large web site that are linked to files over 2000k.

I used the command line to find all files on the server that are larger than 2000k by using the following:

Expand|Select|Wrap|Line Numbers
  1. find ./ -size +2000c > files_over_2000_bytes.txt
  2.  
And I know how to grep all the files in the web site to find a certain word by using the following:

Expand|Select|Wrap|Line Numbers
  1. grep -nr Something * > Something.txt;
  2.  
Since I haven't used awk except for way back in '98, I am at a loss for how I could place these two commands together.

In any type of language that I am familiar with, I would make a function out of the first line.

Then I would create another function to read each line within the text file and place it into a variable.

Then I would grep the web site using the variable.

So, I have the logic, but I do not know the syntax.

Can anyone help me?

Thanx n advance
gpraghuram's Avatar
Expert
 
Join Date: Mar 2007
Location: Chennai
Posts: 1,256
#2: Apr 3 '09

re: grep text file line by line and grep a directory for a match of each line


You want to embedd the whole logic in a AWK program? or whats ur requirement?

raghu
Newbie
 
Join Date: May 2008
Posts: 16
#3: Apr 3 '09

re: grep text file line by line and grep a directory for a match of each line


This is what I came up with:

Expand|Select|Wrap|Line Numbers
  1. #!/bin/bash
  2.  
  3. while read LINE
  4.  
  5. do
  6.  
  7. grep –nr $LINE * > matched_line.txt
  8.  
  9. done < files_over_2000_bytes.txt
  10.  
Expert
 
Join Date: Apr 2006
Posts: 512
#4: Apr 4 '09

re: grep text file line by line and grep a directory for a match of each line


use the -f option of grep, eg
Expand|Select|Wrap|Line Numbers
  1. grep -f file1 file2
  2.  
Expert
 
Join Date: Apr 2006
Posts: 512
#5: Apr 4 '09

re: grep text file line by line and grep a directory for a match of each line


use the -f option of grep, eg
Expand|Select|Wrap|Line Numbers
  1. grep -f file1 file2
  2.  
Newbie
 
Join Date: May 2008
Posts: 16
#6: Apr 6 '09

re: grep text file line by line and grep a directory for a match of each line


I don't know if I understand your reply, because I am trying to read each individual line of file1 to grep the entire web site.

So are you saying that I should write:
Expand|Select|Wrap|Line Numbers
  1. #!/bin/bash
  2.  
  3. grep –f files_over_2000_bytes.txt * > matched_line.txt
  4.  
  5.  
Newbie
 
Join Date: May 2008
Posts: 16
#7: Apr 6 '09

re: grep text file line by line and grep a directory for a match of each line


I have started looking at my own post and I started to think that maybe this wasn't making much sense to anyone other than myself.

Say I have a file with the following:
Expand|Select|Wrap|Line Numbers
  1. /var/www/vhosts/something.com/images/1.jpg
  2. /var/www/vhosts/something.com/images/2.jpg
  3. /var/www/vhosts/something.com/images/3.jpg
  4. /var/www/vhosts/something.com/images/4.jpg
  5. /var/www/vhosts/something.com/images/4.jpg
  6.  
Let's call this file "files_over_2000k.txt"

What I would like to do is grep the entire web site using each line from "files_over_2000k.txt" to find any links within any page for "1.jpg," "2.jpg," etc... and place the results of where they are located, within another text file called "matched_line.txt"
Reply


Similar Unix / Linux / BSD bytes