Match 2 words in a line of file

[elrondrules]

Hi

Am pretty new to python and hence this question..

I have file with an output of a process. I need to search this file one
line at a time and my pattern is that I am looking for the lines that
has the word 'event' and the word 'new'.

Note that i need lines that has both the words only and not either one
of them..

how do i write a regexp for this.. or better yet shd i even be using
regexp or is there a better way to do this....

thanks

[Rickard Lindberg]

el*********@gma il.com wrote:

Hi

Am pretty new to python and hence this question..

I have file with an output of a process. I need to search this file one
line at a time and my pattern is that I am looking for the lines that
has the word 'event' and the word 'new'.

Note that i need lines that has both the words only and not either one
of them..

how do i write a regexp for this.. or better yet shd i even be using
regexp or is there a better way to do this....

thanks

Maybe something like this would do:

import re

def lines_with_word s(file, word1, word2):
"""Print all lines in file that have both words in it."""
for line in file:
if re.search(r"\b" + word1 + r"\b", line) and \
re.search(r"\b" + word2 + r"\b", line):
print line

Just call the function with a file object and two strings that
represent the words that you want to find in each line.

To match a word in regex you write "\bWORD\b".

I don't know if there is a better way of doing this, but I believe that
this should at least work.

[MrJean1]

Without using re, this may work (untested ;-):

def lines_with_word s(file, word1, word2):
"""Print all lines in file that have both words in it."""
for line in file:
words = line.split()
if word1 in words and word2 in words:
print line
/Jean Brouwers
Rickard Lindberg wrote:

el*********@gma il.com wrote:

Hi

Am pretty new to python and hence this question..

I have file with an output of a process. I need to search this file one
line at a time and my pattern is that I am looking for the lines that
has the word 'event' and the word 'new'.

Note that i need lines that has both the words only and not either one
of them..

how do i write a regexp for this.. or better yet shd i even be using
regexp or is there a better way to do this....

thanks

Maybe something like this would do:

import re

def lines_with_word s(file, word1, word2):
"""Print all lines in file that have both words in it."""
for line in file:
if re.search(r"\b" + word1 + r"\b", line) and \
re.search(r"\b" + word2 + r"\b", line):
print line

Just call the function with a file object and two strings that
represent the words that you want to find in each line.

To match a word in regex you write "\bWORD\b".

I don't know if there is a better way of doing this, but I believe that
this should at least work.

[bearophileHUGS]

MrJean1 wrote:

def lines_with_word s(file, word1, word2):
"""Print all lines in file that have both words in it."""
for line in file:
words = line.split()
if word1 in words and word2 in words:
print line

This sounds better, it's probably faster than the RE version, Python
2.5 has a really fast str.__contains_ _ method, done by effbot:

def lines_with_word s(file, word1, word2):
"""Print all lines in file that have both words in it.
(word1 may be the less frequent word of the two)."""
for line in file:
if word1 in line and word2 in line:
print line

Bye,
bearophile

[Rickard Lindberg]

I see two potential problems with the non regex solutions.

1) Consider a line: "foo (bar)". When you split it you will only get
two strings, as split by default only splits the string on white space
characters. Thus "'bar' in words" will return false, even though bar is
a word in that line.

2) If you have a line something like this: "foobar hello" then "'foo'
in line" will return true, even though foo is not a word (it is part of
a word).

[harvey.thomas]

Rickard Lindberg wrote:

I see two potential problems with the non regex solutions.

1) Consider a line: "foo (bar)". When you split it you will only get
two strings, as split by default only splits the string on white space
characters. Thus "'bar' in words" will return false, even though bar is
a word in that line.

2) If you have a line something like this: "foobar hello" then "'foo'
in line" will return true, even though foo is not a word (it is part of
a word).

Here's a solution using re.split:

import re
import StringIO

wordsplit = re.compile('\W+ ').split
def matchlines(fh, w1, w2):
w1 = w1.lower()
w2 = w2.lower()
for line in fh:
words = [x.lower() for x in wordsplit(line)]
if w1 in words and w2 in words:
print line.rstrip()

test = """1st line of text (not matched)
2nd line of words (not matched)
3rd line (Word test) should match (case insensitivity)
4th line simple test of word's (matches)
5th line simple test of words not found (plural words)
6th line tests produce strange words (no match - plural)
7th line "word test" should find this
"""
matchlines(Stri ngIO.StringIO(t est), 'test', 'word')

[harvey.thomas]

Rickard Lindberg wrote:

I see two potential problems with the non regex solutions.

1) Consider a line: "foo (bar)". When you split it you will only get
two strings, as split by default only splits the string on white space
characters. Thus "'bar' in words" will return false, even though bar is
a word in that line.

2) If you have a line something like this: "foobar hello" then "'foo'
in line" will return true, even though foo is not a word (it is part of
a word).

Here's a solution using re.split:

import re
import StringIO

wordsplit = re.compile('\W+ ').split
def matchlines(fh, w1, w2):
w1 = w1.lower()
w2 = w2.lower()
for line in fh:
words = [x.lower() for x in wordsplit(line)]
if w1 in words and w2 in words:
print line.rstrip()

test = """1st line of text (not matched)
2nd line of words (not matched)
3rd line (Word test) should match (case insensitivity)
4th line simple test of word's (matches)
5th line simple test of words not found (plural words)
6th line tests produce strange words (no match - plural)
7th line "word test" should find this
"""
matchlines(Stri ngIO.StringIO(t est), 'test', 'word')

[bearophileHUGS]

Rickard Lindberg, yesterday I was sleepy and my solution was wrong.

2) If you have a line something like this: "foobar hello" then "'foo'
in line" will return true, even though foo is not a word (it is part of
a word).

Right. Now I think the best solution is to use __contains__ (in) to
quickly find the lines that surely contains both substrings, then on
such possibly rare cases you can use a correctly done RE. If the words
are uncommon enough, such solution may be fast and reliable.
Using raw tests followed by slow and reliable ones on the rare positive
results of the first test is a solution commonly used in Computer
Science, that often is both fast and reliable. (It breaks when the
first test is passed too much often, or when it has some false
negatives).

Probably there are even faster solutions, scanning the whole text at
once instead of inside its lines, but the code becomes too much hairy
and probably it's not worth it.

Bye,
bearophile

[Daniel Klein]

On 18 Jan 2007 18:54:59 -0800, "Rickard Lindberg"
<ri******@stude nt.liu.sewrote:

>I see two potential problems with the non regex solutions.

1) Consider a line: "foo (bar)". When you split it you will only get
two strings, as split by default only splits the string on white space
characters. Thus "'bar' in words" will return false, even though bar is
a word in that line.

2) If you have a line something like this: "foobar hello" then "'foo'
in line" will return true, even though foo is not a word (it is part of
a word).

1) Depends how you define a 'word'.

2) This can be resolved with

templine = ' ' + line + ' '
if ' ' + word1 + ' ' in templine and ' ' + word2 + ' ' in templine:
Dan

[Rickard Lindberg]

Daniel Klein wrote:

2) This can be resolved with

templine = ' ' + line + ' '
if ' ' + word1 + ' ' in templine and ' ' + word2 + ' ' in templine:

But then you will still have a problem to match the word "foo" in a
string like "bar (foo)".