Percentage matching of text

[Bruce Eckel]

Background: for the 4th edition of Thinking in Java, I'm trying to
once again improve the testing scheme for the examples in the book. I
want to verify that the output I show in the book is "reasonably
correct." I say "Reasonably" because a number of examples produce
random numbers or text or the time of day or in general things that do
not repeat themselves from one execution to the next. So, much of the
text will be the same between the "control sample" and the "test
sample," but some of it will be different.

I will be using Python or Jython for the test framework.

What I'd like to do is find an algorithm that produces the results of
a text comparison as a percentage-match. Thus I would be able to
assert that my test samples must match the control sample by at least
(for example) 83% for the test to pass. Clearly, this wouldn't be a
perfect test but it would help flag problems, which is primarily what
I need.

Does anyone know of an algorithm or library that would do this? Thanks
in advance.

Bruce Eckel
Br***@EckelObjects.com

[Helmut Jarausch]

Bruce Eckel wrote:

Background: for the 4th edition of Thinking in Java, I'm trying to
once again improve the testing scheme for the examples in the book. I
want to verify that the output I show in the book is "reasonably
correct." I say "Reasonably" because a number of examples produce
random numbers or text or the time of day or in general things that do
not repeat themselves from one execution to the next. So, much of the
text will be the same between the "control sample" and the "test
sample," but some of it will be different.

I will be using Python or Jython for the test framework.

What I'd like to do is find an algorithm that produces the results of
a text comparison as a percentage-match. Thus I would be able to
assert that my test samples must match the control sample by at least
(for example) 83% for the test to pass. Clearly, this wouldn't be a
perfect test but it would help flag problems, which is primarily what
I need.

Does anyone know of an algorithm or library that would do this? Thanks
in advance.

Sorry, not in Python, but only in Perl
I think
ftp://ftp.funet.fi/pub/languages/per...ox-3.23.tar.gz
can be tweaked to do that.
--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

[Helmut Jarausch]

Bruce Eckel wrote:

Background: for the 4th edition of Thinking in Java, I'm trying to
once again improve the testing scheme for the examples in the book. I
want to verify that the output I show in the book is "reasonably
correct." I say "Reasonably" because a number of examples produce
random numbers or text or the time of day or in general things that do
not repeat themselves from one execution to the next. So, much of the
text will be the same between the "control sample" and the "test
sample," but some of it will be different.

I will be using Python or Jython for the test framework.

What I'd like to do is find an algorithm that produces the results of
a text comparison as a percentage-match. Thus I would be able to
assert that my test samples must match the control sample by at least
(for example) 83% for the test to pass. Clearly, this wouldn't be a
perfect test but it would help flag problems, which is primarily what
I need.

Does anyone know of an algorithm or library that would do this? Thanks
in advance.

Sorry, not in Python, but only in Perl
I think
ftp://ftp.funet.fi/pub/languages/per...ox-3.23.tar.gz
can be tweaked to do that.
--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany

[Dan Bishop]

Bruce Eckel <Br********@MailBlocks.com> wrote in message news:<ma*************************************@pyth on.org>...

Background: for the 4th edition of Thinking in Java, I'm trying to
once again improve the testing scheme for the examples in the book. I
want to verify that the output I show in the book is "reasonably
correct." I say "Reasonably" because a number of examples produce
random numbers or text or the time of day or in general things that do
not repeat themselves from one execution to the next. So, much of the
text will be the same between the "control sample" and the "test
sample," but some of it will be different.

I will be using Python or Jython for the test framework.

What I'd like to do is find an algorithm that produces the results of
a text comparison as a percentage-match. Thus I would be able to
assert that my test samples must match the control sample by at least
(for example) 83% for the test to pass. Clearly, this wouldn't be a
perfect test but it would help flag problems, which is primarily what
I need.

Does anyone know of an algorithm or library that would do this? Thanks
in advance.

One of the simpler ones is to calculate the length of the longest
common subsequence of the test output and the control output.

def lcsLength(seqA, seqB):
lenTable = [[0] * len(seqB) for i in seqA]
# Set each lenTable[i][j] to lcsLength(seqA[:i+1], seqB[:j+1])
for i, a in enumerate(seqA):
for j, b in enumerate(seqB):
if a == b:
lenTable[i][j] = lenTable[i-1][j-1] + 1
else:
lenTable[i][j] = max(lenTable[i-1][j], lenTable[i][j-1])
return lenTable[-1][-1]

To convert this to a percentage value, simply divide by the length of
the control output.

Btw, thank you for those footnotes in Thinking in Java that encouraged
me to try Python :-)

[Oleg Paraschenko]

Hello Bruce,

Bruce Eckel <Br********@MailBlocks.com> wrote in message
news:<ma*************************************@pyth on.org>

...
What I'd like to do is find an algorithm that produces the results
of a text comparison as a percentage-match.
...
Does anyone know of an algorithm or library that would do this?
Thanks in advance.

I suggest you to look at my software, GetReuse and its SDK:

http://getreuse.com/
http://getreuse.com/sdk/

The formula for the calculation of the similarity is based on the
scientific research. Any other "good" method of calculations should
produce results that are equivalent in some terms to the GetReuse
results. I have not wrote a paper yet; the formula is a improvement
of the formula from http://www.cs.ucsb.edu/~mli/sid.ps . Unfortunately,
I froze the project but the current code is tested and should work well.
Bruce Eckel
Br***@EckelObjects.com

Regards, Oleg

[Mark 'Kamikaze' Hughes]

Bruce Eckel <Br********@MailBlocks.com>
wrote on Fri, 30 Jul 2004 07:52:39 -0600:

Background: for the 4th edition of Thinking in Java, I'm trying to
once again improve the testing scheme for the examples in the book. I
want to verify that the output I show in the book is "reasonably
correct." I say "Reasonably" because a number of examples produce
random numbers or text or the time of day or in general things that do
not repeat themselves from one execution to the next. So, much of the
text will be the same between the "control sample" and the "test
sample," but some of it will be different.

I will be using Python or Jython for the test framework.

What I'd like to do is find an algorithm that produces the results of
a text comparison as a percentage-match. Thus I would be able to
assert that my test samples must match the control sample by at least
(for example) 83% for the test to pass. Clearly, this wouldn't be a
perfect test but it would help flag problems, which is primarily what
I need.

Does anyone know of an algorithm or library that would do this? Thanks
in advance.

Here's an outside-the-box solution: set the random number seed and use
a fixed date in your tests. Now you can test fixed values, even though
the application is "random".

--
<a href="http://kuoi.asui.uidaho.edu/~kamikaze/"> Mark Hughes </a>
"Virtues foster one another; so too, vices.
Bad English kills trees, consumes energy, and befouls the Earth.
Good English renews it." -The Underground Grammarian, v1n2

[Steve Christensen]

In article <ma*************************************@python.or g>, Bruce
Eckel wrote:

What I'd like to do is find an algorithm that produces the results of
a text comparison as a percentage-match. Thus I would be able to
assert that my test samples must match the control sample by at least
(for example) 83% for the test to pass. Clearly, this wouldn't be a
perfect test but it would help flag problems, which is primarily what
I need.

Does anyone know of an algorithm or library that would do this? Thanks
in advance.

Have you come across the following yet?

Levenshtein C extension module for Python:
http://trific.ath.cx/resources/python/levenshtein/
And/or:
http://hetland.org/python/distance.py
-Steve