470,810 Members | 855 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 470,810 developers. It's quick & easy.

Percentage matching of text

Background: for the 4th edition of Thinking in Java, I'm trying to
once again improve the testing scheme for the examples in the book. I
want to verify that the output I show in the book is "reasonably
correct." I say "Reasonably" because a number of examples produce
random numbers or text or the time of day or in general things that do
not repeat themselves from one execution to the next. So, much of the
text will be the same between the "control sample" and the "test
sample," but some of it will be different.

I will be using Python or Jython for the test framework.

What I'd like to do is find an algorithm that produces the results of
a text comparison as a percentage-match. Thus I would be able to
assert that my test samples must match the control sample by at least
(for example) 83% for the test to pass. Clearly, this wouldn't be a
perfect test but it would help flag problems, which is primarily what
I need.

Does anyone know of an algorithm or library that would do this? Thanks
in advance.

Bruce Eckel
Br***@EckelObjects.com
Jul 18 '05 #1
6 2257
Bruce Eckel wrote:
Background: for the 4th edition of Thinking in Java, I'm trying to
once again improve the testing scheme for the examples in the book. I
want to verify that the output I show in the book is "reasonably
correct." I say "Reasonably" because a number of examples produce
random numbers or text or the time of day or in general things that do
not repeat themselves from one execution to the next. So, much of the
text will be the same between the "control sample" and the "test
sample," but some of it will be different.

I will be using Python or Jython for the test framework.

What I'd like to do is find an algorithm that produces the results of
a text comparison as a percentage-match. Thus I would be able to
assert that my test samples must match the control sample by at least
(for example) 83% for the test to pass. Clearly, this wouldn't be a
perfect test but it would help flag problems, which is primarily what
I need.

Does anyone know of an algorithm or library that would do this? Thanks
in advance.


Sorry, not in Python, but only in Perl
I think
ftp://ftp.funet.fi/pub/languages/per...ox-3.23.tar.gz
can be tweaked to do that.
--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany
Jul 18 '05 #2
Bruce Eckel wrote:
Background: for the 4th edition of Thinking in Java, I'm trying to
once again improve the testing scheme for the examples in the book. I
want to verify that the output I show in the book is "reasonably
correct." I say "Reasonably" because a number of examples produce
random numbers or text or the time of day or in general things that do
not repeat themselves from one execution to the next. So, much of the
text will be the same between the "control sample" and the "test
sample," but some of it will be different.

I will be using Python or Jython for the test framework.

What I'd like to do is find an algorithm that produces the results of
a text comparison as a percentage-match. Thus I would be able to
assert that my test samples must match the control sample by at least
(for example) 83% for the test to pass. Clearly, this wouldn't be a
perfect test but it would help flag problems, which is primarily what
I need.

Does anyone know of an algorithm or library that would do this? Thanks
in advance.


Sorry, not in Python, but only in Perl
I think
ftp://ftp.funet.fi/pub/languages/per...ox-3.23.tar.gz
can be tweaked to do that.
--
Helmut Jarausch

Lehrstuhl fuer Numerische Mathematik
RWTH - Aachen University
D 52056 Aachen, Germany
Jul 18 '05 #3
Bruce Eckel <Br********@MailBlocks.com> wrote in message news:<ma*************************************@pyth on.org>...
Background: for the 4th edition of Thinking in Java, I'm trying to
once again improve the testing scheme for the examples in the book. I
want to verify that the output I show in the book is "reasonably
correct." I say "Reasonably" because a number of examples produce
random numbers or text or the time of day or in general things that do
not repeat themselves from one execution to the next. So, much of the
text will be the same between the "control sample" and the "test
sample," but some of it will be different.

I will be using Python or Jython for the test framework.

What I'd like to do is find an algorithm that produces the results of
a text comparison as a percentage-match. Thus I would be able to
assert that my test samples must match the control sample by at least
(for example) 83% for the test to pass. Clearly, this wouldn't be a
perfect test but it would help flag problems, which is primarily what
I need.

Does anyone know of an algorithm or library that would do this? Thanks
in advance.


One of the simpler ones is to calculate the length of the longest
common subsequence of the test output and the control output.

def lcsLength(seqA, seqB):
lenTable = [[0] * len(seqB) for i in seqA]
# Set each lenTable[i][j] to lcsLength(seqA[:i+1], seqB[:j+1])
for i, a in enumerate(seqA):
for j, b in enumerate(seqB):
if a == b:
lenTable[i][j] = lenTable[i-1][j-1] + 1
else:
lenTable[i][j] = max(lenTable[i-1][j], lenTable[i][j-1])
return lenTable[-1][-1]

To convert this to a percentage value, simply divide by the length of
the control output.

Btw, thank you for those footnotes in Thinking in Java that encouraged
me to try Python :-)
Jul 18 '05 #4
Hello Bruce,

Bruce Eckel <Br********@MailBlocks.com> wrote in message
news:<ma*************************************@pyth on.org>
...
What I'd like to do is find an algorithm that produces the results
of a text comparison as a percentage-match.
...
Does anyone know of an algorithm or library that would do this?
Thanks in advance.

I suggest you to look at my software, GetReuse and its SDK:

http://getreuse.com/
http://getreuse.com/sdk/

The formula for the calculation of the similarity is based on the
scientific research. Any other "good" method of calculations should
produce results that are equivalent in some terms to the GetReuse
results. I have not wrote a paper yet; the formula is a improvement
of the formula from http://www.cs.ucsb.edu/~mli/sid.ps . Unfortunately,
I froze the project but the current code is tested and should work well.
Bruce Eckel
Br***@EckelObjects.com


Regards, Oleg
Jul 18 '05 #5
Bruce Eckel <Br********@MailBlocks.com>
wrote on Fri, 30 Jul 2004 07:52:39 -0600:
Background: for the 4th edition of Thinking in Java, I'm trying to
once again improve the testing scheme for the examples in the book. I
want to verify that the output I show in the book is "reasonably
correct." I say "Reasonably" because a number of examples produce
random numbers or text or the time of day or in general things that do
not repeat themselves from one execution to the next. So, much of the
text will be the same between the "control sample" and the "test
sample," but some of it will be different.

I will be using Python or Jython for the test framework.

What I'd like to do is find an algorithm that produces the results of
a text comparison as a percentage-match. Thus I would be able to
assert that my test samples must match the control sample by at least
(for example) 83% for the test to pass. Clearly, this wouldn't be a
perfect test but it would help flag problems, which is primarily what
I need.

Does anyone know of an algorithm or library that would do this? Thanks
in advance.


Here's an outside-the-box solution: set the random number seed and use
a fixed date in your tests. Now you can test fixed values, even though
the application is "random".

--
<a href="http://kuoi.asui.uidaho.edu/~kamikaze/"> Mark Hughes </a>
"Virtues foster one another; so too, vices.
Bad English kills trees, consumes energy, and befouls the Earth.
Good English renews it." -The Underground Grammarian, v1n2
Jul 18 '05 #6
In article <ma*************************************@python.or g>, Bruce
Eckel wrote:

What I'd like to do is find an algorithm that produces the results of
a text comparison as a percentage-match. Thus I would be able to
assert that my test samples must match the control sample by at least
(for example) 83% for the test to pass. Clearly, this wouldn't be a
perfect test but it would help flag problems, which is primarily what
I need.

Does anyone know of an algorithm or library that would do this? Thanks
in advance.


Have you come across the following yet?

Levenshtein C extension module for Python:
http://trific.ath.cx/resources/python/levenshtein/
And/or:
http://hetland.org/python/distance.py
-Steve
Jul 18 '05 #7

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

9 posts views Thread by Xah Lee | last post: by
7 posts views Thread by Rob Meade | last post: by
2 posts views Thread by Oliver Burnett-Hall | last post: by
3 posts views Thread by Day Of The Eagle | last post: by
6 posts views Thread by Hacking Bear | last post: by
1 post views Thread by VanKha | last post: by
11 posts views Thread by tech | last post: by
reply views Thread by mihailmihai484 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.