473,396 Members | 1,760 Online
Bytes | Software Development & Data Engineering Community
Post Job

Home Posts Topics Members FAQ

Join Bytes to post your question to a community of 473,396 software developers and data experts.

How to remove duplicate words from text reference to thread89492

2
I read this thread and realized that it was exactly what I was looking for. The problem is I need it to be done with sentences not words. I have a problems with typing the same thought down twice in a large text column of thoughts and ideas. This piece of software that I found on thread thread89492 works really well for words, but not for sentences. I have not been able to figure out how to get it to work.


here goes the original code:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title></title>
<script type="text/javascript">

function uniqueTownsLRN(frm){
var towns = frm.elements['inputText'].value;
var arrTowns = towns.split(/\s+/);
var arrNewTowns = [];
var seenTowns = {};
for(var i=0;i<arrTowns.length;i++) {
if (!seenTowns[arrTowns[i]]) {
seenTowns[arrTowns[i]]=true;
arrNewTowns.push(arrTowns[i]);
}
}
frm.elements['outputText'].value = arrNewTowns.join(" ");
}

function uniqueTownsDJS(frm){
var towns = frm.elements['inputText'].value;
var arrTowns = towns.split(/\s+/).sort(), ans = "", Had = "";
for(var i=0;i < arrTowns.length;i++)
if( Had != (Had=arrTowns[i]) )ans += Had + " ";
frm.elements['outputText'].value = ans;
}

function uniqueTownsRC(frm){
var towns = frm.elements['inputText'].value;
var st = '', seenTowns = {}, arrTowns = towns.split(/\s+/);
for(var c = arrTowns.length;c--;){
seenTowns[arrTowns[c]] = true;
}
for(var prop in seenTowns){
st += prop+' ';
}
frm.elements['outputText'].value = st;
}

</script>

</head>
<body>
<form name="testForm" action="" method="get"
onsubmit="return subTests()">
<textarea name="inputText" cols="70" rows="8">St_Yves St_George_s_Mall
Aberdeen Aberdeen New_York Cape_Town Aberdeen Edinburgh
Edinburgh Inverness St_Yves St_George_s_Mall New_York
Aberdeen Aberdeen New_York Cape_Town Aberdeen Edinburgh
Edinburgh Inverness St_Yves St_George_s_Mall New_York
Cape_Town New_York Cape_Town</textarea><br>
<input type="button" value="Lasse Reichstein Nielsen Version"
onclick="uniqueTownsLRN(this.form);"><br>
<input type="button" value="Dr John Stockton Version"
onclick="uniqueTownsDJS(this.form);"><br>
<input type="button" value="Richard Cornford Version"
onclick="uniqueTownsRC(this.form);"><br>
<textarea name="outputText" cols="70" rows="8"></textarea>
</form>
</body>
</html>

Thnk You
Jun 27 '07 #1
2 2105
pbmods
5,821 Expert 4TB
Heya, woocha. Welcome to TSDN!

Easiest way to do it:
Expand|Select|Wrap|Line Numbers
  1. var theText = document.forms[0].inputText.value.split('\n');
  2.  
  3. var uniques = {};
  4. for(var i = 0; i < theText.length; i++)
  5.     uniques[theText[i]] = true;
  6.  
  7. document.forms[0].inputText.value = '';
  8. for(var i in uniques)
  9.     document.forms[0].inputText.value += i + '\n';
  10.  
This works because objects cannot have two properties with the same name. If you try to add a property to an object with the same name as an existing property, the newer property simply overwrites the older one.

Now, this script assumes that each sentence is delimited by a newline character. If you want to detect sentences that end with a period and two spaces, you'll need to make a slight change (replace '\n' with '. ').
Jun 27 '07 #2
woocha
2
Heya, woocha. Welcome to TSDN!

Easiest way to do it:
Expand|Select|Wrap|Line Numbers
  1. var theText = document.forms[0].inputText.value.split('\n');
  2.  
  3. var uniques = {};
  4. for(var i = 0; i < theText.length; i++)
  5.     uniques[theText[i]] = true;
  6.  
  7. document.forms[0].inputText.value = '';
  8. for(var i in uniques)
  9.     document.forms[0].inputText.value += i + '\n';
  10.  
This works because objects cannot have two properties with the same name. If you try to add a property to an object with the same name as an existing property, the newer property simply overwrites the older one.

Now, this script assumes that each sentence is delimited by a newline character. If you want to detect sentences that end with a period and two spaces, you'll need to make a slight change (replace '\n' with '. ').

Thank you for the tip, but I could not get it squared away. Any other advise?

I tried using these sentences:
the christmas cookies are great. I love christmas. the christmas cookies are great. so this is christmas. the christmas tree is so beautiful.
Jun 27 '07 #3

Sign in to post your reply or Sign up for a free account.

Similar topics

12
by: Sam Collett | last post by:
How do I remove an item with a specified value from an array? i.e. array values 1,2,2,5,7,12,15,21 remove 2 from array would return 1,5,7,12,15,21 (12 and 21 are NOT removed, duplicates are...
7
by: Voetleuce en fênsievry | last post by:
Hello everyone. I'm not a JavaScript author myself, but I'm looking for a method to remove duplicate words from a piece of text. This text would presumably be pasted into a text box. I have,...
11
by: Tony Johansson | last post by:
Hello! I have some problem with STL function remove I have two classes called Handle which is a template class and Integer which is not a template class. The Integer class is just a wrapper...
8
by: Joseph | last post by:
I have a textBox that people writes stories in it. They can use for format. I have Aspell installed on the server, so people can make correction to their text. Sometimes, they forget to add a...
1
by: Robert | last post by:
How can I query an existing table and update a field in each record in the table with the occurrence count of each record e.g. update field to 1 (= first record occurrence), update field to 2 for...
12
by: Jared Carr | last post by:
First I wish I knew how this was caused but here is our problem. Sometime in the recent past we got a duplicate table. Here is the result of a pg_dump with a pg_restore for just that table. ...
5
by: RSBakshi | last post by:
Can any body tell me how to find duplicate lines in C i have tried to find using Binary tree and Text files but not suceeded .. It works for Word but not for lines please help me you can...
5
by: kosta.triantafillou | last post by:
Hi all, I have a form that contains a lot of values. On this form there are also alot of popups that can be brought up. One of them does the following: Takes 2 values (x and y), concatenates...
8
natalie99
by: natalie99 | last post by:
hi everyone i have reseached this topic and cannot seem to find a solution that suits my problem, please help!! I have two tables, "Inventory" which contains 30,000 or so records, with about...
0
by: Charles Arthur | last post by:
How do i turn on java script on a villaon, callus and itel keypad mobile phone
0
by: ryjfgjl | last post by:
In our work, we often receive Excel tables with data in the same format. If we want to analyze these data, it can be difficult to analyze them because the data is spread across multiple Excel files...
1
by: nemocccc | last post by:
hello, everyone, I want to develop a software for my android phone for daily needs, any suggestions?
1
by: Sonnysonu | last post by:
This is the data of csv file 1 2 3 1 2 3 1 2 3 1 2 3 2 3 2 3 3 the lengths should be different i have to store the data by column-wise with in the specific length. suppose the i have to...
0
by: Hystou | last post by:
There are some requirements for setting up RAID: 1. The motherboard and BIOS support RAID configuration. 2. The motherboard has 2 or more available SATA protocol SSD/HDD slots (including MSATA, M.2...
0
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However,...
0
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven...
0
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows...
0
agi2029
by: agi2029 | last post by:
Let's talk about the concept of autonomous AI software engineers and no-code agents. These AIs are designed to manage the entire lifecycle of a software development project—planning, coding, testing,...

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.