473,804 Members | 2,615 Online
Bytes | Software Development & Data Engineering Community
+ Post

Home Posts Topics Members FAQ

How to remove duplicate words from text reference to thread89492

2 New Member
I read this thread and realized that it was exactly what I was looking for. The problem is I need it to be done with sentences not words. I have a problems with typing the same thought down twice in a large text column of thoughts and ideas. This piece of software that I found on thread thread89492 works really well for words, but not for sentences. I have not been able to figure out how to get it to work.


here goes the original code:

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN"
"http://www.w3.org/TR/html4/loose.dtd">
<html>
<head>
<title></title>
<script type="text/javascript">

function uniqueTownsLRN( frm){
var towns = frm.elements['inputText'].value;
var arrTowns = towns.split(/\s+/);
var arrNewTowns = [];
var seenTowns = {};
for(var i=0;i<arrTowns. length;i++) {
if (!seenTowns[arrTowns[i]]) {
seenTowns[arrTowns[i]]=true;
arrNewTowns.pus h(arrTowns[i]);
}
}
frm.elements['outputText'].value = arrNewTowns.joi n(" ");
}

function uniqueTownsDJS( frm){
var towns = frm.elements['inputText'].value;
var arrTowns = towns.split(/\s+/).sort(), ans = "", Had = "";
for(var i=0;i < arrTowns.length ;i++)
if( Had != (Had=arrTowns[i]) )ans += Had + " ";
frm.elements['outputText'].value = ans;
}

function uniqueTownsRC(f rm){
var towns = frm.elements['inputText'].value;
var st = '', seenTowns = {}, arrTowns = towns.split(/\s+/);
for(var c = arrTowns.length ;c--;){
seenTowns[arrTowns[c]] = true;
}
for(var prop in seenTowns){
st += prop+' ';
}
frm.elements['outputText'].value = st;
}

</script>

</head>
<body>
<form name="testForm" action="" method="get"
onsubmit="retur n subTests()">
<textarea name="inputText " cols="70" rows="8">St_Yve s St_George_s_Mal l
Aberdeen Aberdeen New_York Cape_Town Aberdeen Edinburgh
Edinburgh Inverness St_Yves St_George_s_Mal l New_York
Aberdeen Aberdeen New_York Cape_Town Aberdeen Edinburgh
Edinburgh Inverness St_Yves St_George_s_Mal l New_York
Cape_Town New_York Cape_Town</textarea><br>
<input type="button" value="Lasse Reichstein Nielsen Version"
onclick="unique TownsLRN(this.f orm);"><br>
<input type="button" value="Dr John Stockton Version"
onclick="unique TownsDJS(this.f orm);"><br>
<input type="button" value="Richard Cornford Version"
onclick="unique TownsRC(this.fo rm);"><br>
<textarea name="outputTex t" cols="70" rows="8"></textarea>
</form>
</body>
</html>

Thnk You
Jun 27 '07 #1
2 2144
pbmods
5,821 Recognized Expert Expert
Heya, woocha. Welcome to TSDN!

Easiest way to do it:
Expand|Select|Wrap|Line Numbers
  1. var theText = document.forms[0].inputText.value.split('\n');
  2.  
  3. var uniques = {};
  4. for(var i = 0; i < theText.length; i++)
  5.     uniques[theText[i]] = true;
  6.  
  7. document.forms[0].inputText.value = '';
  8. for(var i in uniques)
  9.     document.forms[0].inputText.value += i + '\n';
  10.  
This works because objects cannot have two properties with the same name. If you try to add a property to an object with the same name as an existing property, the newer property simply overwrites the older one.

Now, this script assumes that each sentence is delimited by a newline character. If you want to detect sentences that end with a period and two spaces, you'll need to make a slight change (replace '\n' with '. ').
Jun 27 '07 #2
woocha
2 New Member
Heya, woocha. Welcome to TSDN!

Easiest way to do it:
Expand|Select|Wrap|Line Numbers
  1. var theText = document.forms[0].inputText.value.split('\n');
  2.  
  3. var uniques = {};
  4. for(var i = 0; i < theText.length; i++)
  5.     uniques[theText[i]] = true;
  6.  
  7. document.forms[0].inputText.value = '';
  8. for(var i in uniques)
  9.     document.forms[0].inputText.value += i + '\n';
  10.  
This works because objects cannot have two properties with the same name. If you try to add a property to an object with the same name as an existing property, the newer property simply overwrites the older one.

Now, this script assumes that each sentence is delimited by a newline character. If you want to detect sentences that end with a period and two spaces, you'll need to make a slight change (replace '\n' with '. ').

Thank you for the tip, but I could not get it squared away. Any other advise?

I tried using these sentences:
the christmas cookies are great. I love christmas. the christmas cookies are great. so this is christmas. the christmas tree is so beautiful.
Jun 27 '07 #3

Sign in to post your reply or Sign up for a free account.

Similar topics

12
55575
by: Sam Collett | last post by:
How do I remove an item with a specified value from an array? i.e. array values 1,2,2,5,7,12,15,21 remove 2 from array would return 1,5,7,12,15,21 (12 and 21 are NOT removed, duplicates are also removed) So far I have (val is value, ar is array, returns new array):
7
23937
by: Voetleuce en fênsievry | last post by:
Hello everyone. I'm not a JavaScript author myself, but I'm looking for a method to remove duplicate words from a piece of text. This text would presumably be pasted into a text box. I have, for example, a list of town names, but there are hundreds of duplicates, like: "Aberdeen Aberdeen Aberdeen Edinburg Edinburg Inverness etc."
11
3096
by: Tony Johansson | last post by:
Hello! I have some problem with STL function remove I have two classes called Handle which is a template class and Integer which is not a template class. The Integer class is just a wrapper class for a primitive int with some methods. I don't show the Integer class because it will not add any information to my problem. Main is using some STL function Now to my problem.
8
4210
by: Joseph | last post by:
I have a textBox that people writes stories in it. They can use for format. I have Aspell installed on the server, so people can make correction to their text. Sometimes, they forget to add a space after a period. So Aspell reads the last word of a sentence and the first word of the next sentence as a single world, which gives an error, word not found. Before sending the text to Aspell, i add a space after all the periods with:
1
4353
by: Robert | last post by:
How can I query an existing table and update a field in each record in the table with the occurrence count of each record e.g. update field to 1 (= first record occurrence), update field to 2 for 2nd record occurrence i.e. first duplicate record), update the field to 3 for the 3rd record occurrence i.e. 2nd duplicate record Example for a duplicated record (field with value ABCD) Rec 1 ABCD 1 Rec 2 ABCD ...
12
3720
by: Jared Carr | last post by:
First I wish I knew how this was caused but here is our problem. Sometime in the recent past we got a duplicate table. Here is the result of a pg_dump with a pg_restore for just that table. -- -- TOC entry 59 (OID 11462032) -- Name: order_to_do; Type: TABLE; Schema: public; Owner: www -- Data Pos: 0 --
5
2135
by: RSBakshi | last post by:
Can any body tell me how to find duplicate lines in C i have tried to find using Binary tree and Text files but not suceeded .. It works for Word but not for lines please help me you can write suggetion to my email : rsbakshi@gmail.com
5
2469
by: kosta.triantafillou | last post by:
Hi all, I have a form that contains a lot of values. On this form there are also alot of popups that can be brought up. One of them does the following: Takes 2 values (x and y), concatenates them together with an underscore in between and adds them to a select box in the same popup (z). Then, when a user clicks a save button located on the popup, it sends that information back to the main form and populates the same three values (x,...
8
3867
natalie99
by: natalie99 | last post by:
hi everyone i have reseached this topic and cannot seem to find a solution that suits my problem, please help!! I have two tables, "Inventory" which contains 30,000 or so records, with about 2000 duplicates. Each duplicate contains the exact same data in every field except one, Field "Code", which may have "Agreed" or "Rejected". The second table, "Duplicate Coding" contains the correct field value for "Code" for the records which are...
0
9706
marktang
by: marktang | last post by:
ONU (Optical Network Unit) is one of the key components for providing high-speed Internet services. Its primary function is to act as an endpoint device located at the user's premises. However, people are often confused as to whether an ONU can Work As a Router. In this blog post, we’ll explore What is ONU, What Is Router, ONU & Router’s main usage, and What is the difference between ONU and Router. Let’s take a closer look ! Part I. Meaning of...
0
9577
by: Hystou | last post by:
Most computers default to English, but sometimes we require a different language, especially when relocating. Forgot to request a specific language before your computer shipped? No problem! You can effortlessly switch the default language on Windows 10 without reinstalling. I'll walk you through it. First, let's disable language synchronization. With a Microsoft account, language settings sync across devices. To prevent any complications,...
0
10569
Oralloy
by: Oralloy | last post by:
Hello folks, I am unable to find appropriate documentation on the type promotion of bit-fields when using the generalised comparison operator "<=>". The problem is that using the GNU compilers, it seems that the internal comparison operator "<=>" tries to promote arguments from unsigned to signed. This is as boiled down as I can make it. Here is my compilation command: g++-12 -std=c++20 -Wnarrowing bit_field.cpp Here is the code in...
0
10325
jinu1996
by: jinu1996 | last post by:
In today's digital age, having a compelling online presence is paramount for businesses aiming to thrive in a competitive landscape. At the heart of this digital strategy lies an intricately woven tapestry of website design and digital marketing. It's not merely about having a website; it's about crafting an immersive digital experience that captivates audiences and drives business growth. The Art of Business Website Design Your website is...
1
10315
by: Hystou | last post by:
Overview: Windows 11 and 10 have less user interface control over operating system update behaviour than previous versions of Windows. In Windows 11 and 10, there is no way to turn off the Windows Update option using the Control Panel or Settings app; it automatically checks for updates and installs any it finds, whether you like it or not. For most users, this new feature is actually very convenient. If you want to control the update process,...
0
10075
tracyyun
by: tracyyun | last post by:
Dear forum friends, With the development of smart home technology, a variety of wireless communication protocols have appeared on the market, such as Zigbee, Z-Wave, Wi-Fi, Bluetooth, etc. Each protocol has its own unique characteristics and advantages, but as a user who is planning to build a smart home system, I am a bit confused by the choice of these technologies. I'm particularly interested in Zigbee because I've heard it does some...
1
7615
isladogs
by: isladogs | last post by:
The next Access Europe User Group meeting will be on Wednesday 1 May 2024 starting at 18:00 UK time (6PM UTC+1) and finishing by 19:30 (7.30PM). In this session, we are pleased to welcome a new presenter, Adolph Dupré who will be discussing some powerful techniques for using class modules. He will explain when you may want to use classes instead of User Defined Types (UDT). For example, to manage the data in unbound forms. Adolph will...
0
5519
by: TSSRALBI | last post by:
Hello I'm a network technician in training and I need your help. I am currently learning how to create and manage the different types of VPNs and I have a question about LAN-to-LAN VPNs. The last exercise I practiced was to create a LAN-to-LAN VPN between two Pfsense firewalls, by using IPSEC protocols. I succeeded, with both firewalls in the same network. But I'm wondering if it's possible to do the same thing, with 2 Pfsense firewalls...
1
4295
by: 6302768590 | last post by:
Hai team i want code for transfer the data from one system to another through IP address by using C# our system has to for every 5mins then we have to update the data what the data is updated we have to send another system

By using Bytes.com and it's services, you agree to our Privacy Policy and Terms of Use.

To disable or enable advertisements and analytics tracking please visit the manage ads & tracking page.