Removing Duplicates from a string

[M B HONG 20]

Hi all -

I was wondering if Javascript has a way to easily remove duplicates
from a string. For example, if I had a string:

"car truck car truck truck tree post post tree"

it should turn into:

"car truck tree post"

or something to that effect. The order of words does not matter for
me, so it does not have to be sorted in any particular way. If anyone
can shed some light on this I would greatly appreciate it. Thanks in
advance.

[web.dev]

M B HONG 20 wrote:

Hi all -

I was wondering if Javascript has a way to easily remove duplicates
from a string. For example, if I had a string:

"car truck car truck truck tree post post tree"

it should turn into:

"car truck tree post"

or something to that effect. The order of words does not matter for
me, so it does not have to be sorted in any particular way. If anyone
can shed some light on this I would greatly appreciate it. Thanks in
advance.

Hi M.B.

There is no built in method to remove duplicates from a string. But
you can make one. This is going off the top of my head so the code is
not complete:

function rmDuplicates(myStr)
{
/*take your string, split, sort, then join them back together.
your duplicates will be sorted and will be in order.
this should make it easier to remove if they are in a consecutive
pattern*/

var tmp = myStr;
var myArray = tmp.split(" ");
myArray.sort();
tmp = myArray.join(" ");

//use regular expression to find consecutive patterns and replace
...code...
}

Hope this will get you started on the right path.

[Thomas 'PointedEars' Lahn]

M B HONG 20 wrote:

I was wondering if Javascript has a way to easily remove duplicates
from a string. For example, if I had a string:

"car truck car truck truck tree post post tree"

it should turn into:

"car truck tree post"
[...]

JS objects based on Object are best for recognizing and removing dupes.
Quick hack (`s' refers to the string containing dupes):

for (var a = s.split(/\s+/), i = a.length, o = {}; i--;)
{
if (i < a.length - 1)
s += ' ';

if (!o[a[i]]) // [1]
s += a[i];
else
o[a[i]] = true;
}

or use

if (typeof o[a[i]] == "undefined")

instead of [1]. After that, `s' then refers to the reduced string.
One could also use another array, push elements and join them afterwards
instead of string concatenation, but after all it's only a quick hack.
HTH

PointedEars

[Thomas 'PointedEars' Lahn]

(Forget about my previous nonsense on this subject, I cancelled it.)

M B HONG 20 wrote:

I was wondering if Javascript has a way to easily remove duplicates
from a string. For example, if I had a string:

"car truck car truck truck tree post post tree"

it should turn into:

"car truck tree post"
[...]

JS objects based on Object are best for recognizing and removing dupes.
Quick hack (`s' refers to the string containing dupes):

for (var a = s.split(/\s+/), i = 0, o = {}, s2 = ''; i < a.length; i++)
{
if (i > 0)
s2 += ' ';

if (!o[a[i]])
{
s2 += a[i];
o[a[i]] = true;
}
}

or use

if (typeof o[a[i]] == "undefined")

instead of [1]. After that, `s2' refers to the reduced string.
One could also use another array, push elements and join them afterwards
instead of string concatenation, but after all it's only a quick hack.
HTH

PointedEars

[M B HONG 20]

Thanks a lot for your input, guys.

[Randy Webb]

Thomas 'PointedEars' Lahn said the following on 10/18/2005 3:32 PM:

(Forget about my previous nonsense on this subject, I cancelled it.)

No, you didn't cancel it, you just thought you did.

--
Randy
comp.lang.javascript FAQ - http://jibbering.com/faq & newsgroup weekly

[Mark Szlazak]

See if this or a similar regex solution will work.

inText = "car truck car truck truck tree post post tree";
rx = /\b(\w+)(?:(\s+)\1\b)+/g
outText = inText.replace(rx,'$1$2');
If the non-capture group (?: is not supported, change the regex and
ordinals to:
rx = /\b(\w+)((\s+)\1\b)+/g
outText = inText.replace(rx,'$1$3');
Also, if you need other non-word-character stuff in between like a tag
then replace (\s+) with ((?:\s|<[^>]+>)+)