469,085 Members | 1,030 Online
Bytes | Developer Community
New Post

Home Posts Topics Members FAQ

Post your question to a community of 469,085 developers. It's quick & easy.

Anti-spam techniques for web pages?

JW
I'm putting together a bulletin board system for a community that allows folks
to put up for sale postings, notices, etc. These notices usually include the
posters' email addresses.

A similar posting service which I use is in place in another town and I do
occasionally get spam from my notices (not a lot, usually 419 spam.)

This is a free system and doesn't require a login, and I want to avoid/minimize
the possibility of spammers from harvesting these addresses while at the same
time maintaining the mailto: links.

Basic requirements are 1) easily automated when I create the posting pages
(MySQL->PHP->HTML) and 2) maintain user convenience (clickable links regardless
of browser, usable by sight impaired readers, ...)

But should I even bother? Is there anything that is even partially effective
against harvesting bots? Is there a tried and true obfuscation method that I can
use?

Thanks!
--
jwayne@_myrealbox_no_spam.com
Jul 20 '05 #1
25 4067
See http://www.cdt.org/speech/spam/030319spamreport.shtml
and http://www.wbwip.com/wbw/emailencoder.html

In my experience, newsgroups are the biggest exposure!

--
######################
## PH, London ##
######################

"JW" <jwayne@_myrealbox_no_spam.com> wrote in message
news:qi********************************@4ax.com...
I'm putting together a bulletin board system for a community that allows folks to put up for sale postings, notices, etc. These notices usually include the posters' email addresses.

A similar posting service which I use is in place in another town and I do
occasionally get spam from my notices (not a lot, usually 419 spam.)

This is a free system and doesn't require a login, and I want to avoid/minimize the possibility of spammers from harvesting these addresses while at the same time maintaining the mailto: links.

Basic requirements are 1) easily automated when I create the posting pages
(MySQL->PHP->HTML) and 2) maintain user convenience (clickable links regardless of browser, usable by sight impaired readers, ...)

But should I even bother? Is there anything that is even partially effective against harvesting bots? Is there a tried and true obfuscation method that I can use?

Thanks!
--
jwayne@_myrealbox_no_spam.com

Jul 20 '05 #2
JW wrote:
I'm putting together a bulletin board system for a community that allows folks
to put up for sale postings, notices, etc. [...]
This is a free system and doesn't require a login, and I want to avoid/minimize
the possibility of spammers from harvesting these addresses while at the same
time maintaining the mailto: links. [...]


I've had a good deal of success using character encoding to obfuscate
email addresses. HTML allows "&#xxx;" as a substitute for a character
where 'xxx' is the appropriate number (usually the ASCII code).

<A href="mailto:willondon@bigfoot.com">
willondon@bigfoot.com</A>
uses 'mailto' to send to 'w********@bigfoot.com'.

Advantages:
(1) easy to program;
(2) seems to work with all browsers and email clients I've tried; the mail
client opens a compose window with the correct (uncoded) address in the
'To:' field;
(3) I have two pages, published for over 6 years, where I'm sure I've
never used the address anywhere else, and have yet to receive spam to
those addresses.

Disadvantages (apart from 'mailto' inherent disadvantages):
(1) from autumn 2002, I've read reports that scumbots are now decoding
these addresses;
(2) even 6 years ago, it seemed that ultimately, it's harvestable, and I'm
very surprised that I've had as much success as I have; not guaranteed to
last forever.

If you make an email address visible to a browser or mail client, it *has*
to be visible to a scumbot, too. There's no way around that.

A common approach is to not publish the address, but use CGI or other
server-side programming to take care of inputting the message and mailing
it; but you might consider this too much processing.
Hope that helps,

--
Willondon
Jul 20 '05 #3
JW wrote:
I'm putting together a bulletin board system for a community that allows folks
to put up for sale postings, notices, etc. These notices usually include the
posters' email addresses.
This is a free system and doesn't require a login, and I want to avoid/minimize
the possibility of spammers from harvesting these addresses while at the same
time maintaining the mailto: links.


The techniques mentioned in other posts, such as using URL encoding or numeric
entities in the email address, prevent harvesting of information by simple bots, and
most of today's bots are simple because they get enough email addresses using simple
techniques and don't bother going for the difficult ones. As more people make their
email addresses harder to extract, this may change.

If you could replace the mailto links with a Web form for sending messages, you
could display only partial email addresses like "jwayne@myrealb..." and keep the
full email address in your database instead of displaying it on the Web page.

--
Klaus Johannes Rusch
Kl********@atmedia.net
http://www.atmedia.net/KlausRusch/
Jul 20 '05 #4
I use a technique which uses JavaScript to assemble the email address on the
fly. Users with JavaScript disabled get a warning. I also use the @
code instead of @. The JavaScript is in a function loaded from a separate
file.

However, even these paranoid eccentricities may be in vain. I have a domain
with a default forwarding, so I can make up email addresses on the fly.
I've spotted spam arriving less than an hour after first use of a new
address, so someone is using a packet sniffer somewhere.

Hanging's too good for them.

--
######################
## PH, London ##
######################

"Klaus Johannes Rusch" <Kl********@atmedia.net> wrote in message
news:3F***************@atmedia.net...
JW wrote:
I'm putting together a bulletin board system for a community that allows folks to put up for sale postings, notices, etc. These notices usually include the posters' email addresses.
This is a free system and doesn't require a login, and I want to avoid/minimize the possibility of spammers from harvesting these addresses while at the same time maintaining the mailto: links.
The techniques mentioned in other posts, such as using URL encoding or

numeric entities in the email address, prevent harvesting of information by simple bots, and most of today's bots are simple because they get enough email addresses using simple techniques and don't bother going for the difficult ones. As more people make their email addresses harder to extract, this may change.

If you could replace the mailto links with a Web form for sending messages, you could display only partial email addresses like "jwayne@myrealb..." and keep the full email address in your database instead of displaying it on the Web page.
--
Klaus Johannes Rusch
Kl********@atmedia.net
http://www.atmedia.net/KlausRusch/

Jul 20 '05 #5
In article <3F***************@atmedia.net> in
comp.infosystems.www.authoring.html, Klaus Johannes Rusch
<Kl********@atmedia.net> wrote:
If you could replace the mailto links with a Web form for sending messages, you
could display only partial email addresses like "jwayne@myrealb..." and keep the
full email address in your database instead of displaying it on the Web page.


I _hate_ those forms. If I'm sending mail, I want to use my mailer.
It has things like spell checking and the copy-to-self feature. And
half the time I type a message into those forms and then nothing
happens when I click 'send'.

--
Stan Brown, Oak Road Systems, Cortland County, New York, USA
http://OakRoadSystems.com/
HTML 4.01 spec: http://www.w3.org/TR/html401/
validator: http://validator.w3.org/
CSS 2 spec: http://www.w3.org/TR/REC-CSS2/
validator: http://jigsaw.w3.org/css-validator/
Jul 20 '05 #6
"Philip Herlihy" <fo******@REMOVEherlihy.eu.com> wrote in
news:be**********@hercules.btinternet.com:
I use a technique which uses JavaScript to assemble the email address
on the fly. Users with JavaScript disabled get a warning.


A slightly better approach(in my opinion) is what we do on our
'Contact Us' page:
http://www.confluence.org/contact.php
With javascript enabled, you'll get mailto links, but the
email addresses aren't in a readable format in the source.
If javascript is disabled, you'll see the email addresses,
as images(no good for 'cut & paste", but at least they are available).

--
Dave Patton
Canadian Coordinator, the Degree Confluence Project
http://www.confluence.org dpatton at confluence dot org
My website: http://members.shaw.ca/davepatton/
Vancouver/Whistler - host of the 2010 Winter Olympics
Jul 20 '05 #7
Neat!

--
######################
## PH, London ##
######################

"Dave Patton" <dp*****@remove-for-nospam.confluence.org> wrote in message
news:Xn******************************@24.71.223.15 9...
"Philip Herlihy" <fo******@REMOVEherlihy.eu.com> wrote in
news:be**********@hercules.btinternet.com:
I use a technique which uses JavaScript to assemble the email address
on the fly. Users with JavaScript disabled get a warning.


A slightly better approach(in my opinion) is what we do on our
'Contact Us' page:
http://www.confluence.org/contact.php
With javascript enabled, you'll get mailto links, but the
email addresses aren't in a readable format in the source.
If javascript is disabled, you'll see the email addresses,
as images(no good for 'cut & paste", but at least they are available).

--
Dave Patton
Canadian Coordinator, the Degree Confluence Project
http://www.confluence.org dpatton at confluence dot org
My website: http://members.shaw.ca/davepatton/
Vancouver/Whistler - host of the 2010 Winter Olympics

Jul 20 '05 #8

"Stan Brown" <th************@fastmail.fm> wrote in message
news:MP***********************@news.odyssey.net...
In article <3F***************@atmedia.net> in
comp.infosystems.www.authoring.html, Klaus Johannes Rusch
<Kl********@atmedia.net> wrote:
If you could replace the mailto links with a Web form for sending messages, youcould display only partial email addresses like "jwayne@myrealb..." and keep thefull email address in your database instead of displaying it on the Web
page.
I _hate_ those forms. If I'm sending mail, I want to use my mailer.
It has things like spell checking and the copy-to-self feature. And
half the time I type a message into those forms and then nothing
happens when I click 'send'.


I've actually begun getting more contact from my website after converting to
contact form only.
The number of new website contacts is exponentially higher than before.
--
Karl Core

Charles Sweeney says my sig is fine as it is.
Jul 20 '05 #9
In article <3F***************@atmedia.net> in
comp.infosystems.www.authoring.html, Klaus Johannes Rusch
<Kl********@atmedia.net> wrote:
One option might be to not
show the email address but let you retrieve the email address by having it sent to
your address, so you can then use your mailer to respond.


How many people do you think are willing to jump through such hoops?
If even 10% refuse (or can't figure out your scheme), that's 10% of
sales lost.

--
Stan Brown, Oak Road Systems, Cortland County, New York, USA
http://OakRoadSystems.com/
HTML 4.01 spec: http://www.w3.org/TR/html401/
validator: http://validator.w3.org/
CSS 2 spec: http://www.w3.org/TR/REC-CSS2/
validator: http://jigsaw.w3.org/css-validator/
Jul 20 '05 #10
On Mon, 7 Jul 2003, Willondon wrote:
JW wrote:
... This is a free system and doesn't require a login, and I want to
avoid/minimize the possibility of spammers from harvesting these addresses
while at the same time maintaining the mailto: links. [...]
I've had a good deal of success using character encoding to obfuscate
email addresses. HTML allows "&#xxx;" as a substitute for a character
where 'xxx' is the appropriate number (usually the ASCII code).
...
Disadvantages (apart from 'mailto' inherent disadvantages):
(1) from autumn 2002, I've read reports that scumbots are now decoding
these addresses;
(2) even 6 years ago, it seemed that ultimately, it's harvestable, and I'm
very surprised that I've had as much success as I have; not guaranteed to
last forever.


Even if they weren't decoding these, it would be inevitable that some spammer
would write a robot that could. Obfuscation isn't a viable solution. When
spammers started using the method of "mixing and matching" usernames and
domains to create mailboxes they hadn't seen (and other dictionary and random
attacks), it didn't even matter that a mailbox appeared on a web page - it
could still be randomly hit.
A common approach is to not publish the address, but use CGI or other
server-side programming to take care of inputting the message and mailing
it; but you might consider this too much processing.


I certainly agree with this. Unfortunately, for sites hosted on machines not
owned by the web site owner, this isn't always an option.

The usual tricks of embedding other mailto's not visible to those using the
standard web browsers which contain spam-trap mailboxes and such can still be
employed.

I use the approach of making the mailbox address useless to the spammer, even
if he has it. If he cannot get his message through, then what good is it to
him? Blocking alone shouldn't be the only step, but in many cases, it is
satisfactory, especially if one lacks the time to investigate the spam back to
the spammer's true mailbox or web site (for those which advertise web sites)
and get those shut down.
Jul 20 '05 #11
On Tue, 8 Jul 2003, Klaus Johannes Rusch wrote:
Stan Brown wrote:
...
I _hate_ those forms. If I'm sending mail, I want to use my mailer.
It has things like spell checking and the copy-to-self feature. And
half the time I type a message into those forms and then nothing
happens when I click 'send'.


Everyone does, but the same is true for being spammed. One option might be to not
show the email address but let you retrieve the email address by having it sent to
your address, so you can then use your mailer to respond.


Interesting idea, but once spammers see this sort of thing, what's to stop THEM
from having your web server deliver your e-mail address to them?
Jul 20 '05 #12
Stan Brown wrote:
How many people do you think are willing to jump through such hoops?
If even 10% refuse (or can't figure out your scheme), that's 10% of
sales lost.


If 20% of the visitors don't post their ads because they are afraid of being spammed,
that's 20% of sales lost. No single technique will work for everyone.

--
Klaus Johannes Rusch
Kl********@atmedia.net
http://www.atmedia.net/KlausRusch/
Jul 20 '05 #13
"D. Stussy" wrote:
Interesting idea, but once spammers see this sort of thing, what's to stop THEM
from having your web server deliver your e-mail address to them?


Theoretically nothing. Practically unless you have a very attractive and huge set of
email addresses spambot writers will not bother writing custom code to gather email
addresses from your site, but instead will go for easier targets.

--
Klaus Johannes Rusch
Kl********@atmedia.net
http://www.atmedia.net/KlausRusch/
Jul 20 '05 #14
Dave Patton wrote:
I use a technique which uses JavaScript to assemble the email address
on the fly. Users with JavaScript disabled get a warning.


A slightly better approach(in my opinion) is what we do on our
'Contact Us' page:
http://www.confluence.org/contact.php
With javascript enabled, you'll get mailto links, but the
email addresses aren't in a readable format in the source.
If javascript is disabled, you'll see the email addresses,
as images(no good for 'cut & paste", but at least they are available).


I use js to write the mailto: link. In the noscript element, I put my
email address in as English words, i.e.,
username (at) example.com

Like other solutions, not perfect. Perhaps useless. But I want to do
*something*. My solutions to junk postal mail and telemarketing calls
are better, but spam, well, I just don't know what to do. I tried the
react thing: read the headers, find the real source, contact the host.
But at 5 minutes or more for *each* spam, it's too much time. Not
with the number of spams I receive.

--
Brian
follow the directions in my address to email me

Jul 20 '05 #15
In article <GjYOa.19674$ye4.16889@sccrnsc01>, br***@wfcr.org.invalid-remove-
this-part says...
Dave Patton wrote:
I use a technique which uses JavaScript to assemble the email address
on the fly. Users with JavaScript disabled get a warning.


A slightly better approach(in my opinion) is what we do on our
'Contact Us' page:
http://www.confluence.org/contact.php
With javascript enabled, you'll get mailto links, but the
email addresses aren't in a readable format in the source.
If javascript is disabled, you'll see the email addresses,
as images(no good for 'cut & paste", but at least they are available).


I use js to write the mailto: link. In the noscript element, I put my
email address in as English words, i.e.,
username (at) example.com

Like other solutions, not perfect. Perhaps useless. But I want to do
*something*. My solutions to junk postal mail and telemarketing calls
are better, but spam, well, I just don't know what to do. I tried the
react thing: read the headers, find the real source, contact the host.
But at 5 minutes or more for *each* spam, it's too much time. Not
with the number of spams I receive.

Download RedSquirrel's SecurEmailL script. Works very nicely on PHP hosts.
http://rsscripts.tripod.com/scripts/securemaill.htm
Jul 20 '05 #16
On Wed, 9 Jul 2003, Klaus Johannes Rusch wrote:
"D. Stussy" wrote:
Interesting idea, but once spammers see this sort of thing, what's to stop THEM
from having your web server deliver your e-mail address to them?


Theoretically nothing. Practically unless you have a very attractive and huge set of
email addresses spambot writers will not bother writing custom code to gather email
addresses from your site, but instead will go for easier targets.


An assumption about their behavior? As time goes on, even spammers will have
computers with more processing power, so with that added ability, why shouldn't
they? I bet that the next spam innovation will be the javascript-interpreting
spambot, so the only thing that I see in the way of this is simply "higher
harvesting priorities" and that they will eventually get to this (maybe a few
years from now; maybe months, but they will get there).

Such a target will be an "attractive target" to them because of the method
employed, the spammers will KNOW that the mailbox is a valid one.
Jul 20 '05 #17
Mr. Clean wrote:
I use js to write the mailto: link. In the noscript element, I
put my email address in as English words, i.e., username (at)
example.com


Download RedSquirrel's SecurEmailL script. Works very nicely on PHP
hosts. http://rsscripts.tripod.com/scripts/securemaill.htm


I already have a contact form. But some users prefer email, since
their email clients allow them to keep a copy for themselves, or cc:
it to someone else. Thus, I have a form and an email link.

--
Brian
follow the directions in my address to email me

Jul 20 '05 #18
Mr. Clean wrote:
I use js to write the mailto: link. In the noscript element, I
put my email address in as English words, i.e., username (at)
example.com


Download RedSquirrel's SecurEmailL script. Works very nicely on PHP
hosts. http://rsscripts.tripod.com/scripts/securemaill.htm


I already have a contact form. But some users prefer email, since
their email clients allow them to keep a copy for themselves, or cc:
it to someone else. Thus, I have a form and an email link.

--
Brian
follow the directions in my address to email me

Jul 20 '05 #19
In article <3RlRa.80008$Ph3.8779@sccrnsc04>, br***@wfcr.org.invalid-remove-
this-part says...
Mr. Clean wrote:
I use js to write the mailto: link. In the noscript element, I
put my email address in as English words, i.e., username (at)
example.com


Download RedSquirrel's SecurEmailL script. Works very nicely on PHP
hosts. http://rsscripts.tripod.com/scripts/securemaill.htm


I already have a contact form. But some users prefer email, since
their email clients allow them to keep a copy for themselves, or cc:
it to someone else. Thus, I have a form and an email link.

You can change that to your own design. It was very easy for me.
Jul 20 '05 #20
In article <3RlRa.80008$Ph3.8779@sccrnsc04>, br***@wfcr.org.invalid-
remove-this-part says...
Mr. Clean wrote:
I use js to write the mailto: link. In the noscript element, I
put my email address in as English words, i.e., username (at)
example.com


Download RedSquirrel's SecurEmailL script. Works very nicely on PHP
hosts. http://rsscripts.tripod.com/scripts/securemaill.htm


I already have a contact form. But some users prefer email, since
their email clients allow them to keep a copy for themselves, or cc:
it to someone else. Thus, I have a form and an email link.

I have a contact form which allows the sender to send a copy to him\her
self - problem solved.

--
**************************************
The Eldritch Dark:
Dedicated to Clark Ashton Smith
http://www.eldritchdark.com/
Jul 20 '05 #21
On Fri, 18 Jul 2003, Lauri Raittila wrote:
In article <MP************************@news.slingshot.co.nz >, Boyd
Pearson wrote:
I have a contact form which allows the sender to send a copy to him\her
self - problem solved.


And when someone notices and uses it for spamming?


Then they get locked out. There's not much of a point since often such forms
have a destination address that the client user (or spammer) cannot change -
so: 1) He doesn't even know where it's going, 2) It will usually only go
to a single person; not a "productive" activity for a spammer to cater to, and
3) The field names often differ from one site to another, thus requiring the
"personal attention" of the spammer to customize their feeding of the form.
Spammers don't really have that kind of time - they have to hit many in as
little time as possible before their ISP kills their account.

In my form (a PHP script), I actually check to see that the return address
given is e-mailable by (here's what I will disclose; perhaps not complete):
- Making certain that the part before "@" is not null
- That there is a SINGLE "@"
- That the domain/hostname which follows "@" is deliverable by:
- Having a valid "A" or "AAAA" or "A6" IP address, or by
- Having a valid MX record (which can have its addresses tested
as well - for the really paranoid).
If it's a CNAME, then I test the canonical name for these DNS records.

Of course, there may be other things to check, such as: "Are all characters in
the mailbox name valid per RFC-2821 and -2822 for mailboxes?"

There are some issues with using the PHP library to do the DNS lookups: Most
functions will only return ONE value, even if multiple records of a given type
exist. That's not a problem for the address-type records, but it may be an
issue for the MX record(s) if one then subsequently tests them for addresses.
For the sufficiently skilled, there may be a way, especially if the DNS serves
records "round-robin" to acquire the entire RR-set for a given DNS query type,
but that may be overkill....

I also spamtrap the "/cgi-bin/formmail.cgi" and ".pl" URIs. There are just too
many "kiddie scripts" out there that look for those.
Jul 20 '05 #22
Mr. Clean wrote:
I use js to write the mailto: link. In the noscript element, I
put my email address in as English words, i.e., username (at)
example.com


Download RedSquirrel's SecurEmailL script. Works very nicely on PHP
hosts. http://rsscripts.tripod.com/scripts/securemaill.htm


I already have a contact form. But some users prefer email, since
their email clients allow them to keep a copy for themselves, or cc:
it to someone else. Thus, I have a form and an email link.

--
Brian
follow the directions in my address to email me

Jul 20 '05 #23
In article <Pi*******************************@kd6lvw.ampr.org >, D. Stussy
wrote:
On Fri, 18 Jul 2003, Lauri Raittila wrote:
In article <MP************************@news.slingshot.co.nz >, Boyd
Pearson wrote:
I have a contact form which allows the sender to send a copy to him\her ^^^^^^^ self - problem solved.
^^^^ And when someone notices and uses it for spamming?


Then they get locked out. There's not much of a point since often such forms
have a destination address that the client user (or spammer) cannot change -
so: 1) He doesn't even know where it's going,


But this case there was possibility for CC, and you can't send CC to self
by form whitout it knowing your address. That was what made me wonder. I
don't know if he has security problem there, but there might.
--
Lauri Raittila <http://www.iki.fi/lr> <http://www.iki.fi/zwak/fonts>
Saapi lähettää meiliä, jos aihe ei liity ryhmään, tai on yksityinen
tjsp., mutta älä lähetä samaa viestiä meilitse ja ryhmään.

Jul 20 '05 #24
Mr. Clean wrote:
I use js to write the mailto: link. In the noscript element, I
put my email address in as English words, i.e., username (at)
example.com

Download RedSquirrel's SecurEmailL script. Works very nicely on PHP
hosts. http://rsscripts.tripod.com/scripts/securemaill.htm


I already have a contact form. But some users prefer email, since
their email clients allow them to keep a copy for themselves, or cc:
it to someone else. Thus, I have a form and an email link.


You can change that to your own design. It was very easy for me.


I am not being clear, I guess. Or I have misunderstood. Either way,
my apologies.

Just to be clear: I already have a contact form. I can change it any
way I want: add or remove fields at will.

I provode a contact form *and* an email address, because some users
prefer (or need) one, and some prefer another. Since I provide an
email address, I try to do something to slow down the spambot.

--
Brian
follow the directions in my address to email me

Jul 20 '05 #25
On Fri, 18 Jul 2003, Lauri Raittila wrote:
In article <Pi*******************************@kd6lvw.ampr.org >, D. Stussy wrote:
On Fri, 18 Jul 2003, Lauri Raittila wrote:
In article <MP************************@news.slingshot.co.nz >, Boyd
Pearson wrote:

> I have a contact form which allows the sender to send a copy to him\her ^^^^^^^ > self - problem solved. ^^^^ And when someone notices and uses it for spamming?


Then they get locked out. There's not much of a point since often such forms
have a destination address that the client user (or spammer) cannot change -
so: 1) He doesn't even know where it's going,


But this case there was possibility for CC, and you can't send CC to self
by form whitout it knowing your address. That was what made me wonder. I
don't know if he has security problem there, but there might.


Then that form and its CGI are poorly designed. A design that would overcome
this would effectively submit both addresses as BCC'ed (or at least the
user/owner of the page who is to be hidden).
Jul 20 '05 #26

This discussion thread is closed

Replies have been disabled for this discussion.

Similar topics

8 posts views Thread by jcnews | last post: by
reply views Thread by Jason | last post: by
7 posts views Thread by Maxim Shemanarev | last post: by
3 posts views Thread by Michel | last post: by
6 posts views Thread by bissatch | last post: by
22 posts views Thread by Martin Eyles | last post: by
2 posts views Thread by adam.timberlake | last post: by
11 posts views Thread by blickensdoerfer | last post: by
reply views Thread by zhoujie | last post: by
reply views Thread by kglaser89 | last post: by
By using this site, you agree to our Privacy Policy and Terms of Use.