[warning: 99% OT] does anything like this exist ?...

[Fred Pacquier]

First off, sorry for this message-in-a-bottle-like post... I haven't been
able to phrase my questions well enough to get a meaningful answer from
Google in my research. OTOH, it is standard flattery (but true) that this
group has a bunch of the nicest and most knowledgeable Usenet people
around, and I know for a fact that there are some pretty good spam-
related tools written in Python, so I thought I might get away with it
:-)

Yes, it's about spam.

I have a (very old, POP3) email address that's flooded with spam (about
500 msgs per 24 hrs, no thanks to the ISP...) but that I still need to
use for various reasons. Of course it's unmanageable without some sort of
bayesian spam filter.

When I work from home or office or my laptop, tools like Thunderbird or
Pegasus+K9 do the job adequately. But I also frequently need to access it
"on the road" with whatever comes in handy (Webmail etc.), and that
becomes a problem if I've been away for even a short period.

One possible solution I've been mulling over, and looking for, would be
some sort of selective, on-line filter/downloader. I have an always-on,
Linux box at home on a decent DSL line ; on this I could have a daemon to
frequently poll that POP3 address, pulling only the headers (or maybe the
first body KB or so) and running those through a bayesian filter. If a
message comes out "clean" it is left on the server (so I can access it
from wherever I am), if it doesn't it is downloaded to a local mbox and
removed from the server (like getmail.py does) so I can still dig out the
occasional false positive when I'm home...

Is there a ready-made tool that fills this need ? If it's in Python so
much the better, but actually I'll run anything else within my means
(even perl :-) if it works without having to tinker the code... Also,
it's possible that my single-minded approach is misled and there's a
better way to achieve that goal, so I'm ready to change my mind too :)

TIA - again, sorry for the noise,
fp

--
YAFAP : http://www.multimania.com/fredp/

[Cliff Wells]

On Tue, 2004-10-26 at 13:40 +0000, Fred Pacquier wrote:

I have a (very old, POP3) email address that's flooded with spam (about
500 msgs per 24 hrs, no thanks to the ISP...) but that I still need to
use for various reasons. Of course it's unmanageable without some sort of
bayesian spam filter. One possible solution I've been mulling over, and looking for, would be
some sort of selective, on-line filter/downloader. I have an always-on,
Linux box at home on a decent DSL line ; on this I could have a daemon to
frequently poll that POP3 address, pulling only the headers (or maybe the
first body KB or so) and running those through a bayesian filter. If a
message comes out "clean" it is left on the server (so I can access it
from wherever I am), if it doesn't it is downloaded to a local mbox and
removed from the server (like getmail.py does) so I can still dig out the
occasional false positive when I'm home...

Rather that doing that, you might consider simply setting up a local
IMAP server on your Linux box. Have a program (such as fetchmail) pull
down your POP3 email, filter it using procmail and feed it into the IMAP
server so that you can then access it from anywhere.

--
Cliff Wells <cl************ @comcast.net>

[Fred Pacquier]

Cliff Wells <cl************ @comcast.net> said :

Rather that doing that, you might consider simply setting up a local
IMAP server on your Linux box. Have a program (such as fetchmail)
pull down your POP3 email, filter it using procmail and feed it into
the IMAP server so that you can then access it from anywhere.

Thanks, Cliff. I realize this would be the "standard" way of going about
such things (didn't want to bloat the original post :-). However I tend to
think of it as a fallback solution if nothing else comes up, if only for
the following reasons :

* a lot of upfront work : several major packages (fetchmail, procmail,
imapd) I have no hands-on experience with, a lot of reading/learning and a
lot of little things to get just right in each... In my mind the solution
I'm after could be a much simpler affair, but then maybe I'm daydreaming :)

* one more potential security hole on my home machine : I tend to limit
open ports on that one to a bare minimum, my admin skills & available time
being quite limited...

* availability : my ISP doesn't filter spam but otherwise does a good job
of keeping its server up. Depending on my own setup means I could get shut
out when away from home because of a power failure, DSL downtime, a full
disk, or any of a myriad other domestic problems...

I've been paged about so many similar setups in the workplace that I'll
avoid them at home if I can... But then, sometimes you _do_ have to turn to
plan B :-)

--
YAFAP : http://www.multimania.com/fredp/

[Richie Hindle]

[Fred]

I could have a daemon to
frequently poll that POP3 address, pulling only the headers (or maybe the
first body KB or so) and running those through a bayesian filter. If a
message comes out "clean" it is left on the server (so I can access it
from wherever I am), if it doesn't it is downloaded to a local mbox and
removed from the server (like getmail.py does) so I can still dig out the
occasional false positive when I'm home...

Andrew Dalke contributed a program to the SpamBayes wiki that does
exactly this:

http://www.entrian.com/sbwiki/SpamBayesCuller
http://www.entrian.com/sbwiki/Recent...pamBayesCuller

"This program, sb_culler, uses SpamBayes to run a POP3 email culler. It
connects to my email servers every few minutes, downloads the emails,
classifies each one, and deletes the spam and viruses. (It makes a
local copy of the spam, just in case.)"

You'll need to download and install the SpamBayes source code as well,
from:
http://sourceforge.net/project/showf...ckage_id=58141

Take either the .tgz or the .zip, not the .exe (which installs a binary
application rather than the source).

--
Richie Hindle
ri****@entrian. com

[Maciej Dziardziel]

Fred Pacquier wrote:

Is there a ready-made tool that fills this need ?

I don't know any, but writing such is not a big problem - python has builtin
support for pop3 and mail processing (examples are in its documentation),
so just download mail, pipe it through spamassassin and eventually delete
from server.

--
Maciej "Fiedzia" Dziardziel (fiedzia (at) fiedzia (dot) prv (dot) pl)
www.fiedzia.prv.pl

My drinking team has a soccer problem.

[Andrew Dalke]

Richie Hindle wrote:

Andrew Dalke contributed a program to the SpamBayes wiki that does
exactly this:

http://www.entrian.com/sbwiki/SpamBayesCuller
http://www.entrian.com/sbwiki/Recent...pamBayesCuller You'll need to download and install the SpamBayes source code as well,

I got email about 6 months ago asking if it is okay to include my
sb_culler program with the SpamBayes source code. I haven't checked
to see if that actually happened.

As written it's designed for a programmer to use. You had to
edit code to change options.

I've added a few new features since then, like reloading the
good_emails file if there are changes. Want to add another to
whitelist based on domain, and to remove emails with a given
subject, like "Sprava nebola dorucena". Perhaps I'll update
the wiki afterwards.

Andrew
da***@dalkescie ntific.com

[Fred Pacquier]

Richie Hindle <ri****@entrian .com> said :

Andrew Dalke contributed a program to the SpamBayes wiki that does
exactly this:

"This program, sb_culler, uses SpamBayes to run a POP3 email culler.
It connects to my email servers every few minutes, downloads the emails,
classifies each one, and deletes the spam and viruses. (It makes a
local copy of the spam, just in case.)"

Wo-ow ! Talk about being spoiled... :-)
Of course I suspected I was not alone in wanting to do this, and that
someone had already done it. But to see it described so exactly (only
better and in fewer words), and done in python too, and using Spambayes, no
less... well, I hadn't dared expect that much. Sort of makes me regret not
asking up front for that flashy pink finish and the leather upholstery, too
:)

Actually I should feel ashamed : such a happy ending will undoubtedly
unleash a tsunami of rabid OT posters on this last peaceful corner of a
beleaguered Usenet, which will subsequently collapse in the next hundred
days, taking down with it the Internet and civilization as we know it. I'm
really sorry folks...

Meanwhile, thanks a lot for the pointer Richie !

--
YAFAP : http://www.multimania.com/fredp/

[Fred Pacquier]

Andrew Dalke <ad****@mindspr ing.com> said :

I got email about 6 months ago asking if it is okay to include my
sb_culler program with the SpamBayes source code. I haven't checked
to see if that actually happened.
I just did : it's been added to CVS on June 11, same code as on the Wiki,
but apparently not updated since, and not included in the contribs for
the 1.0 release.

I already have a couple of naive questions :

* is the documented change on line 348 enough to run the script with a
current version of Spambayes ?

* does use of sb_culler contribute to the training of the Spambayes db,
or does it assume that it is kept current independently (by means of
normal use by a mail client through the POP3 proxy for instance) ?
As written it's designed for a programmer to use. You had to
edit code to change options.
That's OK with me, as long as the code is in python :)
I've added a few new features since then, like reloading the
good_emails file if there are changes. Want to add another to
whitelist based on domain, and to remove emails with a given
subject, like "Sprava nebola dorucena". Perhaps I'll update
the wiki afterwards.

Well, if any of these new additions are available for outside use, I'd
sure appreciate a copy of the updated script...

TIA,
fp

--
YAFAP : http://www.multimania.com/fredp/

[Fred Pacquier]

Maciej Dziardziel <fi*****@fiedzi a.prv.pl> said :

Is there a ready-made tool that fills this need ?

I don't know any, but writing such is not a big problem - python has
builtin support for pop3 and mail processing (examples are in its
documentation), so just download mail, pipe it through spamassassin
and eventually delete from server.

I sort of knew this one would be forthcoming :)

I won't say I haven't been tempted - like everyone else here I've written
my own 'experimental' POP3 checker using the python standard lib, and
several more from the 'yet another' category... but I know my limits.

Also, over the years I've had my share of "unique/original" (heh) ideas,
spending ages to implement them, only to find out afterwards that someone
had already done it, and generally much better... just like Andrew Dalke
with sb_culler in this case.

I'm not looking for a pet project here, just to solve a problem. If I can
do it in a few hours of tinkering with a ready-made answer I certainly
won't try to roll my own. If I were a student, or in that mythical state
between retirement and brain-rot, maybe, but for now it's definitely Plan C
(or D :-)

--
YAFAP : http://www.multimania.com/fredp/

[Andrew MacIntyre]

On Tue, 26 Oct 2004, Cliff Wells wrote:

Rather that doing that, you might consider simply setting up a local
IMAP server on your Linux box. Have a program (such as fetchmail) pull
down your POP3 email, filter it using procmail and feed it into the IMAP
server so that you can then access it from anywhere.

I've been using Charles Cazabon's getmail (pure Python) for quite some
time in place of fetchmail.

-------------------------------------------------------------------------
Andrew I MacIntyre "These thoughts are mine alone..."
E-mail: an*****@bullsey e.apana.org.au (pref) | Snail: PO Box 370
an*****@pcug.or g.au (alt) | Belconnen ACT 2616
Web: http://www.andymac.org/ | Australia