Spam filtering on SMTP level

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

Spam filtering on SMTP level

Bernd Wurst
Hello.

We have a mixed setup, some domains are handles via aliases and .courier
files (the qmail way) and others are going to a dedicated account via
hosteddomains and are handled there with a self written delivery agent.

We love the simplicity and flexibility that .courier files give to
users, so we would like to keep that for users that can handle it.

Otoh, we recently get into trouble when spam filtering happens on local
delivery. This leads to bounces and such, so we would like to offer some
kind of smtp level spam filtering. We have a lot of advanced users that
like to have their mail unfiltered or use a special configuration. So we
need a solution where we can whitelist which recipients to filter and
pass through all others.

Is there an existing filter setup that can be adapted for this need?

Sincerely,
Bernd



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
courier-users mailing list
[hidden email]
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Spam filtering on SMTP level

Bowie Bailey
On 2/28/2017 7:22 AM, Bernd Wurst wrote:

> Hello.
>
> We have a mixed setup, some domains are handles via aliases and .courier
> files (the qmail way) and others are going to a dedicated account via
> hosteddomains and are handled there with a self written delivery agent.
>
> We love the simplicity and flexibility that .courier files give to
> users, so we would like to keep that for users that can handle it.
>
> Otoh, we recently get into trouble when spam filtering happens on local
> delivery. This leads to bounces and such, so we would like to offer some
> kind of smtp level spam filtering. We have a lot of advanced users that
> like to have their mail unfiltered or use a special configuration. So we
> need a solution where we can whitelist which recipients to filter and
> pass through all others.
>
> Is there an existing filter setup that can be adapted for this need?

You can't bounce messages once you've accepted them.  The only sane
options are quarantine, tag-and-deliver, or delete.

Take a look at courier-pythonfilter it has all sorts of options for
ClamAV, SpamAssassin, greylisting, whitelisting, quarantine, etc.

https://pypi.python.org/pypi/courier-pythonfilter

--
Bowie

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
courier-users mailing list
[hidden email]
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Reply | Threaded
Open this post in threaded view
|

Re: Spam filtering on SMTP level

Lindsay Haisley-3
On Tue, 2017-02-28 at 09:06 -0500, Bowie Bailey wrote:
> Take a look at courier-pythonfilter it has all sorts of options for 
> ClamAV, SpamAssassin, greylisting, whitelisting, quarantine, etc.
>
> https://pypi.python.org/pypi/courier-pythonfilter

I'll second that. If you're at all conversant in python, Gordon Messmer
has done an excellent job with this package, its API, and with the
documentation for it. I've written a couple of modules for it and
expanded on others, and it's one of the reasons I'll probably stick
with Courier as long as I run mail servers :) 

--
Lindsay Haisley       | "UNIX is user-friendly, it just
FMP Computer Services |       chooses its friends."
512-259-1190          |          -- Andreas Bogk
http://www.fmp.com    |


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
courier-users mailing list
[hidden email]
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Reply | Threaded
Open this post in threaded view
|

Re: Spam filtering on SMTP level

Bowie Bailey
On 2/28/2017 11:27 AM, Lindsay Haisley wrote:

> On Tue, 2017-02-28 at 09:06 -0500, Bowie Bailey wrote:
>> Take a look at courier-pythonfilter it has all sorts of options for
>> ClamAV, SpamAssassin, greylisting, whitelisting, quarantine, etc.
>>
>> https://pypi.python.org/pypi/courier-pythonfilter
> I'll second that. If you're at all conversant in python, Gordon Messmer
> has done an excellent job with this package, its API, and with the
> documentation for it. I've written a couple of modules for it and
> expanded on others, and it's one of the reasons I'll probably stick
> with Courier as long as I run mail servers :)

And even if you're not conversant in Python, you can easily set up the
included modules just by adjusting the config files.

--
Bowie

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
courier-users mailing list
[hidden email]
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Reply | Threaded
Open this post in threaded view
|

Re: Spam filtering on SMTP level

Gordon Messmer-2
In reply to this post by Lindsay Haisley-3
On 02/28/2017 08:27 AM, Lindsay Haisley wrote:
> Gordon Messmer has done an excellent job with this package, its API, and with the documentation for it.


That's very kind.  I'll note that as much as possible, the pythonfilter
API is simply a direct reflection of the mechanisms used in Courier.


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
courier-users mailing list
[hidden email]
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Reply | Threaded
Open this post in threaded view
|

Re: Spam filtering on SMTP level

Bernd Wurst
Hello.

Am 28.02.2017 um 21:16 schrieb Gordon Messmer:
> On 02/28/2017 08:27 AM, Lindsay Haisley wrote:
>> Gordon Messmer has done an excellent job with this package, its API, and with the documentation for it.
> That's very kind.  I'll note that as much as possible, the pythonfilter
> API is simply a direct reflection of the mechanisms used in Courier.

Thank you all for the replies.

I have had a look on pythonfilter and we decided to cut down our needs
to what pythonfilter can do or what can be easily build with
pythonfilter. :)

I did a test setup and have a problem with clamav.
When I use "pyclamav" as stated in the docs, clamd is not used and I
have a breakdown (refuse all messages) when the running freshclam daemon
reloads the database. Restarting the pythonfilter fixes this but it's
annoying.
Also, pyclamd is unsupported since 9 years!

Looking in the code, pyclamd is also supported but with an old interface
and does not work with the current pyclamd-0.3.17.

I would like to write a patch but don't know how to send it in. Github
has https://github.com/szepeviktor/courier-pythonfilter but is this the
master copy?


- Bernd


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
courier-users mailing list
[hidden email]
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Spam filtering on SMTP level

Bernd Wurst
Hi again.

Sorry for the noise, I am wrong.


Am 03.03.2017 um 06:42 schrieb Bernd Wurst:
> Also, pyclamd is unsupported since 9 years!

This is about pyclamav, for sure.


> Looking in the code, pyclamd is also supported but with an old interface
> and does not work with the current pyclamd-0.3.17.

I'm sorry, that's not true. I have had another module named pyclamd
somewhere on my path that interferred with it but had nothing to do with
the desired one. :(

But I'd still like to know where to submit patches when We extend something.




------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
courier-users mailing list
[hidden email]
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Spam filtering on SMTP level

Alessandro Vesely
In reply to this post by Bernd Wurst
On Fri 03/Mar/2017 06:42:38 +0100 Bernd Wurst wrote:
> When I use "pyclamav" as stated in the docs, clamd is not used and I
> have a breakdown (refuse all messages) when the running freshclam daemon
> reloads the database. Restarting the pythonfilter fixes this but it's
> annoying.

If you're looking for alternatives, avfilter spawns a child while parent
reloads --which takes 19~20 secs these days, IME.
http://www.tana.it/sw/avfilter/

Ciao
Ale

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
courier-users mailing list
[hidden email]
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Reply | Threaded
Open this post in threaded view
|

Re: Spam filtering on SMTP level

Gordon Messmer-2
In reply to this post by Bernd Wurst
On 03/02/2017 09:58 PM, Bernd Wurst wrote:
> But I'd still like to know where to submit patches when We extend something.


You can email them to me directly, or you can fork the code on bitbucket
and send a pull request:

https://bitbucket.org/gordonmessmer/courier-pythonfilter


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, SlashDot.org! http://sdm.link/slashdot
_______________________________________________
courier-users mailing list
[hidden email]
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Reply | Threaded
Open this post in threaded view
|

Re: Spam filtering on SMTP level

Bernd Wurst
In reply to this post by Bernd Wurst
Again, thank you all for your replies.

We decided to go with pythonfilter, with some modifications.

When I look at my logs and my inbox, I still could imagine some
improvements for spam filtering apart from tweaking spamassassin. So I'd
like to know if some of you have written additional filters for
pythonfilter and would like to share them. Is there a repository for
such custom filters?



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
courier-users mailing list
[hidden email]
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Spam filtering on SMTP level

Jérôme Blion
Le 2017-03-20 10:36, Bernd Wurst a écrit :

> Again, thank you all for your replies.
>
> We decided to go with pythonfilter, with some modifications.
>
> When I look at my logs and my inbox, I still could imagine some
> improvements for spam filtering apart from tweaking spamassassin. So
> I'd
> like to know if some of you have written additional filters for
> pythonfilter and would like to share them. Is there a repository for
> such custom filters?

Hello Bernd,

I tunned the spamassassin and clamav filters and provided them directly
to Gordon. Don't hesitate to share your work with us, we will appreciate
it.

HTH.
Jérôme Blion.

------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
courier-users mailing list
[hidden email]
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users
Reply | Threaded
Open this post in threaded view
|

Re: Spam filtering on SMTP level

Bernd Wurst
Am 20.03.2017 um 15:12 schrieb Jérôme Blion:
> I tunned the spamassassin and clamav filters and provided them directly
> to Gordon. Don't hesitate to share your work with us, we will appreciate
> it.

I pushed my changes to the greylist filter in my github-repo
https://github.com/bwurst/courier-pythonfilter

Gordon has received a pull request.



------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
courier-users mailing list
[hidden email]
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Spam filtering on SMTP level

Gordon Messmer-2
On 03/21/2017 09:27 AM, Bernd Wurst wrote:
> I pushed my changes to the greylist filter in my github-repo
> https://github.com/bwurst/courier-pythonfilter
>
> Gordon has received a pull request.


I see the PR, yes, but the pythonfilter repo is on bitbucket:

https://bitbucket.org/gordonmessmer/courier-pythonfilter

Your PR was sent to szepeviktor.

Regarding the IPv6 code: The greylist module was originally written by
Mickael Marchand, and parses IPv4 addresses with a regex.  That's not
ideal.  Rather than perpetuate that code, I'd prefer to use a validating
library to get the addresses (especially with IPv6).  Python 3 has such
a library, and it's been ported to earlier releases, so that seems like
the obvious solution:

https://pypi.python.org/pypi/ipaddress

Regarding whitelisting networks: it's probably a good idea.  I'd like to
think about that a bit and see if there's any less complex way to do that.

I'll get these changes made shortly, but I want to clean them up a bit
first.


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot
_______________________________________________
courier-users mailing list
[hidden email]
Unsubscribe: https://lists.sourceforge.net/lists/listinfo/courier-users