gmane.mail.postfix.policyd-weight

New Mailinglist Home

Robert Felber — 2010-03-25T20:46:19

Hello people,


I have to move the ML to a new host because the old
one will soon no longer be available.

For everyone who wants to keep track of
changes or possibly new development the
new location is at

https://listen.jpberlin.de/mailman/listinfo/policyd-weight-users

Re: policyd-weight as a tarpit

Robert Felber — 2010-03-22T07:42:28

This also eats your resources as your MTA has to spawn new smtpd(8)
processes for new clients (the other smtpd-ones wait for policyd-weight's
answer).

This could also enable a DoS (one can circumvent the cache-rejects by
changing the sender-envelope and thus trigger your sleep)

Also there is smtpd_error_sleep_time (default 1s)

See man 8 smtpd | less +/TARPIT

policyd-weight as a tarpit

Gregor Glashüttner — 2010-03-21T22:20:48

Hi!

I´m using policyd-weight and think it does a great job in rejecting
spam. I would like to do a little more and eat up the spammers
resources by tarpitting them. My system doesn´t have too much traffic
so i simply added "sleep(5);" in policyd-weight´s section "parse and
store results, do some cleanup, return results", right before the two
lines "return($EREJECTMSG.$RHSBLMSG.$RELAYMSG.$DYN_DNS_MSG);". This
should eat 5 secs of the spammers time, right? In /var/log/mail.log i
see "delay 5s" after the reject message (used to be 0-1s before).

Of course that could be improved, by calculating the sleep-time
depending on score, running processes compared to $MAX_PROC, a
config-file configurable $MAX_TARPIT_TIME etc.

Regards
Gregor

Re: policyd-weight: blocked because of previous errors

Robert Felber — 2009-12-23T09:35:47

Loghost. Hmhm.



In the cache entry for blocked entries is only the timestamp for the last
attempt. This should be improved, yes.



One reason could be that the first attempt was longer than 8 days.

The blocked: 82.220.17.226 1 0 1261408171 indicates that it has been seen before because
policyd-weight decremented the NTTL (config setting, default 1) counter.

It then depends on how large the cache is, or how often it has to throw out
some of the oldest entries.

Another possibility could be, that the log message(s) simply got lost. In case
of UDP-loses no one can tell what happened.

(maybe by some luck there is some evidence in /tmp/.policyd-weight/polw-emergency.log,
resp. $LOCKPATH/polw-emergency.log. Would only be there if the syslog daemon was not
available. but not, if the syslog daemon have had problems with logging)

Re: policyd-weight: blocked because of previous errors

Helga Mayer — 2009-12-23T08:56:21

yes

What I expected was something like

weighted check:  NOT_IN_SBL_XBL_SPAMHAUS=-1.5 NOT_IN_SPAMCOP=-1.5 
NOT_IN_BL_NJABL=-1.5 HELO_IP_IN_CL_SUBNET=-1.2 (check from: .hotmail. - 
helo: .some.domain - helo-domain: .domain.) 
FROM/MX_MATCHES_NOT_HELO(DOMAIN)=1 IN_PM_RFCI=3.2 IN_ABUSE_RFCI=3.2; 
<client=xxx.xxx.xxx.xxx> <helo=helo.some.domain> 
<from=SENDER-PMXAeEypde+6dnWWwvwaGw< at >public.gmane.org> <to=RECIPIENT-t7PsS/mkLKX22yMXT8v7CQ< at >public.gmane.org>; rate: 
1.7

but there's no entry like this for 82.220.17.226
I apologize, this was my mistake. I checked the logs on a common loghost.
The loghost is 2 second back.
by 2 machines with identical setup. 2 smtp servers with the same priority
in the MX Record. Each is running policyd-weight locally.
On the particular host the time of the cache entry is exactly the time
of the first reject. So the problem remaining is:
why is there no reason given in the logs for the reject and for rate:1,
as seen in the cache entry.

Regards
Helga Mayer

Re: policyd-weight: blocked because of previous errors

Robert Felber — 2009-12-23T08:03:33

[bz]grep 82.220.17.226 /var/log/...log...

results only in this snippet?



+ 2 seconds indeed sounds strange but could be explained if the
log is done in GMT (which would make it then a retry after 59:57 minutes).


Is the policy service used by many machines or _only_ by localhost?


 

Does lead to a reject, yes.

SENDER
% host telecontrol.ch
telecontrol.ch has address 93.88.240.108
telecontrol.ch mail is handled by 5 mta-gw.infomaniak.ch.

% host mta-gw.infomaniak.ch
mta-gw.infomaniak.ch has address 84.16.68.126
mta-gw.infomaniak.ch has address 84.16.68.125


HELO
% host smtp.telecontrol.ch
smtp.telecontrol.ch is an alias for mail.infomaniak.ch.
mail.infomaniak.ch has address 84.16.68.123
mail.infomaniak.ch has address 84.16.68.124



CLIENT
% host mail-telecontrol.customer.solnet.ch
mail-telecontrol.customer.solnet.ch has address 82.220.17.226


The client is in no relation (naming or subnet-wise) to sender or helo.
Would the sender use a correct HELO, he wouldn't have this problem.

policyd-weight: blocked because of previous errors

Helga Mayer — 2009-12-22T12:45:56

Hello list,

I have a problem with rejects due to cache entries.
We use policyd-weight-0.1.14-beta-17.

This is the message found in the logfile:

Dec 21 16:09:28 smtp2 postfix/smtpd[16364]: connect from 
mail-telecontrol.customer.solnet.ch[82.220.17.226]
Dec 21 16:09:29 smtp2 postfix/policyd-weight[30193]: decided action=550 
temporarily blocked because of previous errors - retrying too fast. 
penalty: 30 seconds x 0 retries.; <client=82.220.17.226> 
<helo=smtp.telecontrol.ch> <from=$SENDER-guzGq5xpyfxj/79lGKjTxQ< at >public.gmane.org> 
<to=$RECIPIENT-t7PsS/mkLKX22yMXT8v7CQ< at >public.gmane.org>; delay: 0s
Dec 21 16:09:29 smtp2 postfix/smtpd[16364]: NOQUEUE: reject: RCPT from 
mail-telecontrol.customer.solnet.ch[82.220.17.226]: 550 5.7.1 
<$RECIPIENT-t7PsS/mkLKX22yMXT8v7CQ< at >public.gmane.org>: Recipient address rejected: temporarily 
blocked 
because of previous errors - retrying too fast. penalty: 30 seconds x 0 
retries.; from=<$SENDER-guzGq5xpyfxj/79lGKjTxQ< at >public.gmane.org> to=<$RECIPIENT-t7PsS/mkLKX22yMXT8v7CQ< at >public.gmane.org> 
proto=ESMTP helo=<smtp.telecontrol.ch>


There are no other log entries for 82.220.17.226 during the last 8 days.
The cache entry is:
policyd-weight -s|grep 82.220.17.226
blocked: 82.220.17.226 1 0 1261408171
1261408171 (UNIX) is the date of the first (and only) reject + 2 seconds :
1261408171 = Mon, 21 Dec 2009 15:09:31 GMT

which is IMO contradictious: The mail has been rejected due to
cache entries but hasn't been in the cache before.

As a workaround we whitelisted the particular IP.
The headers of a mail received from this server are:

Received: from smtp.telecontrol.ch (mail-telecontrol.customer.solnet.ch
     [82.220.17.226])
     by smtp2.rz.uni-hohenheim.de (Postfix) with ESMTP
     for <$RECIPIENT-t7PsS/mkLKX22yMXT8v7CQ< at >public.gmane.org>; Tue,
     22 Dec 2009 12:23:13 +0100 (CET)
Received: from PRISM.telecontrol.local ([192.168.30.11]) by
  PRISM.telecontrol.local ([192.168.30.11]) with mapi; Tue, 22 Dec 2009
  12:23:18 +0100

Regards
Helga Mayer

____________________________________________________________
Policyd-weight Mailinglist - http://www.policyd-weight.org/

Re: update submission process

Morgan Weetman — 2009-11-01T21:23:07

Thanks, I've removed my version from SF

____________________________________________________________
Policyd-weight Mailinglist - http://www.policyd-weight.org/

Re: update submission process

Robert Felber — 2009-10-31T08:58:21

Sure.
I am reading a bit into

http://sourceforge.net/apps/trac/sourceforge/wiki/Git
and 
http://git-scm.com/documentation


 

I'll try to reproduce it.

Re: update submission process

Morgan Weetman — 2009-10-31T03:49:58

Hi Rob,

I'm happy to leave it any format you like, these are the sort of issues
I was trying to avoid. Is it possible that we can have an approval
process defined ? 

In regards to the spawn_cache issue, maybe it was just my
implementation. Can anybody else verify that polw cache fails to start
after a power failure or unclean shutdown ?

cheers

On Fri, 2009-10-30 at 10:38 +0100, Robert Felber wrote:

____________________________________________________________
Policyd-weight Mailinglist - http://www.policyd-weight.org/

Re: update submission process

Robert Felber — 2009-10-30T09:38:18


First,

 is it okay to keep line-breaks as is, resp. to format it for 80-char
displays? It's otherwise a hard reading. Also, as suggestion: format the
code at a black-bg,white-fg-terminal without syntax-highlighting. That's
why I kept those long 

#
# #############################################################################
#

lines.



Second,

 code:

I don't see the reason for the massive change.

If no cache proecess exists, every process is allowed to create one.
In the start-phase of policyd-weight or in case of a death of a cache
this could mean a couple of hundreds cache-process trying to start up. If you
let them sit around by sleeping 2 seconds, you will cause a fork problem 
(imagine 20 or 30 smtp requests per second).
They have to return undef, and non verbose instantly if they detect that other
caches are ahead.

As soon as a cache is successfully forked, it does delete the lockfile.

A stale socket will always be deleted.


In order to make sure that there is no stale lock-file at the beginning of
the world we could remove an existing lock-dir before (like):

line: 1152
+ # a cache-lock-file shouldn't be there yet
+ if( -d $LOCKPATH.'/cache_lock )
+ {
+    unlink $LOCKPATH.'/cache_lock;
+ }
cache_query("start"); # pre-launch cache


Rationale: the master/child have to control when to start caches. As such
it is their, in this case the masters, responsibility to make precautions
for a clean environment.

If a cache crashes between 'mkdir lock' and 'rmdir lock' then it
has to be logged.


Also, for such things that concern security and robustness, I'd suggest that
we talk first about it. I haven't included it on policyd-weight.org because
of the 2-seconds-of-vague-sleep, which really shouldn't be there.

We should also start to sign the changes made in changes.txt.
Or maybe use the sourceforge SVN.

Re: update submission process

Robert Felber — 2009-10-29T06:37:55

You can release it on sourceforge and I'll include it on policyd-weight.org.

update submission process

Morgan Weetman — 2009-10-29T01:20:40

Hi,

  I wasn't sure what the process was to submit updates to polw, if you
could please let me know.

thanks

____________________________________________________________
Policyd-weight Mailinglist - http://www.policyd-weight.org/