gmane.mail.spam.spamassassin.general

sa-learn expire not working. Keeps complaining about delta atime

rtsit — 2012-05-26T04:04:42

I've got bayes running on a per-user database.  When I run:

sa-learn -D -u SOMEUSER --force-expire

It goes through and tries to expire. It decides it wants to remove 34399
tokens, but it can't do it due to an invalid delta atime. I'm confused.

This particular user has the following values in the bayes_vars MySQL table:
spam_count: 1440
ham_count: 652
token_count: 146899

Since it's over the threshold of 112500 I figured that it would expire the
extra tokens. 

INSTEAD, it complains about delta not having a good atime.

I'm watching this MySQL DB getting bigger and bigger, and expecting it's
eventually going to grow exponentially large unless the expire functions
work.

I'm currently at 227 megs for the .MYD file and 428 megs for the .MYI file.

The last time I had an issue, the database was in excess of 4 gigs.

Here's the debug output. If someone can give me a hint on how to give a
proper delta atime, I'd be really thankful.

May 25 20:55:12.219 [8792] dbg: bayes: using username: SOMEUSER< at >example.com
Ma

Re: X-Spam-Checker-Version doesn't reflect what sa-update we are at

Duane Hill — 2012-05-26T02:48:12

On Friday, May 25, 2012 at 21:26:45 UTC, jidanni< at >jidanni.org confabulated:


The  version header shows the version of SA you are running. sa-update
doesn't affect that.

X-Spam-Checker-Version doesn't reflect what sa-update we are at

2012-05-25T21:26:45

Isn't it bad that X-Spam-Checker-Version doesn't report what sa-update
we are up to so far, and that there is no additional other variable that
we can toggle on in reports to do that.

Re: Suddenly getting lots of false positives.

corpus.defero — 2012-05-24T16:09:59

The answer is at the botton:

40.152.71.64.list.dnswl.org. 43200 IN   A       127.0.6.3
;; Received 61 bytes from 208.67.172.131#53(c.ns.dnswl.org) in 76 ms

So, according to c.ns.dnswl.org it's a hit.

And if we do:

dig +short < at >208.67.172.131 40.152.71.64.list.dnswl.org
127.0.6.3

It appears to be a hit.

Re: Suddenly getting lots of false positives.

Benny Pedersen — 2012-05-24T16:14:57

Den 2012-05-24 17:22, Jeremy Morton skrev:

this is working as designed, no refused or errors, if its not working 
again then report it as so, with a +trace, report the last ns that fails 
if it do

syswrite() to parent failed:

Andy Jezierski — 2012-05-24T15:28:18

Just installed a new instance of spamassassin to offload some of the spam 
processing from our main server. Occaisionally the following message will 
appear and I'll have to restart spamd in order to get things going again.

syswrite() to parent failed: Broken pipe at 
/usr/local/lib/perl5/site_perl/5.14.2/Mail/SpamAssassin/SpamdForkScaling.pm 
line 579.

FreeBSD 8.2
perl 5.14.2_2
SA 3.3.2_6

Any ideas?

Thanks
Andy

Re: Suddenly getting lots of false positives.

Jeremy Morton — 2012-05-24T15:22:33

Not sure.  I get this:

http://pastebin.com/0U3WrgSS

Re: Suddenly getting lots of false positives.

Benny Pedersen — 2012-05-24T15:12:55

Den 2012-05-24 17:03, Jeremy Morton skrev:

where it timeout or rejected ?, where in the dns chain is it failing ?

Re: Suddenly getting lots of false positives.

Jeremy Morton — 2012-05-24T15:03:48

Nope, but it doesn't actually give an answer section as part of its output.

Re: Suddenly getting lots of false positives.

Benny Pedersen — 2012-05-24T15:06:20

Den 2012-05-24 16:41, Jeremy Morton skrev:


dig +trace 40.152.71.64.list.dnswl.org

refused ?

Re: Suddenly getting lots of false positives.

Benny Pedersen — 2012-05-24T15:01:54

Den 2012-05-24 16:06, Kevin A. McGrail skrev:


surbl.org is one of the problematic dns servers for me, sent a email 
about it to surbl, got nothing in return


local dns server is good aslong as remote servers dont reject querys 
from dynamic ips [1]

1: dig +trace surbl.org
2: dig +tcp +norecurse < at >ns100.surbl.org surbl.org any
3: dig +notcp +norecurse < at >ns100.surbl.org surbl.org any

none of them should be rejected

[1] dynamic in sense of ips is hard to know if its static

Re: Suddenly getting lots of false positives.

Jeremy Morton — 2012-05-24T14:41:01

I actually get:
Host 40.152.71.64.list.dnswl.org not found: 5(REFUSED)

Re: Suddenly getting lots of false positives.

2012-05-24T14:30:37

Yup, that could do it.  Icky.  

Jeremy: You could manually check if you're getting the wrong DNS results by
running:

$ host 26.13.94.59.list.dnswl.org
Host 26.13.94.59.list.dnswl.org not found: 3(NXDOMAIN)

(IP address reversed, then .list.dnswl.org.)

If an IP address is listed (as that one should not be), you'll see
something like:

$ host 40.152.71.64.list.dnswl.org
40.152.71.64.list.dnswl.org has address 127.0.6.3


I did?  Are you thinking of
https://wiki.apache.org/spamassassin/CachingNameserver ?  I didn't write
it.


Yup.

Re: Suddenly getting lots of false positives.

2012-05-24T14:20:56

SPF_SOFTFAIL kind of sucks:
http://ruleqa.spamassassin.org/?daterev=20120519-r1340375-n&rule=%2Fspf

  MSECS    SPAM%     HAM%     S/O    RANK   SCORE  NAME   WHO/AGE
      0   3.2640  27.9430   0.105    0.67    0.00  SPF_PASS  
      0   6.3320   0.6518   0.907    0.58    0.00  SPF_SOFTFAIL  
      0   4.0263   1.1272   0.781    0.50    0.00  SPF_NEUTRAL  
      0        0        0   0.500    0.50    0.00  SPF_NONE  
      0   1.7415   1.6254   0.517    0.39    0.00  SPF_FAIL  

SPF_SOFTFAIL hits 6.3% of spam and 0.7% of ham, which is a pretty terrible
ratio, which gives it a rank of 0.58, where 1 is best (RCVD_IN_DNSWL_HI, in
fact), and 0 is worst.  A rank of 0.58 sucks.

Therefore rejecting on it at your MTA is a bad idea.  But it's your MTA.
I've done lots of things with my MTA on purpose that were a bad idea.


I don't think they did.


Generally because people configure their SPF records badly.  SOFTFAIL
*means* the sending domain isn't certain they have all their legit sending
IPs listed.  So based o

Re: Suddenly getting lots of false positives.

Kevin A. McGrail — 2012-05-24T14:06:30

Normally, I blame a DNS server.  See pages like this for more information:

http://www.surbl.org/faqs#dnsproxy

Darxus, you wrote a good wiki about using other DNS servers, etc. somewhere I thought about but I can't find it.

In general, I recommend running your own caching nameserver.

Regards,
KAM

Re: Suddenly getting lots of false positives.

2012-05-24T14:02:00

I don't think this was ever actually listed by dnswl.org.  I have
archives back to last June, which don't show it, and in the dnswl.org
admin interface when a listing is removed it generally deactivated not
deleted - and there is nothing there.

That leaves interesting possibilities.  I'd start by running this email
through spamassassin again to see if it repeatably says this IP is listed
by dnswl.  SpamAssassin could be doing something wrong, a DNS server
somewhere could be doing something wrong.... 

And it might be useful to provide more examples.  Just IPs might be best.
And generally we prefer you provide spams via pastebin instead of including
them in emails to this list.

Re: Suddenly getting lots of false positives.

2012-05-24T13:54:05

No.

I'm a (basically inactive) dnswl.org admin.  

Anybody can request to be added to the list, but all changes get looked
over pretty thoroughly by a human, using lots of available data.  


Listing on dnswl.org does not involve payment, it is not a 'pay to spam'
whitelist.

Re: Suddenly getting lots of false positives.

Benny Pedersen — 2012-05-24T11:40:36

Den 2012-05-24 11:37, corpus.defero skrev:


as long users can report spamming ips aswell as get listed for not 
sending spam at all, its fine with me that some use it, for my self its 
a way to know if i have users sending spam aswell

Re: Suddenly getting lots of false positives.

Benny Pedersen — 2012-05-24T11:36:19

Den 2012-05-24 11:14, Jeremy Morton skrev:


reject spf_softfail in mta, or report to http://www.dnswl.org/ (why did 
thay list a dynamic ip ?)

if sender is legit why is it softfailing ?

Re: Suddenly getting lots of false positives.

corpus.defero — 2012-05-24T10:43:24

In later rulesets (forget when they added it) it looks something like
this:

ifplugin Mail::SpamAssassin::Plugin::DNSEval
header RCVD_IN_BRBL_LASTEXT
eval:check_rbl('brbl-lastexternal','bb.barracudacentral.org')
tflags RCVD_IN_BRBL_LASTEXT   net
endif


And tends to live in 72_active.cf


Grep for it with:
grep -Hl -r "RCVD_IN_BRBL_LASTEXT" /usr/share/spamassassin/*
or
grep -Hl -r "RCVD_IN_BRBL_LASTEXT" /*
if you get stuck (it's slow this way, but if you don't know where your
rules are, this will tell you if it's there or not)

If it's not there just add it to your local.cf file with something like
this:

header BARRACUDA_BL          eval:check_rbl('Barracuda',
'b.barracudacentral.org.')
describe BARRACUDA_BL        listed by BARRACUDA
tflags BARRACUDA_BL          net
score  BARRACUDA_BL        4.5

It's also worth adding that taking out the Spamhaus WHITELIST is worth
doing - it's rubbish and wastes a DNS lookup:

score DKIMDOMAIN_IN_DWL 0

On the subject of Spamhaus, if you are using big name resolvers (li

Re: Suddenly getting lots of false positives.

Niamh Holding — 2012-05-24T10:31:31

Hello Jeremy,

Thursday, May 24, 2012, 11:11:22 AM, you wrote:

JM> Where would the rules for these blocklists be, so I can check my rules
JM> files to see whether they're there?

Mine are in  /var/lib/spamassassin/3.003002/updates_spamassassin_org