gmane.mail.spam.dcc

Re: DCC and SpamAssassin

Vernon Schryver — 2012-01-12T15:29:56



DCC is still what it has always been, a mechanism for detecting and
somewhat quantifying the 'bulkness' mail.  Mail messages that have
hit some sort of a spam trap including the DCC.pm dcc_learn_score
mechanism has DCC target counts of "many" and should be considered
"very bulky."  The DCC target count of "many" is in fact 16777200
or the largest target count possible the DCC server databases,
client-server protocol, and server-to-server flooding protocol,
(It's not a coincidence that 16777200 is is approximately 0xffffff.)

The messages that contribute the pink areas in
http://www.rhyolite.com/dcc/graphs/?BIG=1
have target counts of "many" because they have hit some sort of
spam trap.



I recommend running dccifd (or dccm), setting the DCC thresholds
in /var/dcc/dcc_conf, and letting SpamAssassin (if you use SpamAssassin)
look for the string "bulk" in X-DCC headers that SpamAssassin gets
from dccifd or finds already present.

However, DCC.pm has always used the values of dcc_body_max,
dcc_fuz1_max, and

Re: DCC and SpamAssassin

Matus UHLAR - fantomas — 2012-01-12T13:34:29

Sorry for not getting to this e-mail sooner...

On 10.11.11 02:07, Vernon Schryver wrote:

So, is DCC no more a system for detecting bulkiness, but a 
spamminess of a message?


I was wondering, if I can change the MANY value for reporting and 
threshold for checking, outside of a DCC client (e.g. DCC.pm)

DCC version 1.3.141/2.3.141 released

Vernon Schryver — 2012-01-10T22:06:16

Version 1.3.141 of the DCC source is in
http://www.dcc-servers.net/dcc/source/dcc.tar.Z  and
http://www.rhyolite.com/dcc/source/dcc.tar.Z

Commercial version 2.3.141 of the DCC Reputation code is in the usual place.

The CHANGES file in
http://www.dcc-servers.net/dcc/dcc-tree/CHANGES and
http://www.rhyolite.com/dcc/dcc-tree/CHANGES and
starts with
    Fix "MTA-last" in dcc man page as suggested by Bram Grietens.
    Fix no_forced-discard typo reported by Bram Grietens.
    Fix dccm to honor `hackmc -R` and discard relay attacks.
    misc/DCC.pm, which is generated from misc/DCC.pm.in, is now very
similar to what will probably be in SpamAssassin 3.4.
    Fix problems finding native milter library for dccm pointed out by
Kevin A. McGrail.
    Improve documentation or help output from the nagios plugin,
/var/dcc/libexec/dcc-nagios
    Fix bug in misc/DCC.pm in dealing with mail that already has an
X-DCC header found and diagnosed by Herbert J. Skuhra.

dcc_learn_score probably would not help sites that have

DCC and SpamAssassin

Vernon Schryver — 2011-11-10T02:07:52

For the last 18 months, the DCC source has included a new version of
the SpamAssassin DCC plugin.  As far as I know, it is completely upward
compatible.  Besides a general clean up of the Perl source and a bunch
of changes that should make DCC.pm faster and more robust, it includes
a mechanism that reports message detected as spam by SpamAssassin to DCC
with target counts of 'many'.

Two or three months ago a DCC installation started using that mechanism
on its incoming 750K messages/day.  The effect is to increase DCC
target counts for messages considered spam by SpamAssassin to 'many'.
That should have little effect on DCC installations that set DCC
thresholds to values below 1000.  750K messages/day is too few to
support certain conclusions, but judging from the DCC server graphs,
it has noticable effects on the majority of other DCC installatios
that use the DCC.pm default synonym of 9999999 for 'many'.

That increased target count of 'many' can not only help other DCC
installations, but also sites using

Re: open(/var/dcc/map): Permission denied

Vernon Schryver — 2011-09-23T14:28:11


According to http://spamassassin.apache.org/downloads.cgi
the current version is 3.3.2.  But I doubt that matters.



That shows that the problem is related to how dccproc is run for
real mail by postfix+SpamAssassin.  It also shows that postfix+SpamAssassin
is breaking access to the /var/dcc/dccifd socket.



That is my mistake.  /usr/var/dcc/build/dcc exists only after running
/var/dcc/libexec/updatedcc.  You can find DCC.pm in the dcc-1.3.140/misc
directory of the DCC tarball that you expanded and after you run ./configure.

I trust you used the DCC source tarball from
http://www.dcc-servers.net/dcc/#download or
http://www.dcc-servers.net/dcc/source/dcc.tar.Z
I have been wasting my time answering your email if you are trying
to use one of the "improved" (spelled b-r-o-k-e-n) DCC versions
redistributed by Linux repackagers.



Yes and then dccproc fails to find another file in /var/dcc.
Doesn't that tell you that SpamAssassin (spamd) and all of its
children are unable to reach /var/dcc, probably because

Re: open(/var/dcc/map): Permission denied

Aldo Necci — 2011-09-23T09:57:15


Yes I have SpamAssassin 3.3.1 and this is its output:
# spamassassin -V
SpamAssassin version 3.3.1
  running on Perl version 5.10.1


I try this method of testing DCC with Spamassassin as described somewhere:
# spamassassin -D < /usr/share/doc/spamassassin-3.3.1/sample-nonspam.txt

and I got a lot of output, but this is the most important:
Sep 23 10:12:38.503 [2790] dbg: dcc: dccifd local socket chosen:
/var/dcc/dccifd
Sep 23 10:12:38.503 [2790] dbg: dns: entering helper-app run mode
Sep 23 10:12:38.503 [2790] dbg: dcc: connecting to a local socket
/var/dcc/dccifd
Sep 23 10:12:38.640 [2790] dbg: dcc: dccifd got response:
X-DCC-dcc1-Metrics: mbox2 1182; Body=many Fuz1=many Fuz2=many
Sep 23 10:12:38.640 [2790] dbg: dns: leaving helper-app run mode
Sep 23 10:12:38.642 [2790] dbg: dcc: listed: BODY=999999/999999
FUZ1=999999/999999 FUZ2=999999/999999 REP=0/90
Sep 23 10:12:38.644 [2790] dbg: rules: ran eval rule DCC_CHECK ======> got
hit (1)

So It seems to work right! There is *NOT* any warning like:
open(/var/

Re: open(/var/dcc/map): Permission denied

Vernon Schryver — 2011-09-22T19:26:30



Are you using a current version of SpamAssassin?

Have you tried the SpamAssassin DCC test?   I've forgotten how to
invoke it and do not see it mentioned in
http://spamassassin.apache.org/full/3.3.x/doc/Mail_SpamAssassin_Plugin_DCC.html

I understood that the SpamAssassin people were going to ship the
new version of the SpamAssassin DCC plugin in
/usr/var/dcc/build/dcc/misc/DCC.pm
If that file differs from the DCC.pm you are using, 
it might be entertaining to try it.



Is dccifd running?  If dccifd is running and SpamAssassin can reach
the UNIX domain socket at /var/dcc/dccifd, then SpamAssassin should
never try dccproc.  Since SpamAssassin cannot use dccproc to reach
/var/dcc/map, one might expect problems reaching /var/dcc/dccifd.

If dccifd is not running, then perhaps /var/dcc/libexec/rcDCC has
not been sym-linked to the right /etc/rc* directories.




Do the default settings involve jails or chroot?
I cannot guess after some Google for Scientific Linux.
Are you using sendmail, postfix, or something

Re: open(/var/dcc/map): Permission denied

Aldo Necci — 2011-09-22T08:26:49


Scientific Linux 6 uses the dot after the permission bits as a symbol
of "this is the end of permission bits".


The command "cdcc" works as well and this is its permission bits:
-r-sr-xr-x.  1 root bin  179830 Sep 21 17:38 cdcc


No, the directory /usr/local/bin is on a filesystem under /


SpamAssassin is configured to use the right path, this is its configuration:
use_dcc 1
dcc_path /usr/local/bin/dccproc
dcc_home /var/dcc
dcc_dccifd_path /var/dcc/dccifd

Any explanation on
http://spamassassin.apache.org/full/3.3.x/doc/Mail_SpamAssassin_Plugin_DCC.html

Even if I comment the second line and let SpamAssassin to find the
path for dccproc, I get the same log:
dccproc[1982]: open(/var/dcc/map): Permission denied

Another serious question is:
I don't see any UDP connection after dccifd started,
the output of the command "netstat -pu" is empty and
there isn't any firewall (I disabled the default software firewall).


No, I let the default settings.


Nothing, the log directory under /var/dcc/ is also empty.

Re: open(/var/dcc/map): Permission denied

Vernon Schryver — 2011-09-21T16:45:24



What is the significance of the period (.) after the permission bits?


Just now I tried:
   pax -rzf ...
   cd dcc-1.3.*
   ./configure --homedir=/tmp/dcc --bindir=/tmp/dcc/bin --mandir=/tmp/dcc
   make install

That made:
  -r-sr-xr-x  1 root  wheel  919480 Sep 21 16:18 bin/dccproc*
  -rw-------  1 root  wheel    7668 Sep 21 16:18 map

I see no problems with dccproc:
    % bin/dccproc -C
    asdf: asdf

    asdf
    X-DCC--Metrics: calcite.rhyolite.com 0; Body=1
reported: 1               checksum
   Message-ID: d41d8cd9 8f00b204 e9800998 ecf8427e

Does the `cdcc` command also fail?  Cdcc is also installed set-UID
too the --with-uid value.

Is /usr/local/bin be mounted with an option that turns off set-UID
or set-UID=0?

Is it possible that that the dccproc used by SpamAssassin (or
whatever) is not /usr/local/bin/dccproc but some other file such
as /usr/etc/bin/dccproc?   If SpamAssassin is involved, is
SpamAssassin configured to use /usr/local/bin/dccproc?

Are you doing anything with "jails" or c

Re: open(/var/dcc/map): Permission denied

John R. Levine — 2011-09-21T16:11:41

The dcc process usually runs as user "dcc", so the file should belong to 
dcc, not to root.

This is a familiar problem when an update doesn't work quite right.

Regards,
John Levine, johnl-ShX0hXQ9Y8s< at >public.gmane.org, Primary Perpetrator of "The Internet for Dummies",
Please consider the environment before reading this e-mail. http://jl.ly

Re: open(/var/dcc/map): Permission denied

Aldo Necci — 2011-09-21T15:36:08


OK. The UID is root and that file is private:
-rw-------. 1 root root 7668 Sep 21 17:21 /var/dcc/map




I have done all. But the problem is still there:
dccproc[9126]: open(/var/dcc/map): Permission denied

The dccproc file is:
-r-sr-xr-x. 1 root bin 496487 Sep 21 17:21 /usr/local/bin/dccproc

Thanks,
Aldo Necci




-----------------------------------------
This email was sent using SquirrelMail.
https://email.dia.uniroma3.it
Web Site: http://www.squirrelmail.org

Re: open(/var/dcc/map): Permission denied

Vernon Schryver — 2011-09-19T16:10:59


The /var/dcc/map file must be private and readable only by owner,
because it can contain client-IDs and passwords for accessing private
DCC servers.  (The anonmous client-ID of 1 does not use a password.)

`make install` installs dccproc set-UID to the UID specified with
`./configure --with-uid=UID`  The default UID is 0 or root.

It is quite possible that running /var/dcc/libexec/updatedcc
would fix the problem.  Configure builds updatedcc with the
./configure parameters including with-uid.
After fetching the tarball, updatedcc uses `./configure` and `make
install` to `chown` and so forth.
Updatedcc checks to see if it needs to be run by root so that `make`
can `chown` and so forth.

/var/dcc/dcc_conf-new is also built by updatedcc,
and can usually be installed as /var/dcc/dcc_conf


Vernon Schryver    vjs-w9Ndhk1xg/VWk0Htik3J/w< at >public.gmane.org

open(/var/dcc/map): Permission denied

Aldo Necci — 2011-09-19T14:25:38

Hi,

does anyone know why the following would appear in the logs:
dccproc[2304]: open(/var/dcc/map): Permission denied

I'm using last version (dcc-1.3.140) on Scientific Linux release 6.1
and when spamassassin (ver 3.3.1) process an e-mail I get that message.

The same configuration on Scientific Linux CERN SLC release 5.7
and DCC version 1.3.134 works without that message:
DCCD_ENABLE=off
DCCM_ENABLE=off
DCCIFD_ENABLE=on

thanks,
Aldo Necci


-----------------------------------------
This email was sent using SquirrelMail.
https://email.dia.uniroma3.it
Web Site: http://www.squirrelmail.org

Re: Using "ok env_From" to whitelist mail from whole domain

Vernon Schryver — 2011-08-27T16:42:11


You are right, and the FAQ is wrong.

To whitelist mail from the single mailbox such as, test-hcDgGtZH8xNBDgjK7y7TUQ< at >public.gmane.org,
use a line like this:

  OK      env_Fromtest-hcDgGtZH8xNBDgjK7y7TUQ< at >public.gmane.org

With the contents of /var/dcc/dcc_conf that are in the official source
(i.e. not from a third party), the following line will whitelist mail
from all mailboxes at the domain name:

  OK      substitute mail_host test-hcDgGtZH8xNBDgjK7y7TUQ< at >public.gmane.org


Thanks for pointing out the FAQ error,
Vernon Schryver    vjs-w9Ndhk1xg/VWk0Htik3J/w< at >public.gmane.org

Using "ok env_From" to whitelist mail from whole domain

Matthew Richardson — 2011-08-27T15:32:57

The FAQ at http://www.rhyolite.com/dcc/FAQ.htm has an entry about
whitelisting incoming email FROM a whole domain saying:-


My tests indicate that this does not work as described.  Whereras:-

OK      env_Fromtest-hcDgGtZH8xNBDgjK7y7TUQ< at >public.gmane.org

in the global whiteclnt file does cause both greylisting and checking to be
bypassed, a similar entry for the whole domain seems to be completely
ignored.

Should this feature work, or is the FAQ somehow mistaken?

I am using DCC 1.3.134 with Postfix using DCCM as a milter.

With many thanks.

Best wishes,
Matthew

[PATCH] Parsing clientwhitelist: unrecognized "optionno-forced-discard"

Bram — 2011-08-03T15:05:41

There appears to be a typo in the parsing of the clientwhitelist file.
When  option no-forced-discard is added then it produces the message:

'unrecognized "option no-forced-discard"'

- "no-forced-discard" is used 13 times in the package
- "no_forced-discard" is used 1 time in the package

=> I'm guessing this is a typo, a patch (against dcc-1.3.140) for this  
is attached.

Best regards,

Bram

Re: [PATCH] Create the socket of dccifd in the rundir

Vernon Schryver — 2011-08-03T15:33:38


dccifd is not a sendmail (or postfix) milter.


I agree that the dccifd socket by defailt should be created in the
run directory.  


However, changing the default location of the dccifd socket would break
many existing installations.

It is not necessary to patch the dccifd source to put the socket
anywhere you like.  Simply use -p, perhaps by adding -p/var/run/dcc
to DCCIFD_ARGS in /var/dcc/dcc_conf See the dccifd man page, perhaps
at http://www.dcc-servers.net/dcc/dcc-tree/dccifd.html#OPTION-p



} From: Bram <dcc-RusutVdil2g+VrKNYHDOSw< at >public.gmane.org>

}                   option DNSBL4-off
}                   MTA-last

} The last line should read "option MTA-last" instead of "MTA-last"

thanks.


Vernon Schryver    vjs-w9Ndhk1xg/VWk0Htik3J/w< at >public.gmane.org

[DOC-PATCH] Error in man page

Bram — 2011-08-03T15:06:36

There appears to be an error in the dcc manpage:

Quote from man dcc:
"
                ....
                option MTA-first
                option MTA-last
                    consider MTA determinations of spam or not-spam  
first so they can be overridden by whiteclnt files, or last so that  
they can override whiteclnt files.
                ...
              In the absence of explicit settings, the default in the  
main whiteclnt file is equivalent to
                  option log-normal
                  option dcc-on
                  option greylist-on
                  option greylist-ignore-spam-off
                  option greylist-log-on
                  option DCC-rep-off
                  option DNSBL1-off
                  option DNSBL2-off
                  option DNSBL3-off
                  option DNSBL4-off
                  MTA-last
"

The last line should read "option MTA-last" instead of "MTA-last"

=> Patch (against dcc-1.3.140) attached.

Best regards,

Bram

[PATCH] Create the socket of dccifd in the rundir

Bram — 2011-08-03T15:04:09

Configure/the documentation suggest that the pid file and the sockets  
are created in the run directory:

 From ./Configure --help:
   --with-rundir=DIR    for PID files and milter socket

This however is not the case for dccifd.
It creates the socket in the homedir and not the run directory.

Attached is a patch (against dcc-1.3.140) that creates the socket for  
dccifd in the run directory and not in the homedir.


Best regards,

Bram

Re: errors in mail log

Vernon Schryver — 2011-07-05T13:25:21


It means that according to the entry in the /var/dcc/map file for
the DCC server at 127.0.0.1, the version number of the DCC client-server
protocol supposedly used by that DCC server is 0.  That suggests
that the DCC software has been modified, that the /var/dcc/map file
has been corrupted, or that the UNIX-like operating system being
used has bugs in its implementation of memory mapped files.

By the way, unless I'm confused, your current organization does not
have a DCC server connected to the global network of DCC servers.
The DCC mechanism is far less effective (and often useless) without
access to the real time reports of bulk email from the global network
of DCC servers.  In addition, running a DCC server that does not
exchange reports of bulk email with the global network violates the
license on the free version of the DCC software.


Vernon Schryver    vjs-w9Ndhk1xg/VWk0Htik3J/w< at >public.gmane.org

errors in mail log

Tom Kinghorn — 2011-07-05T09:49:09

Good morning all.

does anyone know why the following would appear in the logs

dccifd[29182]: impossible pkt_vers 0 for < at > (127.0.0.1,6277)

thanks

Tom