gmane.mail.procmail

Re: generate line number in log files

2013-05-05T12:26:25

in message <20130505115657.GB1901< at >ratatosk>,
wrote Erik Christiansen thusly...

In which case ...

  # cat -n log

  # less --LINE-NUMBERS log




--

Re: generate line number in log files

Erik Christiansen — 2013-05-05T11:56:58

Line numbers only need to be visible when the file is viewed by a human,
I figure. So, if you're viewing the file in vim:

:set number

That covers the use cases that my imagination offers up.

Erik

Re: generate line number in log files

LuKreme — 2013-05-04T20:31:00

On 04 May 2013, at 09:10 , Eric Smith <es< at >fruitcom.com> wrote:


No, not really.


Sure. just log the filename at the beginning of the file.

LOG="Starting .procmailr"
LOG="Starting .proclistprocess"

etc.

generate line number in log files

Eric Smith — 2013-05-04T15:10:54

Is there any method for having line numbers and script filenames
printed in the procmail log files.

Avoid brute force method of multiple LOG="foo" invocations.

--
Eric Smith

Dealing with list messages that are Cc to you

LuKreme — 2013-03-06T18:44:30

I know we covered this several times many years ago, but at the time my solution was "If it's to my list account and it's not from a list, discard it"

Sadly, that is no longer an option, so I need a way to do the following:

If a message is TO me and ONLY to me, file it.
If a message is to me and ALSO to a list, file it somewhere else (possibly /dev/null :)
If a message is NOT to me, treat it as a list message (that is, let it fall through)

I have all the recipes in place to deal with the list messages.

I though that maybe this,a after all list processing might work:

:0
* 9876543210^1 ! X-Original-To:.*kremels< at >kreme.com
* 9876543210^1 ! TO_kremels< at >kreme\.com
{
   TRAP='mv "$LASTFOLDER" "${LASTFOLDER}:2,S"'

   :0
   .not-to-me/
}

Which should let all the mail that is addressed to my list account fall through to $DEFAULT (the InBox) right?

Re: Procmail with faceless account

Professional Software Engineering — 2013-02-02T14:48:47

You can invoke procmail from an alias (i.e. a non-account mail 
identity).  Mailing lists are/were often configured this way.  I have 
mail on a large site that pumps through procmail in this fashion.


Sounds rather like they're finding more and more things to ask 
someone else to write for them.


See my preceeding comment.  In some circles, this is called "job 
security".  If nobody else wants to learn the environment, and you're 
the one guy who does, seems you're in a good position.


I know there has been interest in having such an interface, but as 
the power of procmail is in being able to run external processes and 
chaining rules, etc, setting up a library and UI with which a 
neophyte could compose worthwhile rules without shooting themselves 
in the foot would be a major undertaking.

---
  Sean B. Straw / Professional Software Engineering

  Procmail disclaimer: <http://www.professional.org/procmail/disclaimer.html>
  Please DO NOT carbon me on list replies.  I'll get my copy from the list.

Procmail with faceless account

Peters, Ron — 2013-02-01T19:04:53

Hi all,

I've been using Procmail for a bunch of years for various reasons. Currently I'm using it as an enterprise tool using a faceless account instead of a real user's account. We've tightly integrated Procmail with our support desk ticketing system. Messages come from various internal and external distribution lists, automated processes, faceless accounts, web based forms, and various IT devices. Based on our rules, we generate tickets that are auto-routed in the ticketing system pre-defined users/groups. Some messages get tickets, some get deleted etc. With this in mind, I have a couple questions for the group here.

1. Procmail seems to be typically used or end users to manage their own mail. I haven't really run across any sites discussing it being used in this way. Is this a novel approach or have others gone down this path before? It seems that with the way most folks use email today (not from a shell), using Procmail in this way may give it more value and exposure than it's had in the past. I know that once folks in my organization started realizing what could be done (since email is so heavily used in many ways), they started finding more and more uses for it.

2. Because we're using it as an enterprise application that's centrally managed AND there's almost nobody here that understands/uses the CLI in a Linux environment, I was thinking about some type of web based frontend for managing all our rules. Though you'd still have to understand regular expressions and the scripts that I've written, it would reduce one barrier. Is there anything like this or has anyone thought about it? I looked at the Procmail Webmin/Usermin interface and it sort of works but since I INCLUDE a bunch of rule files, I didn't see how to manage those, only the INCLUDE rules. Thoughts?

Thanks in advance.
____________________________________________________________
procmail mailing list Procmail homepage: http://www.procmail.org/
procmail< at >lists.RWTH-Aachen.de
http://mailman.rwth-aachen.de/mailman/listinfo/procmail

Re: The escaped From bugaboo

Robert Bonomi — 2013-01-15T18:16:55



You misunderstood. The _sequence_of_processing_ is what you have to look in
the source to determine.  WHAT happens is documented -- for _when_ it occurs
one must check the source.


Well, then, obviously, it isn't happening! 
You can disregard everything in this thread,
including the initial report.    <grin>

Re: The escaped From bugaboo

LuKreme — 2013-01-15T17:45:21

On 14 Jan 2013, at 23:58 , Robert Bonomi <bonomi< at >mail.r-bonomi.com> wrote:


MH support was added very early, and maildir support was added in 3.14.


So it's *not* documented.

The only part of the source that seems to deal with escaping the From is formail, and there is quite a lot about how to either escape or not escape 'bogus' from lines. But that makes sense as formal is specifically setup to process mbox files by default.

Re: The escaped From bugaboo

Robert Bonomi — 2013-01-15T06:58:19


for the first statement -- firsthand experience.  I used it when it was
'mbox only'.

For the second, "Use the source, Luke."

Re: The escaped From bugaboo

LuKreme — 2013-01-15T00:36:23


Cite?


Cite? It is certainly not documented n any of the procmail manpages on my system.

For the record, procmail was first released in 1990. Maildir in 1995, and MH folders date from 1979.

Re: The escaped From bugaboo

Mike Peeler — 2013-01-14T23:56:35

I disagree.  First, most of the software that can operate on mbox
format works exactly like "grep -i '^From '".  Second, I regularly
work with mboxes where the '^From ' lines aren't preceded by a blank
line.  I use an alias 'fivmail' to do 'formail -bcde -m1'.

To extend your argument, that the extra blank line would rule out the
need for escaping in most cases, the mail program that creates these
pathological files, itself parses '^From ' lines strictly.  It has to
be followed by a 'user< at >host', then something that looks like a date.

And it's case-sensitive.  Just that one requirement could improve the
situation dramatically.  The point is, escaping is all but irrelevant,
to this program, in spite of its output that's so unfriendly to input.

But there are so many programs that "handle" mbox by working exactly
like grep -i '^From ', that this escaping now creeps in at every level
of many mailsystems.  Procmail's fix is stupid, but it is undeniably a
better outcome than the problems it prevents.

Mike

Re: The escaped From bugaboo

Robert Bonomi — 2013-01-14T17:46:19

a) Improper usage.  <grin>

   It's _by_ not "on".

b) it is documented behavior.


Immaterial and irrelevant.  Unfortunate, but accurate.

Procmail originates from the days when 'mbox-format' was the -only- mail
storage format.

Multiple storage format support was added much later.  '^From ' escaping
is done _long_before_ the 'delivery location' storage format is determined.
(More-or-less unavoidable, given the program's internal logic.)

This quirk[1] of procmail is well-known -- people have griped about it for
years (decades) -- and =nobody= has managed to come up with a 'fix' for
it.



[1] It's documented, Thus, per the classic IBM standard "it's a feature, not
    a bug."

Re: The escaped From bugaboo

LuKreme — 2013-01-14T16:11:02

On 13 Jan 2013, at 15:22 , Robert Bonomi <bonomi< at >mail.r-bonomi.com> wrote:


Hoisted on your own petard.


WHich is why I was sure to point out that messages were bing delivered to maildir. There are no mbox files on the server. At all.

Re: The escaped From bugaboo

LuKreme — 2013-01-14T16:09:47

On 13 Jan 2013, at 14:55 , Bart Schaefer <barton.schaefer< at >gmail.com> wrote:


Not very easily, no. Everything gets handed to procmail.

Re: The escaped From bugaboo

Professional Software Engineering — 2013-01-14T15:13:40


IIRC, this is supposed to apply ONLY to "^From " preceeded by a blank line.

---
  Sean B. Straw / Professional Software Engineering

  Procmail disclaimer: <http://www.professional.org/procmail/disclaimer.html>
  Please DO NOT carbon me on list replies.  I'll get my copy from the list.

Re: The escaped From bugaboo

Robert Bonomi — 2013-01-13T22:22:56


RTFM applies. <grin>

This is _necessary_ for mbox-format messages.

A line starting with "From " (5 chars) is the marker for the beginning of
a new message.  

Therefore, an embedded '^From ' is always escaped by the local delivery agent.

Re: The escaped From bugaboo

Bart Schaefer — 2013-01-13T21:55:11

On Jan 13, 2013, at 12:38 PM, LuKreme <kremels< at >kreme.com> wote:

This might be happening in the MTA before the message even reaches procmail.  Are you able to try a delivery with procmail completely disabled?

The escaped From bugaboo

LuKreme — 2013-01-13T20:38:02

I got an email today with this paragraph in it:

***
These new “post-Panamax” ships are the length of aircraft carriers.
of the Lincoln Memorial. The ships can carry as many as 12,000
containers, or about a million flat-screen TVs.
***

As you can see, the "From" is escaped.

I'm not sure why this continues to happen, and it happens on an email that is delivered without a .procmailrc, which makes me think the issue is in the default procmailrc. The only part that modifies the original message is the formail to add the X-Hostname header.

cat /usr/local/etc/procmailrc 
DATE=`date '+%Y%m%d%H%M%S'`
LOGFILE=/var/log/procmail
LOGABSTRACT=NO
HOSTNAME=`hostname`
NL="
"
WS="  "

:0c
{
        :0
        /backup/imap.backup/
}

:0
{
  INCLUDERC=/usr/local/etc/furrin.rc

  LOG="SPAM:${NL} ${SPAMVAL}${NL} ${SPAMMISHNESS}${NL} ${SPAMNOTES}${NL}"
}

:0fw
| formail -I"X-Hostname: $HOSTNAME"

# from Sean B Straw
# provide FROM_USER, FROM_DOMAIN, SUBJECT, TO, FROM, ENVTO, ENVFROM

:0
{
        :0
        * $ ^Subject:$WS*\/[^$WS].*
        { SUBJECT=$MATCH }

        :0
        * $ ^To:[$WS]+\/[^$WS].*
        { TO=$MATCH }

        :0
        * $ ^From:[$WS]+\/[^$WS].*
        { FROM=$MATCH }

        # This is an optional header - your MTA configuration may not insert
        # it (bummer for you).  It is very useful to have
        :0
        * 9876543210^0 ^X-Original-To: *<\/[^>]*
        * 9876543210^0 ^X-Envelope-To: *<\/[^>]*
        { ENVTO=$MATCH }

        # This is also an optional header.  If you don't have this, you can
        # get the same information through the commented out rule which
        # follows.
        :0
        * ^X-Envelope-From: *\/[^$WS].*
        { ENVFROM=$MATCH }

        # alternative to X-Envelope-From:
       :0
       * ^From \/[^ ]*
       { ENVFROM=$MATCH }

        # Here we have to call shell.... -rt will parse return address
        # according to RFC rules.  Note we only process HEADER.
        :0 h
        SENDER=|formail -b -rtzxTo:

        # get the From: address as an address component ONLY (no comments)
        :0 h
        CLEANFROM=|formail -IReply-To: -rtzxTo:

        # username portion
        :0
        * CLEANFROM ?? ^\/[^< at >]+
        { FROM_USER=$MATCH }

        # domain portion
        :0
        * CLEANFROM ?? < at >\/.*
        { FROM_DOMAIN=$MATCH }

:0
* FROM_DOMAIN ?? .*\/([^\.]+)
{ TOPD = $MATCH }
        :0
        * Received:.*from \/.*by mail.covisp.net
        * MATCH ?? ^^\/[^)]+
        { DFROM = $MATCH }


     # Obtain the hostname of the host which relayed the message to us.
     # This is found in the topmost received header.
     
     RELAYHOSTX=`formail -u Received: -czx Received:`
     
     # The hostname provided in the SMTP EHLO exchange will be the first
     # token on this line.
     :0
     * RELAYHOSTX ?? ^from \/[^    ]*
     {
     RELAYHOSTEHLO=$MATCH
     }
     
     # Then isolate the hostname portion (if any) in the parenthetical.
     :0
     * RELAYHOSTX ?? ^from [^   ]* \(\/[^)]*\)\>+by\>
     * MATCH ?? ^\/[^)]+
     {
     RELAYHOSTX=$MATCH
     
     :0
     * RELAYHOSTX ?? ^\/[^[ ]+
     {
     # grab whatever up to the first space or the open
     # brackets for the IP
     RELAYHOST=$MATCH
     }
     
     :0
     * RELAYHOSTX ?? ()\[\/[^] ]+
     {
     # grab the apparent host IP from the brackets
     RELAYHOSTIP=$MATCH
     }
     }
     
     # null out RELAYHOSTX (temp variable used in the extraction process)
     RELAYHOSTX=
     
     # if the relay host has no rDNS, RELAYHOST should be undefined.
}

LOG="RELAYHOSTEHLO=$RELAYHOSTEHLO RELAYHOSTIP=$RELAYHOSTIP $NL"

# the RELAYHOSTELHO should match against all known aliases for your host
# - the IP(s), the hostname(s), and localhost.  RELAYHOSTIP should be the
# IPs and localhost IP (127.0.0.1).
:0
* RELAYHOSTEHLO ?? (75\.148\.117\.93|(mail|akane)\.covisp\.net|localhost)
* ! RELAYHOSTIP ?? (75\.148\.117\.93|127\.0\.0\.1)
{
      :0 fw
      | formail -I"X-Host-RCVD: Foreign sender using our hostname or IP for submission${NL}"
}

LOG=$NL"$DATE: TOPD=\"$TOPD\", 
FromU=\"$FROM_USER\", DFrom=\"$DFROM\", FromD=\"$FROM_DOMAIN\", 
Subject=\"$SUBJECT\", Relayhost=\"$RELAYHOST\",
To=\"$TO\", From=\"$FROM\", EnvTo=\"$ENVTO\", EnvFrom\"$ENVFROM\"$NL"

:0  
* ! ? test -f $HOME/.procmailrc
{
  LOG="User has no procmailrc LOGNAME=$LOGNAME< at >$HOST HOME=$HOME$NL"
  :0 fw
  | /usr/local/bin/spamc -u $LOGNAME

  DROPPRIVS=YES
  DEFAULT=$HOME/Maildir/

   :0 
   * ^X-Spam-Status:(.*\<)?Yes
   {
      :0
      { TRAP='mv "$LASTFOLDER" "${LASTFOLDER}:2,S"' }

      :0
      .SPAM/
   }

   :0
   $DEFAULT
}
LOG="User $LOGNAME< at >$HOST has a .procmailrc, processing...$NL" 

There are no errors thrown by this file,a dn the log looks just as I would expect:

FromU="root", DFrom="", FromD="covisp.net", 
Subject="Test", Relayhost="",
To="faker< at >covisp.net", From="root< at >covisp.net (Charlie Root)", EnvTo="", EnvFrom"root< at >covisp.net"
User has no procmailrc LOGNAME=faker< at >mail.covisp.net HOME=/home/faker

and the email has the from line escaped as above.

Re: save to list dir and conditionally to $DEFAULT -> solved

Eric Smith — 2012-11-16T14:41:09

Richard Ball wrote on Tue-13-Nov 12 11:44PM

Thanks Richard

That is the way I implemented it in the end and it seems fine.

Eric

Re: save to list dir and conditionally to $DEFAULT

Richard Ball — 2012-11-13T22:44:02

On Nov 12, 2012, at 3:05 AM, Eric Smith wrote:


There isn't one correct way. Can you structure your recipes so you determine the proper folder separately from the delivery recipe? Then you just need one delivering recipe which also has the egrep for subjects which, if met, drops a copy into DEFAULT.

Rich