gmane.ietf.rfc822

draft-kucherawy-received-state

Murray S. Kucherawy — 2012-01-09T22:44:07

Hi all,

Over on ietf-smtp, https://datatracker.ietf.org/doc/draft-kucherawy-received-state/ was developed late last year.  This is an optional tweak to Received fields allowing annotation of entry into special states of mail handling, such as quarantines or hold-for-moderation MLM actions that would help to explain large gaps in timestamp sequences.

I'm looking for a wider audience of reviewers and (hopefully supporters).  If this sort of thing seems like a good (or terrible) idea, please do follow up and comment.

How much and what kind of support is evident will guide the choice of what its next steps are (if any).

Thanks,
-MSK

draft-kucherawy-received-state

Murray S. Kucherawy — 2012-01-09T22:44:07

Hi all,

Over on ietf-smtp, https://datatracker.ietf.org/doc/draft-kucherawy-received-state/ was developed late last year.  This is an optional tweak to Received fields allowing annotation of entry into special states of mail handling, such as quarantines or hold-for-moderation MLM actions that would help to explain large gaps in timestamp sequences.

I'm looking for a wider audience of reviewers and (hopefully supporters).  If this sort of thing seems like a good (or terrible) idea, please do follow up and comment.

How much and what kind of support is evident will guide the choice of what its next steps are (if any).

Thanks,
-MSK

Re: Do S/MIME and OpenPGP protect message headers?

Dave CROCKER — 2011-12-28T16:12:33


On 12/28/2011 7:27 AM, Rolf E. Sonneveld wrote:


However, of course, the contained message then ceases to be treated as a normal 
message by the receiving user agent.  It's just content.

Although not it's primary task, data integrity protection /is/ provided for 
listed header fields by DKIM.  (The signature lists the fields it covers. This 
typically does include From:, To:, and cc: and Subject.)

d/

Do S/MIME and OpenPGP protect message headers?

Rolf E. Sonneveld — 2011-12-28T15:27:42

Hi,

just to check: S/MIME (http://tools.ietf.org/html/rfc5751) does not  
does include/cover the 5322.From and the 5322.To fields as part of the 
cryptographic payload. Protection of headers can be achieved by wrapping 
up a the complete message into a message/rfc822 bodypart and 
sign/encrypt that (par. 3.1 of that RFC). How is that with OpenPGP 
(http://tools.ietf.org/html/rfc4880)? OpenPGP does not protect 5322.From 
and 5322.To either, does it?

Par. 5.11 of RFC4880:

    A User ID packet consists of UTF-8 text that is intended to represent
    the name and email address of the key holder.  By convention, it
    includes anRFC 2822  <http://tools.ietf.org/html/rfc2822>  [RFC2822  <http://tools.ietf.org/html/rfc2822>] mail name-addr, but there are no
    restrictions on its content.  The packet length in the header
    specifies the length of the User ID.


The recipient address is not mentioned anywhere. So is the following 
statement correct?:

Neither S/MIME nor OpenPGP protects the '5322.headers' of

Privicons and redaction

Alessandro Vesely — 2011-12-09T12:18:22

In abuse reporting, an established albeit discouraged practice
consists of redacting reported messages.  In a nutshell, users may
have the ability to signal some received messages as being abusive.
Their mailbox providers should then wrap those messages according to
abuse reporting format (ARF, RFC 5965) and send them back to the
senders if they had subscribed to the relevant feedback loop.  See
http://en.wikipedia.org/wiki/Feedback_loop_%28email%29

Redaction conceals the recipient, rather than the sender.  In this
case, authors should markup the parts of the messages that contain
sensible data, in case the recipients report those messages as spam,
possibly inadvertently.  A document describing how to carry out
redaction is now in LC, so it can hardly mention privicons.  However,
privicons could consider this possible use.  The redaction doc is at
http://datatracker.ietf.org/doc/draft-ietf-marf-redaction/

For casual readers, privicons are defined here:
http://tools.ietf.org/html/draft-koenig-privicons

draft-koenig-privicons-03.txt

Bill McQuillan — 2011-12-05T20:54:06

My comments on "Privacy Preferences for E-Mail Messages"

   .
   .
   .

It would be more correct to use the term "header field" rather 
than "header" here and in the rest of the document.
   .
   .
   .

   .
   .
   .

The definition of Sending User needs to be clarified: is it the 
email-address from the Sender: header field? Or one or more of 
the email-address from the From: header field?
   .
   .
   .
   .
   .
   .

This "MUST" seems to conflict with the MAY later in this spec.
   .
   .
   .

These definitions need to be formatted to more clearly 
distinguish the term from its definition. Perhaps like the 
"privicon" definition.


The definition of "first line" is not at all clear except for a 
TEXT/PLAIN body.

In the case of a TEXT/HTML body, the intent seems to be that the 
first line is the topmost "line" visible to the Receiving User. 
However, since the "first line" privicon is the determining 
specification in the case of disagreements and the MUA is 
expected to "enforce" this, the MUA mus

[Fwd: [apps-discuss] Call for adoption of draft-kucherawy-mta-malformed-00.txt as an APPSAWG document]

Alexey Melnikov — 2011-04-20T21:45:29

FYI.

Re: Comments on Malformed Message BCP draft

Hector Santos — 2011-04-20T14:38:21

Keith Moore wrote:



I don't think I was Keith.  I am firm believer all transactions should 
be treated equally when it comes to following standards with a "wait 
for someone to complain" exception.    I was pointing out dealing with 
compromising user issues is a different problem with a different set 
of solutions and I believe relaxing standards does not help with the 
problem.  So even if you wanted to use a negative MUA behavior change 
red-flag as part of a local policy compromised user monitor, you 
couldn't.


For offline MUAs, I generally agree, at least for the things we often 
are concern about. For Online MUAs, there are improvements, including 
support for List-IDs as done with gmail.com

Re: Comments on Malformed Message BCP draft

Keith Moore — 2011-04-20T13:19:19

On Apr 20, 2011, at 5:55 AM, Hector Santos wrote:


Please don't propose to penalize users for switching MUAs.   It's difficult enough for people to do as it is, and that is part of why MUAs are so dysfunctional and have evolved so little over the past 10+ years.   

Keith

Re: Comments on Malformed Message BCP draft

Hector Santos — 2011-04-20T09:55:48

Murray S. Kucherawy wrote:

Isn't this a different problem - compromised users?

When there is a compromised MSA user, there are other more serious 
issues to address - like detecting by whatever means and lock the 
user/machine out immediately.   It could be stolen user credentials or 
machine compromised for users who are still IP based authorized by 
their ISP network.  I seriously doubt the compromised user/machine is 
going to use the same user MUA to blast spam, it could, but then the 
user can see that something is wrong.

This gives more weight to the idea to not tolerate standards 
violations. Relaxing on standards makes it more difficult to use a 
delta in MUA compliance behavior as good or bad.

Re: Comments on Malformed Message BCP draft

Keith Moore — 2011-04-20T04:35:59


On Apr 20, 2011, at 12:29 AM, Murray S. Kucherawy wrote:


The only way I see to deal with these cases is for downstream relays or MXes to bounce or even drop malformed messages.  That will weed out those (hopefully few) MSAs.  

(Bouncing such messages won't get many users to change their MUAs.   But if their mail doesn't get through, they will change MSPs.  Many users have multiple email accounts anyway, and "resending from a different email address" is a common retry strategy.)

Keith

RE: Comments on Malformed Message BCP draft

Murray S. Kucherawy — 2011-04-20T04:29:10


Where the MSA is under the control of someone that wants to exploit browser or MUA weaknesses, or doesn't care about standards as long as his/her crap gets to the inbox, doesn't that view of the world means there's no hope for improvement?

Re: Comments on Malformed Message BCP draft

Keith Moore — 2011-04-20T04:24:41



I strongly suspect that the sender's response would be to:
a) find another mail service provider (even at the expense of getting another email address), and/or
b) stop sending mail to any recipients for whom such bounces happened with any regularity
...neither of which actually fixes the problem (except for providers that provider their own user agents)

People these days are so used to dealing with spam and error messages that they don't understand, that the vast majority of them won't even bother to read such messages.  As soon as they realize it's not intended for them _personally_ , the trigger finger reflexively hits the delete key.  The user may indeed be annoyed, but the chance that he'll take useful corrective action is almost zero.

If we want such measures to work, we need to find a way to either
a) send the notifications to the vendors/providers of such user agents, or
b) arrange for them to get triggered only when the user starts using a new user agent, so he'll immediately associate the user

Re: FW: Comments on Malformed Message BCP draft

Charles Lindsey — 2011-04-19T20:25:07





Yes, but there are other possibilities (yes, 'fixing' is to be avoided).
But for suitable messages (for some meaning of "suitable") you could send
a bounce back up the return path of the following form:

"Your message is non-compliants for reasons <blah>. Although we have
attempted to deliver it, you need to be aware that it may fail to reach
its intended recipient, and we would suggest that you endeavour to make
future messages compliant with the relevant internet standards <blah list
of standards blah>."

That way, the message (probably) gets through, but the sender gets pissed
off sufficiently to complain to his provider, threatening to vote with his
feet, etc. etc.

So this piles pressure on providers to adhere to standards (a Good Thing)
but still allows the mail to get through (also a Good Thing).

Re: [apps-discuss] Comments on Malformed Message BCP draft

Douglas Otis — 2011-04-18T22:56:08

On 4/18/11 5:48 AM, John C Klensin wrote:
Fully Agree.

-Doug

Re: Comments on Malformed Message BCP draft

Hector Santos — 2011-04-18T18:00:09

Keith Moore wrote:


+1

The principle applies in many areas, but overall, it is most 
usefulness is when there existed greater uncertainty.


+1

and I think intent is also related to trust or being non-anonymous.

Exceptions are a fact of life.  If you are unknown, then you get on 
the back of line like everyone else. But if you are known ("on the 
special party list"), you get special treatment and can avoid standing 
on a line.  Its always possible to learn that you are not a 
troublesome, a regular party dude to get a pass, but its all still 
about be recognized.

After 25+ years, the uncertainly principle for relaxation protocols 
should no longer be a quick out. There is simply no excuse today to 
continue tolerating violations of protocols or relaxing it for all. 
Exceptions to the rule of thumb has to be about trusted exceptions - 
not risky exceptions.

Re: Comments on Malformed Message BCP draft

Hector Santos — 2011-04-18T15:44:50

Dave CROCKER wrote:


Paraphrasing Shaw, telling the truth is the funniest joke in the world.

I would like to suggest a reality which it is closer to:

     Receive-side workaround exceptions

and not the rule of (protocol) thumb.

The reason is because most systems are compliant today, especially 
among actively supported software and when there is an issue raised, 
only good guys complain/report, not the bad guys. Hence most problems 
are generally fixed. The historical reality has been the anonymous bad 
guy exploitation of protocol relaxations which has proven to the cusp 
of the highly expensive mail abuse problem.

What we are really talking about here with workaround exceptions, is 
sender trust, in which case, it is not longer anonymous and the 
protocol relaxations begin to make more sense.   But anonymous abuse 
SHOULD NOT be tolerated and continue to be an excuse to violate 
protocol. There are some strong compliancy parts of protocol which has 
certainly shown a practical reality for a reject f

Re: [apps-discuss] Comments on Malformed Message BCP draft

Hector Santos — 2011-04-18T15:05:24

John C Klensin wrote:

+1

You can see one common issue the malformed message I-D shows as an 
example with the field-name having a space:

    "field-name :"

This would be akin to the SMTP space syntax debates, with the example:

    Mail From:space<address>

Recognizing this long time issue, this note was added to RC5322 
(Section 3.3)

    Since it has been a common source of errors, it is worth noting that
    spaces are not permitted on either side of the colon following FROM
    in the MAIL command or TO in the RCPT command.  The syntax is exactly
    as given above.

Implementators are going to decide on whether they will accept the 
illegal form with the space all depending on the severity of 
frequency.  In the past, the #1 MUA was the offending problem so while 
we didn't encourage relaxation, it had to be noted.

IMO, I don't think this a similar consideration applies to similar 
5322 header syntax errors.  It was reasonable to see why the SMTP 
systems had to consider relaxing it due to a wide

Re: [apps-discuss] Comments on Malformed Message BCP draft

Hector Santos — 2011-04-18T14:44:16

Tony Finch wrote:

+1.

My view is that since only Good Guys complain, Bad Guys do not, there 
are some well established areas for strong compliancy checks that has 
proven to have a high efficiency bad mail filtering rate with little 
false positives for good guys. This is because most good guys are 
compliant, especially for these particular areas of compliancy. 
Others are more subjective, so it generally it becomes a decision to 
how its enabled - on/off by default.

Traditionally, the transport is a passive payload transfer system but 
has have options for decisions than it does in applying subjective 
decisions related to the payload.

I think good examples are:

SMTP LEVEL:

   - anal strong syntax checks, i.e, "MAIL FROM:space"

   - Checking for explicit EHLO [ip literal] matching connection IP
     requirement. Exceptions used for growing NATS related issue
     in SOHO market with more than 1 PC.  PORT 587 can be used
     to skip this check.

   - Return path verification - methods in debate, bu

Re: Comments on Malformed Message BCP draft

Dave CROCKER — 2011-04-18T14:36:58



On 4/15/2011 7:09 PM, ned+ietf-822< at >mrochek.com wrote:


Receive-side workarounds, rather than rejections of protocol/format violations, 
seems to be an inherent state of affairs for apps protocols.  No one likes that 
fact, but no one has come up with a way to change the reality.

d/

RE: Comments on Malformed Message BCP draft

2011-04-16T02:09:26


More like "almost always". I estimate we average one complaint that turns out
to be message maformation every week or two, and it's been this way for almost
two decades. But despite the fact we as a matter of policy *always* push back
saying "fix the actual problem", I can't recall more than a handful of cases
where this resulted in the actual problem being tracked down and fixed.
Workarounds are a lot easier than, say, getting a printer manufacturer
to update their firmware so it doesn't produce this content-type header:

    context-type: 'image/tiff'

(Yes, that's an actual example from a few weeks back, and also note that while
it is clearly maformed, it's syntactically legal.)

Additionally, the various options we provide for stricter message checking are
among the few options we've implemented that AFAIK are essentially never used
by anyone, ever.


I agree, although I suspect getting consensus on this is going to be difficult.

Ned