gmane.mail.imap.dovecot

Re: dsync-2.2.2 incorrectly synchronizes subscription status of deleted mailbox

Karol Jurak — 2013-05-21T13:58:09

**

On Monday 20 of May 2013 17:33:44 Timo Sirainen wrote:








I performed the test and subscription state of a deleted mailbox is still
incorrectly synchronized: the mailbox is being subscribed on the first
server (the one it was originally deleted on) instead of being unsubscribed
on the other. One thing changed however: a subsequent attempt to
unsubscribe this mailbox on the first server is now correctly synchronized.
Previously every attempt to unsubscribe a nonexistent mailbox was being
reverted by dsync.

Re: search and UTF-8 normalization forms (NFD)

Lutz Preßler — 2013-05-21T11:41:53

Thanks, but there seems to be still a problem left. Sender search
yields all Krüger mails without fts_lucene. But with fts_lucene
enabled - and files in lucene-indexes/ existing - it's not.
(If I delete the lucene-index files and search for sender,
result is correct - but only until they are recreated.)

Lutz

Re: CATENATE/literal8 issue

Timo Sirainen — 2013-05-21T10:24:07


Well, the problem is that if it does contain NULs, the MIME part needs to be converted to something that doesn't. And to do that it needs to modify the previous header, which with current code was already read.. So to fix that it would need to read the whole message into a temporary file before actually saving it, which makes performance worse for the normal case..

Or are you saying that the error is fine if the text contains NULs, but simply should be allowed as long as it doesn't?

Re: Outlook 2013 - mounting folders with XLIST

Hajo Locke — 2013-05-21T08:49:02

Hello,



My serverside setup now is completed. I did a lot of tests last weeks and 
experienced some strange behaviour of some clients.
Outlook 2013 is only working when adding XLIST manually to imap_capability
imap_capability = +XLIST
This is because outlook 2013 not supports rfc 6154 but the deprecated XLIST 
standard invented by google.
So the problem with junkfolder is not a bug in Outlook 2013, in rfc 6154 
spamfolder is tagged by \Junk, in XLIST standard \Spam is used.
I did see that when using a gmailaccount in outlook 2013.
Adding XLIST capability to dovecot seems to be a problem for other Clients. 
k9 is able to work with rfc 6154 servers. But if k9 finds XLIST and 
SPECIAL-USE together in capabilitystring it seems to prefer XLIST requests. 
Because of dovecot is accepting XLIST requests, but outputs rfc 6154 
details, k9 seems to be confused and dont finds special Folders. rfc 6154 is 
similar but not identical to XLIST.  If you dont test with really individual 
foldernames, you get tricked by clients behaviour.
I looked around and the most imap-servers of hosting companies etc. provide 
XLIST feature, Special-USE unfortunately only a few.
So i did now some changes to dovecot sources on my own.
I added \Spam as allowed special-use attribute and created a new function 
for XLIST Requests. So if XLIST is requested, Clients gets lines of output 
with XLIST and \Junk is replaced with \Spam.
So all is done in the code and i dont need to change my userdb-config. 
Testing this server with different clients was successful. All of them did 
find their special folders and worked fine, outlook 2013 also finds 
spamfolder now. So this changes contribute to consolidate a deprecated 
standard but i have to find a way where all users can benefit from new 
features.
This is not a request to change something in dovecot, this is a call to 
decision makers to support one rfc Standard.

Hajo

Re: Sieve/pigeonhole with Exim and Dovecot LDA

Sebastian Arcus — 2013-05-21T07:54:15

That makes sense - I was just surprised it hasn't bitten me in the back 
so far. I'll amend the configs to process one message at a time in the 
future. I can only assume so far it was using the sender address of the 
first message to be processed?

CATENATE/literal8 issue

Michael M Slusarz — 2013-05-21T06:40:50

Using 2.2.2, I see this:

C: 6 APPEND "INBOX" (\seen) "16-May-2013 22:05:14 -0600" CATENATE (URL  
"/INBOX;UIDVALIDITY=1255685337/;UID=48812/;SECTION=HEADER" TEXT ~{40}
S: 6 NO [UNKNOWN-CTE] Binary input allowed only when the first part is binary.

Why is there this limitation?  It seems to me that CATENATE is  
confusing the content-type encoding of the data/part itself with the  
encoding of the IMAP literal.

A literal 8 is nothing more than a series of OCTET's that *may*  
contain nulls, but not necessarily.  i.e., in the above example the 40  
octets of data are US-ASCII text, which is perfectly acceptable to  
send as a literal8.  (Client rationale: If BINARY exists on the  
server, we don't bother to scan IMAP literal's for null data -- we  
just send them as literal8's.  It's an optimization that I would hate  
to get rid of.)

michael

Re: Configure dovecot to provide SASL authentication

Ben Morrow — 2013-05-21T02:16:33

While I would agree with you in principle, the documentation
(http://wiki2.dovecot.org/Services#auth) actually says

    client: Only SASL authentication is allowed. This can be safely
    exposed to entire world.

Given that the SASL auth service will eventually be exposed to untrusted
users via SMTP, the only additional risk from making this socket
world-readable is that (AFAIK, at least) there is no rate-limiting. This
makes the socket a password oracle, which can by used be any local user
with access to the socket to mount a dictionary attack.

However, given again that the permissions on /var/spool/postfix/private
should be 0700 postfix:wheel, and that (again AFAIK) all modern systems
check the permissions on the full path when connecting to a Unix-domain
socket, it doesn't actually matter what the permissions on the socket
are as long as postfix can connect, so 0666 is in this case entirely
safe.

Ben

Re: How to configure ssl cert chain in dovecot 10-ssl.conf file

Gedalya — 2013-05-21T01:41:03

Interesting. Technically, every line of text should end with a newline.
Your files had the last line of text unterminated - that's the kind of
thing Windows text editors do.


On 05/20/2013 09:39 PM, Bu Xiaobing wrote:

Re: How to configure ssl cert chain in dovecot 10-ssl.conf file

Bu Xiaobing — 2013-05-21T01:39:11

Gedalya,

Thanks for your reply, it works now, and finally I find it was the format problem, there should been a return between there cert files when cat into one single file.

On 2013-5-18 17:48, Gedalya wrote:

Re: Sieve/pigeonhole with Exim and Dovecot LDA

Gedalya — 2013-05-21T00:38:28

The point is that the transport, and then in turn the authenticator are 
meant to potentially process more than one message in a single 
connection. What is the meaning of $sender_address or $header_*? The 
sender of which message? The headers from which message?
If you do anything message-specific at this stage, you need to set this 
so only one message is sent per connection, so that message-specific 
variables can be meaningful.

Re: Sieve/pigeonhole with Exim and Dovecot LDA

Sebastian Arcus — 2013-05-20T23:37:49

Yes - they are in a lookup file.


This is what I'm using at the moment to authenticate against the 
provider's SMTP server (upstream) in smtp smart relay mode:

fixed_plain_client:
   driver = plaintext
   public_name = PLAIN
   client_send = ^$sender_address^${lookup{$sender_address}\
     lsearch{/etc/exim/exim-client.passwd}{$value}{fail}}



I only have internal lan clients connecting to this server - and even 
if, for any reason which I can't think at the moment - they would want 
to pass a fake "From:" header - it would be useless without passing the 
right password that goes with it.


Yes - I've tried it with the colon.

Yea, I know.

I just tried it again, with debugging on, and I get the following:

212.227.15.163 in hosts_try_auth? yes (matched "auth.smtp.1and1.co.uk")
scanning authentication mechanisms
   SMTP>> AUTH PLAIN ************************************
tls_do_write(bfac815f, 49)
SSL_write(SSL, bfac815f, 49)
outbytes=49 error=0
waiting for data on socket
Calling SSL_read(8109288, bfac855f, 4096)
read response data: size=37
   SMTP<< 535 no password in decoded response
fixed_plain_client authenticator yielded 2
LOG: MAIN
   fixed_plain_client authenticator failed H=auth.smtp.1and1.co.uk 
[212.227.15.163] 535 no password in decoded response

I don't think header_from: is available during authentication - or 
something else is happening which is escaping me right now.



That's an interesting one. I've been running several sites  for a few 
years now with exim in smart relay - without connection_max_messages = 1 
- and had no problems so far. Maybe it's because only few lan clients 
are involved - or I've been lucky so far :-)

I know

I think I'll have to do that. Thanks again for all the suggestions.

Re: Sieve/pigeonhole with Exim and Dovecot LDA

Gedalya — 2013-05-20T22:40:16

OK, now I had some more time to look at your situation.
We can ask, do you really need the sender? How do you use it? You're 
trying to authenticate using the sender, do you have the passwords in a 
lookup file?
Perhaps this can be a good idea: set up a special authenticator with:
client_condition = ${if match_ip{$sender_host_address}{:< at >[]}{1}{0}}
so that it can only be used for locally submitted messages (this 
_should_ work, test it), and statically configure it with credentials 
that would work with your upstream SMTP server?
Either way, you shouldn't have an authenticator that would trust the 
From: header and do something with it, unless the situation is very 
tightly controlled. You probably want to put more restrictions there to 
make sure this works only when intended, i.e. dovecot autoreplies.

Now, as for $header_from, first of all, it's "$header_from:", with the 
colon in the end. Yea, I know.
Secondly, I have no idea if it would be available in an authenticator. 
Consider that an authenticator is not really something that is related 
to processing an individual message.
One thing is for sure, you would need to set connection_max_messages = 1 
in the smtp transport which would be handling these messages. I know 
that that helps to make $sender_address available in the authenticator, 
try your luck with $h_from: or try to pass that data in somehow, ACL 
variables or something, let me know how that goes - I'm curious, but if 
you need further help you should probably ask on the exim-users mailing 
list (and point me at the thread ;-))

Re: Sieve/pigeonhole with Exim and Dovecot LDA

Sebastian Arcus — 2013-05-20T21:13:33

Thanks for that. I've just tried using $header_from: in my exim 
authenticator in client mode when talking to the provider's SMTP server 
in smart relay mode (instead of $sender_address) - but for some strange 
reason it just won't work. I've poured over the exim logs in debug mode 
- and so far I can't make sense of what is happening. I'll try some more 
to figure it out and get it working.

Re: Pigeonhole: extprograms - pipe

Stephan Bosch — 2013-05-20T19:21:50

This should fix it:

http://hg.rename-it.nl/dovecot-2.2-pigeonhole/rev/d4e9ca7fddcf

Regards,

Stephan.

Re: Sieve/pigeonhole with Exim and Dovecot LDA

Gedalya — 2013-05-20T16:12:29

If using the From header actually makes sense to you... then see 
$h_<header name> at 
http://www.exim.org/exim-html-current/doc/html/spec_html/ch-string_expansions.html, 
you probably want to restrict the usage of this as much as possible.
The envelope sender must be empty for bounces and auto-replies, pretty 
good article here: https://github.com/Exim/exim/wiki/EximAutoReply
Later I'll read through your whole message again and maybe I'll come up 
with something more concrete and detailed..

Re: Sieve/pigeonhole with Exim and Dovecot LDA

Sebastian Arcus — 2013-05-20T16:02:44

I have done a bit more testing, and it seems Dovecot LDA uses the 
following command options when sending out email through Exim:

exim -i -f <> -- recipient< at >address.com

The problem with the above is that it sets an empty address for the 
"Sender" field in the message envelope. The message "From" header is set 
correctly - but the envelope "Sender" field is empty. As I use exim in 
smart relay mode, exim can only use the "Sender" field from the envelope 
to authenticate against the provider's SMTP server (Exim doesn't seem to 
have any variable expansion for the "From" field in the header to be 
used during SMTP authentication) - thus the authentication fails and the 
message can't go away.

Re: Configure dovecot to provide SASL

Peter Skensved — 2013-05-20T15:48:58


   Thanks for the reply -

 - all the other sockets in /var/spool/postfix/private have mode
666 ( and the directory itself mode 700 ) but that is probably a
question for the postfix list....

                             peter

v2.2.2 + dsync version incompatibility

Timo Sirainen — 2013-05-20T15:27:58

Looks like dsync v2.2.2 isn't fully compatible with earlier dsync
versions, unless you add:
http://hg.dovecot.org/dovecot-2.2/rev/e0156c479a12

Sieve/pigeonhole with Exim and Dovecot LDA

Sebastian Arcus — 2013-05-20T15:12:25

I am trying to configure my Dovecot installation to provide 
Vacation/Out-of-the-office emails using the Sieve plugin. My setup is a 
little bit peculiar:


     Internet                           Internet
        |                                  ^
        V                                  |
   Provider's POP3 server         Provider's SMTP server
        |                                  ^
        V                                  |    --------------
     getmail                               |     my server
        |                                  |
        V                                  |
   Dovecot LDA ---> Sieve/vacation -----> Exim
        |
        V
     Dovecot                                     my server
                                               ---------------


Sorry for the ASCII art above - I thought it would be quicker than 
trying to explain.

The trouble I'm having is getting the Dovecot LDA to send successfully 
through the local exim instance out-of-office replies back to the 
provider's smtp server - when receiving fresh email from the provider 
(through getmail). Dovecot LDA tries to send the replies - but Exim 
freezes them because they don't contain the sender data in the format 
Exim wants it. Exim can either receive sender info:

1. On the command line, after the "-f" command line switch (but only 
when called by root or other users passed under "trusted_users" in 
exim.conf).
2. In the header of the email - in the "From:" field - but only, 
apparently, if it was called with the "-t" switch. Full exim command 
line documentation here: 
http://www.exim.org/exim-html-current/doc/html/spec_html/ch-the_exim_command_line.html

I can't figure out what command line options the Dovecot LDA is using 
when calling exim. I also couldn't find a way to get Dovecot LDA to pass 
extra options to exim, when trying to send email. The exim log has the 
following:

2013-05-20 15:35:15 1UeRBB-0001xc-Ar Frozen (message created with -f <>)

I've inspected the frozen message - and it has the correct sender in the 
"From:" field - but it seems that exim isn't using that, because it 
wasn't called with the "-t" option.

In dovecot.conf, I have the following for Dovecot LDA and sieve:

protocols = imap sieve

protocol lda {
   log_path = /var/log/dovecot/dovecot-deliver.log
   info_log_path = /var/log/dovecot/dovecot-deliver-info.log
   postmaster_address = admin< at >mydomain.co.uk
   hostname = mydomain.co.uk
   mail_plugins = sieve
   mail_plugin_dir = /usr/lib/dovecot
   sendmail_path = /usr/sbin/exim
}

service managesieve-login {
    inet_listener sieve {
      port = 4190
    }
}

plugin {
    sieve = ~/.dovecot.sieve
    sieve_dir = ~/sieve
}

Dovecot LDA is called by getmail using the vmail user - and using the 
same user it is trying to call exim to deliver the out-of-office replies.

I'm using Dovecot 2.2.1 with pigeonhole 0.4.0.

I can post the rest of dovecot.conf if it would help. I've read through 
the stuff at dovecot.org - but all the Dovecot LDA and exim info refers 
to Exim passing email to Dovecot using Dovecot LDA - not Dovecot LDA 
sending email out using Exim.

Re: Linking mdbox directories

Timo Sirainen — 2013-05-20T14:45:46

Yeah, that would work. Dovecot doesn't use absolute paths anywhere
internally, except for dbox-alt-root symlink. If the alt root path
changes, it logs a warning once, but other than that nothing breaks.

Alternatively you could do this using dsync still with zero downtime.
Basically treat it the same as user migration or mailbox format change,
and afterwards delete the old user's mails (e.g. doveadm expunge -u
user< at >domain mailbox '*' all) before rm -rfing the home dirs.
http://wiki2.dovecot.org/Tools/Dsync#example_converting

On Mon, 2013-05-20 at 12:28 +0100, Alex Crow wrote:

Re: dsync-2.2.2 incorrectly synchronizes subscription status of deleted mailbox

Timo Sirainen — 2013-05-20T14:33:44


Thanks, fixed: http://hg.dovecot.org/dovecot-2.2/rev/9878986a028d

(The fixed dsync protocol is still compatible with the old dsync, but
the deleted mailbox subscription states aren't synced the same until
both run new ones.)