gmane.mail.imap.general

Re: [Imap-protocol] Re: partial fetch of BODY data

Philip Guenther — 2012-05-12T20:35:48

If the part specifier is invalid for the message, then the server may 
return NIL.

So, for example, given a message whose MIME structure is

  multipart/alternative
    text/plain
    text/html

then a request for BODY[1.TEXT] or BODY[1.HEADER] can get you a NIL.


Philip Guenther
_______________________________________________
Imap-protocol mailing list
Imap-protocol< at >u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

[Imap-protocol] Re: partial fetch of BODY data

Timo Sirainen — 2012-05-12T18:22:10


Correct.


Right.

_______________________________________________
Imap-protocol mailing list
Imap-protocol< at >u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

[Imap-protocol] Re: partial fetch of BODY data

Bill Shannon — 2012-05-12T18:18:16

Thanks for the clarifications.  Much of that wasn't clear from the spec.

And just to confirm, a server should never return "NIL" for a BODY
content, even if I attempt to read way past the end of the content,
correct?  At least, as long as the message exists.  It might return
NIL if the message has been expunged, right?

Timo Sirainen wrote on 05/12/2012 01:42 AM:

_______________________________________________
Imap-protocol mailing list
Imap-protocol< at >u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

Re: [Imap-protocol] partial fetch of BODY data

Timo Sirainen — 2012-05-12T08:42:41


This is mainly about how the server is supposed to reply to it, i.e. it should still give BODY[]<0> reply instead of BODY[]. All partial replies may be truncated, this just tries to clarify that offset=0 shouldn't be treated as a special case.


This is an instruction for client implementors about what they may assume about the replies.


Truncation means that a partial fetch ended before the requested number of bytes. BODY[] always requests all bytes.


If the client remembers that it fetched 16384 bytes (or anything more than 2817 bytes) it can assume it's at end of data.


If this was the same mail as above then the server's reply would be an empty string. But assuming a different mail then the message's size can be assumed to be 19201 bytes.


Yes. The reply may also be "" instead of {0}.


No.


A well behaving server should never return NIL to any replies. But many servers do if the fetched message has already been expunged by another session._______________________________________________
Imap-proto

[Imap-protocol] partial fetch of BODY data

Bill Shannon — 2012-05-12T07:21:35

Just to make sure I'm interpreting the spec correctly...


The spec says of a FETCH BODY request:

         Any partial fetch that attempts to read beyond the end of the
         text is truncated as appropriate.  A partial fetch that starts
         at octet 0 is returned as a partial fetch, even if this
         truncation happened.

The spec says of a FETCH BODY response:

         If the origin octet is specified, this string is a substring of
         the entire body contents, starting at that origin octet.  This
         means that BODY[]<0> MAY be truncated, but BODY[] is NEVER
         truncated.

It's not clear whether the truncation being talked about is the same in
both cases.  If it were only allowed to truncate the response at the end
of the data, "BODY[]" would be allowed to be truncated as well, so it
seems that it must be talking about truncation for some other reason.

If I do "FETCH 2 (BODY[]<0.16384>)" and the server responds
"2 FETCH (BODY[]<0> {2817}", can I assume that that's the end
of

Re: [Imap-protocol] Concurrent flag changes

Timo Sirainen — 2012-05-03T21:17:02


For the first +FLAGS vs. -FLAGS I think that's good. I'm actually going to use modseqs since adding timestamps just for this is a bit too much of a waste of space (unless there's some other reason in future that I want to add real timestamps).

But for the latter FLAGS vs. +FLAGS I'm not sure if timestamps matter at all. Assuming S1 didn't already have k2 keyword set, the "FLAGS k" change didn't explicitly tell server to remove k2. The client most likely didn't have any idea that another client had already set k2 elsewhere. So even if the syncing knows that "+FLAGS k2" was done before "FLAGS k", I'm thinking it shouldn't remove k2. So in my internal logs I'd change "FLAGS" to "-FLAGS <list of unwanted keywords>" followed by "+FLAGS <list of wanted keywords>". So in the "FLAGS k" case it might get translated to e.g. "-FLAGS k3 k6" and "+FLAGS k", which wouldn't affect "k2" at all._______________________________________________
Imap-protocol mailing list
Imap-protocol< at >u.washington.edu
http://mailman2.u.washi

Re: [Imap-protocol] Concurrent flag changes

Bron Gondwana — 2012-05-03T21:02:30

I've been thinking about how to handle that case a bit too, and my
eventual position is "all keywords start unset, so in the event of
a conflict, the act of setting the keyword wins".  I.e., the result
should be 'k k2' if you can't resolve timestamps to be certain what
what was intended.

Bron.
_______________________________________________
Imap-protocol mailing list
Imap-protocol< at >u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

Re: [Imap-protocol] Concurrent flag changes

Timo Sirainen — 2012-05-03T18:00:35


Right, so I won't change the normal client-visible behavior at least. I was only thinking about that while writing the mail, because I thought I'd try to make the question more generic. :)

Another try at what I'm really after:

You want to sync two IMAP servers periodically, but not continuously. For an extreme example lets say you have 1) your real always-connected IMAP server, which you sometimes use via webmail and 2) another IMAP server on your laptop, which you carry around and isn't always connected. It's possible that you do changes in both of these servers while they are unsynced.

Or a similar issue could happen with a shared mailbox in a multi-master type of setup during a split brain.

So you have clients that issue STORE commands to the same mail in both servers, and you need to sync them together. Both servers keep a log of what STORE commands the clients have sent, so they can intelligently merge the changes by reading the logs.

So the obvious merges for a keyword called "k" are:

S1: +FLAG

Re: [Imap-protocol] Concurrent flag changes

Timo Sirainen — 2012-05-03T17:26:58


Yes, that was unintentional. I added the \Answered later while editing my mail and forgot to add it to S2 reply.


Like Arnt mentioned, if C1 and C2 are the connections from the same client, then a) really should be used. I realized it only later while sending my mail.


If C1 and C2 are different independent clients, then there's not much of a conceptual difference between this and my b) behavior. It's all about the timing then - C2 might have sent it before C1's second STORE or it might have sent it during it. If there's less than a 1 second difference, no one can really say what happens, so I don't think it would be wrong for server to decide to behave a bit differently in such case._______________________________________________
Imap-protocol mailing list
Imap-protocol< at >u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

Re: [Imap-protocol] Concurrent flag changes

Timo Sirainen — 2012-05-03T17:18:42


Yes, but that's a client issue. And I think with STORE (UNCHANGEDSINCE) FLAGS it's clear that the client wants to do this b) :


while without CONDSTORE it's a bit more ambiguous what the client wants to do.


Not servers, but after client has issued such updates how would the servers sync them. For example I want this to at least work:

Server 1 has seen +FLAGS \Seen
Server 2 has seen +FLAGS \Flagged

When syncing the servers the result clearly should be (\Seen \Flagged). Similarly for -FLAGS. But FLAGS itself is less clear how to merge them._______________________________________________
Imap-protocol mailing list
Imap-protocol< at >u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

Re: [Imap-protocol] Concurrent flag changes

Dave Cridland — 2012-05-03T14:07:15

I think S2's response here is wrong - specifically, I can't see why  
the server shouldn't respond including \Answered from the first  
session. But I admit that very close concurrency might make that hard  
- though really, I'd find that worrying - I expect flag changes to be  
atomic.


That's very stupid. It's not said "Remove \Seen", it's said "Set the  
flags to these".


Only ever this. Anything else strikes me as fundamentally wrong.

If it responds with the second reply, I'd take that to mean that some  
other client had set \Flagged, between the server acting on my set,  
and before it responded. Surprising, but I'd cope.

Dave.

Re: [Imap-protocol] Concurrent flag changes

Arnt Gulbrandsen — 2012-05-03T11:48:48

It can also be connections C1 and C2 by the same client. If it is, C1 is
perfectly aware of what you told C2, and Dovecot is right today and
would be broken by your suggested change.

(IMO STORE FLAGS is a PITA.)

Arnt
_______________________________________________
Imap-protocol mailing list
Imap-protocol< at >u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

Re: [Imap-protocol] Concurrent flag changes

Alexey Melnikov — 2012-05-03T10:47:13

Hi Timo,

On 2 May 2012, at 22:28, Timo Sirainen <tss< at >iki.fi> wrote:


Yes.

If you want to prevent conflicts above, you need to use CONDSTORE.

Don't use STORE FLAGS when synchronizing servers ;). I don't.
 [...]

_______________________________________________
Imap-protocol mailing list
Imap-protocol< at >u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

Re: [Imap-protocol] Gmail IMAP

Brandon Long — 2012-05-02T23:47:03

If you can send me the account off list, I can take a look.

Brandon
On May 2, 2012 4:29 PM, "Jeff McKay" <jeff.mckay< at >comaxis.com> wrote:

_______________________________________________
Imap-protocol mailing list
Imap-protocol< at >u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

[Imap-protocol] Gmail IMAP

Jeff McKay — 2012-05-02T23:28:47

I wonder if any IMAP client developers have had problems using the 
SELECT statement against
Google Gmail "folders" (labels).  I thought this was working fine but 
today I was given access to
a customer's account where some folders work and some do not.  When they 
do not, the response
from the SELECT command is "NO System error (Failure)".  I don't see any 
difference between
working and non-working folders.   All the folders have messages and 
work fine using the web
client.  I get the folder names from the LIST command.  An example of a 
failed SELECT would be:

<send 25>A101 SELECT "Clearwell"
<recv 32>A101 NO System error (Failure)

A working example would be:

<send 24>A101 SELECT "Intransa"
<recv 336>* FLAGS (\Answered \Flagged \Draft \Deleted \Seen $Forwarded 
$MDNSent)
* OK [PERMANENTFLAGS (\Answered \Flagged \Draft \Deleted \Seen 
$Forwarded $MDNSent \*)] Flags permitted.
* OK [UIDVALIDITY 1315401400] UIDs valid.
* 11 EXISTS
* 0 RECENT
* OK [UIDNEXT 12] Predicted next UID.

Anybody have an idea abo

[Imap-protocol] Concurrent flag changes

Timo Sirainen — 2012-05-02T21:28:46

I don't think IMAP RFCs actually require any specific behavior for this, so this is more of a "any recommendations?" type of a question:

Clients C1 and C2 send:

C1: a FETCH 1 FLAGS
S1: * 1 FETCH (FLAGS (\Seen \Answered))
S1: a OK

C2: b STORE 1 +FLAGS (\Flagged)
S2: * 1 FETCH (FLAGS (\Seen \Flagged))
S2: b OK

At this point C1 still thinks that 1's flags are (\Seen), and being a little bit stupid it unsets the \Seen flag by sending:

C1: c STORE 1 FLAGS (\Answered)

Now, I think the possible replies are either of these:

S1: * 1 FETCH (FLAGS (\Answered))
S1: * 1 FETCH (FLAGS (\Answered \Flagged))

Dovecot currently sends the first reply, but I've started thinking that perhaps I should change it to the second one. The question is really: Should STORE FLAGS be thought of as

a) Reset all the flags that you have currently, whatever they are, and only set these flags.

or

b) Atomically add these flags I've listed, and remove those that I used to see previously in this session but aren't listed here.

In the c

Re: [Imap-protocol] BODY.PEEK[HEADER] response

Ashley Clark — 2012-04-20T20:46:17




That's the same conclusion I came to, but I wanted to make sure I wasn't overlooking some detail in the specs elsewhere.


Thanks.
Ashley

_______________________________________________
Imap-protocol mailing list
Imap-protocol< at >u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

Re: [Imap-protocol] BODY.PEEK[HEADER] response

Timo Sirainen — 2012-04-20T20:12:07


Yes, I'd say so.


BODY[HEADER] sends the message header. The RFC 822 header ends with an empty line. The next newline belongs to message body, so it can't be part of the header.

_______________________________________________
Imap-protocol mailing list
Imap-protocol< at >u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

[Imap-protocol] BODY.PEEK[HEADER] response

Ashley Clark — 2012-04-19T00:32:45

I think I may be dealing with a buggy server but I wanted to verify my assumptions first.

upon making this request to a particular server, I'm seeing this kind of response:

C: 3 fetch 1 body.peek[header]
S: * 1 FETCH (BODY[HEADER] {2006}
S: <data>
S: <data>
S: 
S: 
S: )
S: 3 OK FETCH complete

this seems like one newline too many in the response, but I can't find anything specific that dictates how many newlines are acceptable at the end of a BODY[HEADER] request.

when doing the same request against a Dovecot server I see only one trailing newline pair:

C: 3 fetch 1 body.peek[header]
S: * 1 FETCH (BODY[HEADER] {2004}
S: <data>
S: <data>
S: 
S: )
S: 3 OK FETCH complete


but is that just convention or is this server breaking some part of the spec? if so, which part?


Ashley_______________________________________________
Imap-protocol mailing list
Imap-protocol< at >u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

Re: [Imap-protocol] Unseen messages question

Antoine Nguyen — 2012-04-15T15:45:11

Le 15/04/2012 15:12, Barry Leiba a écrit :
Now that you mention it, I do :-)
As I said, it was just an *idea* about how I could achieve that...
Ok, I understand.

Indeed, my client doesn't take advantage at all of what IMAP offers. 
Reading your answer, I think I have a better view of the logic behind 
the procotol and it clearly appears that I can't create an efficient 
client without a proper caching mechanism.

Anyway, thanks for your explanation.

Antoine

_______________________________________________
Imap-protocol mailing list
Imap-protocol< at >u.washington.edu
http://mailman2.u.washington.edu/mailman/listinfo/imap-protocol

Re: [Imap-protocol] Unseen messages question

Barry Leiba — 2012-04-15T13:12:47

...

You realize that there's a race condition in that plan, right?  If a
new message arrives between the STATUS and SELECT commands, you won't
have it in your unseen count.  You could mitigate that by also
considering UIDNEXT, but that still leaves the second-client issue (if
another client makes an existing message seen or unseen between the
commands, you'll still be wrong).

The usefulness of the "first unseen" number is to allow you to limit
FETCH or SEARCH commands to that.  The right way to handle this is to
SELECT the mailbox and use the first-unseen value as bounds for a
FETCH FLAGS command or a SEARCH UNSEEN command.  Those will give you
more than just the count -- they'll also give you the message numbers
of all the unseen messages, along with any other information you
choose to collect if you do FETCH (you might also want UIDs, or
whatever).

Once you have that information, any changes made by new messages or
other clients should come in unsolicited FETCH responses (for flag
changes) and EXISTS re