gmane.mail.imap.dbmail

Re: IDLE Command

Reindl Harald — 2013-06-19T13:38:29


Am 19.06.2013 15:32, schrieb Paul J Stevens:

hmm - not really

our internal cms (written in PHP) has for any module a sql-scheme in an
array (field, name, type, lenght, unsigned 0/1) and calling "create.php"
as loggd in user is since the beginning there to apply any changes like
new fields and keys depending on the current active modules and options

for dbmail this should be in fact easier because there are no conditions
which fields, views and keys the db-scheme needs to have, if i would
be a c-programmer i could step in here but sadly i'm not :-(

_______________________________________________
DBmail mailing list
DBmail< at >dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

Re: IDLE Command

Paul J Stevens — 2013-06-19T13:36:56

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/19/2013 01:02 PM, Reindl Harald (mobile) wrote:


How do you propose we do that?


- -- 
________________________________________________________________
Paul J Stevens        pjstevns < at > gmail, twitter, skype, linkedin

  * Premium Hosting Services and Web Application Consultancy *

           www.nfg.nl/info< at >nfg.nl/+31.85.877.99.97
________________________________________________________________
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEARECAAYFAlHBs/MACgkQ8iITvBH4zTEsaACgyzAONZcyDenqfiIpsQ2RjsLR
wtIAmQFuvNf+Z8eRwqsHB8vWMICRnD+v
=XQJj
-----END PGP SIGNATURE-----

Re: IDLE Command

Paul J Stevens — 2013-06-19T13:32:28

Yes. It adds a field to a view.


I agree. A controlled change mechanism would be really nice to have. I
don't see an elegant way to provide it however.

One option I've looked at is sqlalchemy-migrate, but that would require
trusting schema management and only schema management to sqlalchemy,
which seems a bit overkill to do.



It is actually there alright.

commit 655827cb4d6b69f067f878817b1a60d55a9e6d7d
Author: Paul J Stevens <paul< at >nfg.nl>
Date:   Wed Jun 13 14:54:47 2012 +0200

    IMAP: better fix for SORT

    includes a minor schema migration

Re: IDLE Command

Reindl Harald — 2013-06-19T13:22:55

it is, dbmail 3.0.2 is happy too after apply

there is only one additional field in the view and
the other both are present too, applied on production
servers to not get hurt by forget it later at dbmail-updates

DROP VIEW IF EXISTS dbmail_datefield;
CREATE VIEW dbmail_datefield AS SELECT physmessage_id,datefield,sortfield FROM dbmail_messages m JOIN dbmail_header
h USING (physmessage_id) JOIN dbmail_headername n ON h.headername_id = n.id JOIN dbmail_headervalue v ON
h.headervalue_id = v.id WHERE n.headername='date';

Am 19.06.2013 10:37, schrieb Harald Leithner:

_______________________________________________
DBmail mailing list
DBmail< at >dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

Re: IDLE Command

Reindl Harald — 2013-06-19T12:00:53

to reply to mysqlf and try to explain it more

* nobody than the developer knows the scheme better

* it's hard to verify this all manually
  i had a hard time before upgrade to dbmail3 making
  sure all the keys as example in the dbmail2 database
  has the same names and keys to not fail alter / drop key

* if you find out that keys could be optimized or are useless
  and writing the code on the server side this is the perfect
  moment to manifest this internally and maintain at the same
  time a function which verifies the complete scheme

* apply it blindly and automatically may be dangerous

* *but* if dbmail would have a function which generates hardly
  recommended scheme / key changes this function could be called
  at startup, spit the recommended changes to the syslog so
  the admin is aware of it and as we are here a hint in the log
  "you should call dbmail-util --fix-scheme" and dbmail-util
  using the same code and execute the queries should be easy

keep in mind that also no longer needed keys

Re: IDLE Command

Reindl Harald (mobile — 2013-06-19T11:02:50

Uhm wouldn't it not be a good idea at least verify the scheme and spit out a big fat warning to the syslog and in general give dbmail-util a switch to apply scheme changes which should be called after any dbmail update instead deal with SQL dumps? 


-------- Original-Nachricht --------
Von: Paul J Stevens <paul< at >nfg.nl>
Gesendet: Wed Jun 19 09:54:14 MESZ 2013
An: DBMail mailinglist <dbmail< at >dbmail.org>
Betreff: Re: [Dbmail] IDLE Command

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/18/2013 01:43 PM, Reindl Harald wrote:


Ok, found it. You need to apply

sql/mysql/3_0_2-3_0_3.mysql

since a view was updated to fix sort.



- -- 
________________________________________________________________
Paul J Stevens        pjstevns < at > gmail, twitter, skype, linkedin

  * Premium Hosting Services and Web Application Consultancy *

           www.nfg.nl/info< at >nfg.nl/+31.85.877.99.97
________________________________________________________________
-----BEGIN PGP SIGNATURE-----
Versio

Re: IDLE Command

Harald Leithner — 2013-06-19T08:37:27

Is this change backward compatible?

Maybe a better sql change communication would be great, because I checked  
all commits and no one say that there is a sql structure change.

thx

Harald


Am 19.06.2013, 09:54 Uhr, schrieb Paul J Stevens <paul< at >nfg.nl>:

Re: IDLE Command

Paul J Stevens — 2013-06-19T07:54:14

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/18/2013 01:43 PM, Reindl Harald wrote:


Ok, found it. You need to apply

sql/mysql/3_0_2-3_0_3.mysql

since a view was updated to fix sort.



- -- 
________________________________________________________________
Paul J Stevens        pjstevns < at > gmail, twitter, skype, linkedin

  * Premium Hosting Services and Web Application Consultancy *

           www.nfg.nl/info< at >nfg.nl/+31.85.877.99.97
________________________________________________________________
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enigmail.net/

iEYEARECAAYFAlHBY6EACgkQ8iITvBH4zTEf6QCgzEVqTlkeS7RSmX7Z54RyqYB2
tUgAoJzG3fwNKAn+ywKg6bORqeZtIMj2
=Fd/d
-----END PGP SIGNATURE-----

Re: Reply for IMAP Mail shows no e-mail address, only real name

Paul J Stevens — 2013-06-19T07:41:03

The envelope doesn't show any problems. I've compared it with output
from dovecot, and it's identical.

So it looks like Apple Mail doesn't use the envelope, but something else
- probably something like

x UID FETCH 1:* (BODY.PEEK[HEADER.FIELDS (Subject From)])

but you'd have to run wireshark to confirm.

Re: Reply for IMAP Mail shows no e-mail address, only real name

Michael Monnerie — 2013-06-18T12:51:53

Here is another e-mail which shows destroyed headers in Apple Mail, and 
on iPhone, but Squirrelmail shows it correct. I don't see any errors. 
Maybe it's the mail client going crazy. Do you see any problems?


1 fetch * envelope
* 3 FETCH (ENVELOPE ("Thu, 11 Apr 2013 07:23:32 -0500" 
"=?Windows-1252?Q?Re:_N=D6M_Anfrage_!!?=" (("Seifert, Talin (VIE-LWW)" 
NIL "Talin.Seifert" "loweggk.at")) (("Seifert, Talin (VIE-LWW)" NIL 
"Talin.Seifert" "loweggk.at")) (("Seifert, Talin (VIE-LWW)" NIL 
"Talin.Seifert" "loweggk.at")) (("Michael Braunstorfer" NIL "mib" 
"viennapaint.com")) (("Ricarda" NIL "rr" "viennapaint.com")) NIL 
"<2021E4E6-4081-4673-9C59-B92677CDAE2C< at >viennapaint.com>" 
"<CD8C77D5.1F626%talin.seifert< at >loweggk.at>"))
1 OK FETCH completed
x fetch * rfc822.header
* 3 FETCH (RFC822.HEADER {2958}
Received: from protegate54.zmi.at (protegate54.zmi.at [212.69.164.56])  
(using TLSv1
         with cipher DHE-RSA-CAMELLIA256-SHA (256/256 bits)) (Client CN 
"protegate5.zmi.at", Issuer "power4u.zmi.at" (not verified

Re: IDLE Command

Reindl Harald — 2013-06-18T11:43:18

Am 18.06.2013 13:22, schrieb Paul J Stevens:

confirmed, not much more tests as "it does not crash" from
me currently, as said -> vacation and i should leave the
computer for mental healthy and drink some beer :-)


uhm please be more verbose with things like "I've increased the
message size so the full query should show up" that you speak
about a very recent change!

below the full query and the debug-log with maksed db-password
maybe you can guess what roundcube mail does not like with it too
_________________________________________________

[root< at >testserver:~]$ cat /var/log/dbmail.err | grep "SELECT m.message_idnr FROM dbmail_messages"
Jun 18 13:38:50 testserver.rhso dbmail-imapd[35328]: [0x7f966d04a850] Database:[db] db_query(+354):
[0x7f966d07bbf0] [SELECT m.message_idnr FROM dbmail_messages m LEFT JOIN dbmail_physmessage p ON
m.physmessage_id=p.id LEFT JOIN dbmail_datefield ON m.physmessage_id=dbmail_datefield.physmessage_id WHERE
m.mailbox_idnr = 54 AND m.status IN (0,1) ORDER BY sortfield,message_i

Re: IDLE Command

Reindl Harald — 2013-06-18T11:32:31

Am 18.06.2013 12:57, schrieb Paul J Stevens:

and both from austria :-)

used libraries:
gmime-devel-2.6.12-1.fc18.x86_64
openssl-devel-1.0.1e-4.fc18.x86_64
glib2-devel-2.34.2-2.fc18.x86_64
mhash-devel-0.9.9.9-6.fc18.x86_64
libsieve-devel-2.2.7-6.fc18.x86_64
libzdb-devel-2.8.1-3.fc18.x86_64
libevent-devel-2.0.21-4.fc18.20130612.rh.x86_64
jemalloc-devel-3.4.0-1.fc18.x86_64


my rpmmacros contains optflags with which any server-software is compiled
as well as my compiler/linker-flags are set like below in the SPEC file
any server-software (httpd, trafficserver, php, dovecot,...) is using
these flags as well as dbmail from 2012 which has no problem

maybe FORTIFY_SOURCE or stack-protector steps in, if this is the
case it is a alarm signal in case of security!

optflags: x86_64 -m64 -O3 -march=corei7 -mtune=corei7 -fopenmp -mmmx -msse2 -msse3 -msse4.1 -msse4.2 -maes -pipe
-fstack-protector --param=ssp-buffer-size=4 -mfpmath=sse -D_FORTIFY_SOURCE=2 -fexceptions

export CFLAGS="%{optflags} -fPIC -fPIE -fstack-prot

Re: IDLE Command

Paul J Stevens — 2013-06-18T11:22:22

I hope the last revision fixes this one.

Re: IDLE Command

Paul J Stevens — 2013-06-18T10:57:47

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 06/18/2013 12:14 PM, Reindl Harald wrote:

That's funny! Ask one Harald, and another answers... :-)


Oh, forgot about that one. I'm still unable to reproduce though...


Mmm, I'm using 0.7.1 from ubuntu/precise which works flawless afaict.


Unable to reproduce that one as well.


I'd like to see those queries in full. Can you do a git pull and try
again while logging to file? file_logging_levels=511

I've increased the message size so the full query should show up. It's
still truncated in syslog though, so I need the file logs.


- -- 
________________________________________________________________
Paul J Stevens        pjstevns < at > gmail, twitter, skype, linkedin

  * Premium Hosting Services and Web Application Consultancy *

           www.nfg.nl/info< at >nfg.nl/+31.85.877.99.97
________________________________________________________________
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with undefined - http://www.enig

Re: IDLE Command

Harald Leithner — 2013-06-18T10:52:22

My problem is explained in "Many database queries" thread.

But properly the same as Harald, POP3 crash on client quit.

Since I have many pop3 users (Thats good because imapd couldn't handle  
more clients) I need pop3d ;-)


Am 18.06.2013, 12:02 Uhr, schrieb Paul J Stevens <paul< at >nfg.nl>:

Re: IDLE Command

Reindl Harald — 2013-06-18T10:14:30

Am 18.06.2013 12:02, schrieb Paul J Stevens:

*wow* where should i start? :-)
well dbmail-imapd does not crash right after start but.......

with "a6e4290ba0ce37c7255bd9487232974b05ef07c9" every time a
POP3 user checks mail the service still crashs as example

roundcubemail-0.9.2 does not show me any message while in
a testfolder are 165 auto-submiited messages to verify sort
which are displayed in thunderbird
_______________________________

open a ACL-shared folder via Thunderbird/Roundcube results in
the following ines in /var/log/maillog

Jun 18 12:10:01 testserver dbmail/imap4d[15239]: Error:[db] db_query(+361): SQLException:
Jun 18 12:10:01 testserver dbmail/imap4d[15239]: Error:[db] db_query(+362): failed query [SELECT m.message_idnr
FROM dbmail_messages m LEFT JOIN dbmail_physmessage p ON m.physmessage_id=p.id LEF
Jun 18 12:10:02 testserver dbmail/imap4d[15239]: Error:[db] db_query(+361): SQLException:
Jun 18 12:10:02 testserver dbmail/imap4d[15239]: Error:[db] db_query(+362): failed query [SELECT m

Re: IDLE Command

Paul J Stevens — 2013-06-18T10:02:01


Please let me know what your problem is with the current master. I'm not
aware of any show-stopper at the moment.

Re: IDLE Command

Harald Leithner — 2013-06-18T09:44:39

Am 14.06.2013, 21:08 Uhr, schrieb Paul J Stevens <paul< at >nfg.nl>:


IIRC same as NOOP command?


Atm I turned it off. Master is not working for me atm and figured out that  
dbmail need less cpu time if I deactivate EPOLL on Debian... as mention in  
git.


Tell me a price and I will think about it.

Re: libzdb 2.11.3 & postgres bytea hex encoding

Thomas Raschbacher — 2013-06-17T07:43:18

Ok Maybe I'll give it a try.
btw fyi: I am running dbmail 3.0.2 with libzdb 2.11.1 and did briefly 
test 2.11.2 too so they work for me. (I upgraded due to some security 
fixes)

Re: libzdb 2.11.3 & postgres bytea hex encoding

Paul J Stevens — 2013-06-17T07:31:22

No idea. I havent tested libzdb 2.11 yet. It probably just means hex
output works.

libzdb 2.11.3 & postgres bytea hex encoding

Thomas Raschbacher — 2013-06-17T06:33:01

Hi.

I was about to check out latest libzdb to make a version bump ebuild 
for Gentoo and I noticed this in the latest (2.11.3) libzdb Release 
Notes:

New: Added support for the new bytea hex encoding format introduced in 
PostgreSQL 9.0.
Source: http://www.tildeslash.com/libzdb/#release_notes

Does this mean we could now use bytea_output = 'hex' with dbmail too 
(provided the libzdb version is >=2.11.3 ) or are there other 
implications within dbmail?

Regards,
   Thomas R