gmane.mail.imap.dbmail

Re: Migration of Data from 2.2.17 to 3.0.2 (dbmail-util coredumping)

Alan Hicks — 2012-05-24T17:10:33

On 24/05/2012 16:28, Daniel Schütze wrote:
I've just submitted an upgrade request, 
http://www.freebsd.org/cgi/query-pr.cgi?pr=168308

Alan Hicks

Re: Migration of Data from 2.2.17 to 3.0.2 (dbmail-util coredumping)

Alan Hicks — 2012-05-24T17:07:48

Resending as it does not appear to have arrived!

On 24/05/2012 10:32, Alan Hicks wrote:
Submitted as http://www.freebsd.org/cgi/query-pr.cgi?pr=168308
_______________________________________________
DBmail mailing list
DBmail< at >dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

Re: Migration of Data from 2.2.17 to 3.0.2 (dbmail-util coredumping)

Daniel Schütze — 2012-05-24T15:28:24

Yes I ran it without error (took 24 hrs on the test box mind you!).


I hadn't appreciated that at the time, though it is clearly laid out in the
docs which I saw when I re-read them.  I will want to migrate the messages
in due course to free up space of course.



I am using the freebsd ports and the latest version available is 2.4.24

Given the whole run is a test anyway I've started from scratch after making
some adjustments esp. as I want to see the system in action above rebuilding
the headers but before the message migration, then I'll try the migration
and see if that gives rise to the same issue.

Daniel Schütze

Re: Migration of Data from 2.2.17 to 3.0.2 (dbmail-util coredumping)

Paul J Stevens — 2012-05-24T07:52:40

Did you run the full schema migration successfully first?


Running dbmail-util -M is not required and does not require downtime.
The data-store is backward compatible - only body searches will not work
on messageblks.

That said: I haven't seen this problem yet.

Since the core dump seems to originate from gmime, I would advise you to
upgrade to the latest gmime-2.4. I see rfc2047 related bugfixes at least
in 2.4.30, and probably later ones as well.

Re: switch to GMime-2.6 for development

Reindl Harald — 2012-05-23T17:50:03


Am 23.05.2012 19:46, schrieb Alan Hodgson:

your disks maybe

mine in a HP SAN-Storage are not nor is it the overhead
on upgrades, overhead on backup-space and so on

the "disks are cheap" is a urban legend from homeusers
calculatiing with their cheap SATA disks :-)

having 400 MB more or less on at least three build machines
and testing servers makes really a difference on dist-upgrades
twice a year, on normal updates and consumed time at all

_______________________________________________
DBmail mailing list
DBmail< at >dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

Re: switch to GMime-2.6 for development

Alan Hodgson — 2012-05-23T17:46:15

Well, no offense, but disk is cheap. Give mock 20GB and you can build anything.

Re: switch to GMime-2.6 for development

Reindl Harald — 2012-05-23T17:38:34


Am 23.05.2012 19:24, schrieb Alan Hodgson:

in theorey

with dbmail2.2 i rebuilt gmime22 on our build-environment
a rebuild of the current gmime versions pulls a ton of
BuildRequirements including crap like mono with a summary
around 300 MB

that said: our main server needs 2.0 GB for the rootfs
the specialized ones 1.3 GB

_______________________________________________
DBmail mailing list
DBmail< at >dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

Re: switch to GMime-2.6 for development

Alan Hodgson — 2012-05-23T17:24:39

Not much else really uses gmime. You can just rebuild FC17's and stick it in 
your local package repo.

Re: switch to GMime-2.6 for development

Reindl Harald — 2012-05-23T17:19:08


Am 23.05.2012 18:27, schrieb Jorge Bastos:

no, thanks :-)

no .deb after 10 years Redhat
no back to sysv after all the pain with systemd at begin

and no replacement of > 20 virtual servers with a internal
build/update/test-infrastructure and not after 7 successfull
dist-upgrades in this environment from F9 to F16 since 2008

not for one package and running the mailsevrer another
software as the rest leads in multiple work :-)

_______________________________________________
DBmail mailing list
DBmail< at >dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

Re: switch to GMime-2.6 for development

Jorge Bastos — 2012-05-23T16:27:50

Go Debian :)

Works fine for me, and 2.6 is only for the master not the stable 3.x

switch to GMime-2.6 for development

Reindl Harald — 2012-05-23T16:22:03

Hi


i am fine with this

but


http://koji.fedoraproject.org/koji/packageinfo?packageID=325
Fedora 17 (released in two weeks is the first one with 2.6)

i do not like to upgrade to F17 until it is hardly needed because
currently are too much php-pecl extensions still not supporting
PHP 5.4 (the most important suhosin)

please do not require gmime 2.6 too soon because Fedora
is bleeding edge and other distributions are behind

_______________________________________________
DBmail mailing list
DBmail< at >dbmail.org
http://mailman.fastxs.nl/cgi-bin/mailman/listinfo/dbmail

Re: Sieve script vacation not working

Paul J Stevens — 2012-05-23T09:46:33

You need to specify what addresses you want to respond to. Also you
subject match always matches and is therefor redundant:

require "vacation";
if true {
  vacation :days 7 :addresses [leandro< at >texnet.it]
    :subject "Automatic response" "body text";
}

Sieve script vacation not working

Leandro — 2012-05-22T15:56:04

I am using dbmail 3.0 and while sieve scripts work fine, I can't use the
vacation script. I tried a very simple one like:

require ["vacation"];
   if header :matches "subject" "*" {
       vacation :subject "Automatic response"
                "I'm away -- send mail to foo in my absence";
   }

But it doesn't work. I get the following error in the debug log:

May 22 17:24:37 dbmail32 dbmail-lmtpd[4512]: [0x9bc3fb8] Debug:[sort]
sort_debugtrace(+625): sieve: [sv_interface,script.c,libsieve_eval:
[VACATION aborted: no match found in script.]

Any idea? Can you provide me your sieve script for example?

Leandro

Re: Migration of Data from 2.2.17 to 3.0.2 (dbmail-util coredumping)

Daniel Urstöger — 2012-05-22T13:37:06

Hi Daniel,

just came to my mind: did you run dbmail util on your orginal storage/version?
Maybe there is a problem in the dataset that might get fixed this way?

Otherwise I think you have to wait till Paul gets on this topic.

Greetings,
Daniel

Am 22.05.2012 um 15:02 schrieb Daniel Schütze:

Re: Migration of Data from 2.2.17 to 3.0.2 (dbmail-util coredumping)

Daniel Schütze — 2012-05-22T13:02:36

Actually it's now gotten to the point where dbmail-util -My doesnt run for
more than a couple of seconds before I get the errors I'm encountering.  I'm
not getting so many core dumps as just simple aborts to the command line:

May 22 13:56:22 dbm3 dbmail-util[14952]: [0x80220b040] Error:[db]
db_exec(+319): SQLException: Cannot add or update a child row: a foreign
key constraint fails (`dbmail`.`dbmail_partlists`, CONSTRAINT
`dbmail_partlists_ibfk_1` FOREIGN KEY (`physmessage_id`) REFERENCES
`dbmail_physmessage` (`id`) ON DELETE CASCADE ON UPDATE CASCADE)
May 22 13:56:22 dbm3 dbmail-util[14952]: [0x80220b040] Error:[db]
db_exec(+320): failed query [INSERT INTO dbmail_partlists (physmessage_id,
is_header, part_key, part_depth, part_order, part_id) VALUES
(657895,1,1,0,0,27)]
migrating physmessage_id: 657895
failed

From the messages (phymessage_id) affected it is clear that the frequency is
getting worse and high enough that the conversion exercise now seems futile.

Clearly I'll need to know/figur

Re: Migration of Data from 2.2.17 to 3.0.2 (dbmail-util coredumping)

Daniel Schütze — 2012-05-22T11:17:51

Just as an update to my message, it occurred to me the foreign key error
could well be a result of the core dump and nothing to do with what was
causing the error.

 

Im a little out of my depth here, but a look at the core dumps seems to
point towards a problem around gmime (using gmime-24-2.4.24), is that
possible?

 

#0  0x0000000801e15786 in memcpy () from /lib/libc.so.7

[New Thread 8024b61c0 (LWP 100286)]

[New Thread 8024041c0 (LWP 100208)]

(gdb) where

#0  0x0000000801e15786 in memcpy () from /lib/libc.so.7

#1  0x00000008007dfe51 in rfc2047_encode ()

   from /usr/local/lib/libgmime-2.4.so.6

#2  0x00007fffffffb740 in ?? ()

#3  0x00007fffffffb670 in ?? ()

#4  0x0000000801e1f41c in __uppercase_hex () from /lib/libc.so.7

#5  0x200a3e2200000001 in ?? ()

 

(retrieved from gdb /usr/local/sbin/dbmail-util *.core; where)

 

And a second dump which occurred as I was writing this e-mail

 

(gdb) where

#0  0x0000000801e12786 in memcpy () from /lib/libc.so.7

#1  0x00000008007dc911 in stream_read

Migration of Data from 2.2.17 to 3.0.2 (dbmail-utilcoredumping)

Daniel Schütze — 2012-05-22T09:51:15

Im attempting a migration of a Dbmail 2.2.17 installation to 3.0.2 with a
backup of our production database on a spare machine to test the procedure.


Our dbmail installation contains some 900k messages and is some 360gig
(data; index 1.9gig).  The primary reason for wishing to migrate is for the
single instance storage of mime-parts to hopefully reduce the storage space
significantly (when I tried the same migration last year with the RC there
was a reduction by some 60%).

 

I am at the stage of running dbmail-util My to convert the message
attachments to the new storage system and am hitting a brick wall in the
sense that dbmail-util is repeatedly coredumping.  If I run the dbmail-util
command immediately again I receive foreign key errors:

 

May 22 10:34:35 dbm3 dbmail-util[6041]: [0x80240b040] Error:[db]
db_exec(+319): SQLException: Cannot add or update a child row: a foreign key
constraint fails (`dbmail`.`dbmail_partlists`, CONSTRAINT
`dbmail_partlists_ibfk_1` FOREIGN KEY (`physmessage_id`) RE

innodb tweaks

Kamenik, Aleksander — 2012-05-18T13:22:23

Hi,

Has anybody done any performance and storage tests for innodb with regard to dbmail 3.0. Like using compression and partitioning on the email data tables?

https://dev.mysql.com/doc/refman/5.5/en/innodb-compression.html
https://dev.mysql.com/doc/refman/5.5/en/partitioning.html

Also, any recommendations on setting the various innodb buffer sizes, that deviate from the standard recommendations? Maybe someone can simply share their experiences with tweaking innodb on a dbmail install.

Regards,

Aleksander Kamenik

Re: Housekeeping dbmail_headervalue

ITronic Harald Leithner — 2012-05-15T15:14:39

Hi Leandro,

there is already a good solution for this:  
http://www.gossamer-threads.com/lists/dbmail/users/31530#31530

But I think Paul has other things to develop, may you could write the code  
that do this?

bye

HArald

Am 15.05.2012, 17:08 Uhr, schrieb leandro <leandro< at >texnet.it>:

Re: Housekeeping dbmail_headervalue

Reindl Harald — 2012-05-15T15:11:59

yes this is a missing piece in dbmail-util and should be
done manually this time because this table had 80% of all
recods here some weeks ago

Am 15.05.2012 17:08, schrieb leandro:

Housekeeping dbmail_headervalue

leandro — 2012-05-15T15:08:29

Hello,
am I wrong or there is no dbmail-util procedure to delete headervalues
no more connected to messages?

After deleting some users directly from the dbmail_database and running
the dbmail-util -ay housekeeping utility, I noticed the
dbmail_headervalue table size remains pretty the same.

If I run manually the query:

select count(*) from dbmail_headervalue where id not in ( select
headervalue_id from dbmail_header );

I discovered a large number of no more connected header values.

Almost the same if I check the headernames with the query:

select count(*) from dbmail_headername where id not in ( select
headername_id from dbmail_header );

Leandro