gmane.linux.raid

Re: very slow file deletion on an SSD

Roberto Spadim — 2012-05-25T14:48:01

noop scheduler could be better for ssd?

2012/5/25 Joe Landman <joe.landman< at >gmail.com>:

[PATCH] imsm: fix: correct checking volume's degradation

Lukasz Dorau — 2012-05-25T13:06:41

We do not check the return value of sysfs_get_ll() now. It is wrong.
If reading of the sysfs "degraded" key does not succeed,
the "new_degraded" variable will not be initiated
and accidentally it can have the value of "degraded" variable.
In that case the change of degradation will not be checked.

It happens if mdadm is compiled with gcc's "-fstack-protector" option
when one tries to stop a volume under reshape (e.g. OLCE).
Reshape seems to be finished then (metadata is in normal/clean state)
but it is not finished, it is broken and data are corrupted.

Now we always check the return value of sysfs_get_ll().
Even if reading of the sysfs "degraded" key does not succeed
(rv == -1) the change of degradation will be checked.

Signed-off-by: Lukasz Dorau <lukasz.dorau< at >intel.com>
---
 super-intel.c |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)

diff --git a/super-intel.c b/super-intel.c
index 6c87e20..07ab9ae 100644
--- a/super-intel.c
+++ b/super-intel.c
< at >< at > -10370,8 +10370,10 < at >< at > int check_degradation_change(struct mdinfo *info,
      int degraded)
 {
 unsigned long long new_degraded;
-sysfs_get_ll(info, NULL, "degraded", &new_degraded);
-if (new_degraded != (unsigned long long)degraded) {
+int rv;
+
+rv = sysfs_get_ll(info, NULL, "degraded", &new_degraded);
+if ((rv == -1) || (new_degraded != (unsigned long long)degraded)) {
 /* check each device to ensure it is still working */
 struct mdinfo *sd;
 new_degraded = 0;

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Reason for md raid 01 blksize limited to 4 KiB?

Sebastian Riemer — 2012-05-25T12:35:57

Hi Neil,

On 22/05/12 01:28, NeilBrown wrote:

I've tested the RAID 01 with kernel 3.4 and it isn't fixed. It is even
worse, because direct IO doesn't work any more on the raid1 device (with
kernel 3.2 it worked).

There are still 4k chunks which aren't merged in the raid0 devices below
(blkparse -i md100 -i md200 -i md300 | less).
Could you also check this on your setup, please?

Cheers,
Sebastian


Btw. this is my test script:

#!/bin/bash
if [ "`lsmod | grep brd`" == "" ]; then
        modprobe brd rd_nr=4 rd_size=524288
fi
mdadm -C /dev/md100 --force --assume-clean -n 2 -l raid0 /dev/ram0 /dev/ram1
mdadm -C /dev/md200 --force --assume-clean -n 2 -l raid0 /dev/ram2 /dev/ram3
blktrace /dev/md100 &
pid=$!
dd if=/dev/zero of=/dev/md100 bs=1M oflag=direct
kill -2 $pid
blktrace /dev/md200 &
pid=$!
dd if=/dev/zero of=/dev/md200 bs=1M oflag=direct
kill -2 $pid
mv md100* r0_only/
mv md200* r0_only/

mdadm -C /dev/md300 --force --assume-clean -n 2 -l raid1 /dev/md100
/dev/md200
blktrace -d /dev/md100 -d /dev/md200 -d /dev/md300 -b 4096 &
pid=$!
# Kernel 3.4 doesn't support direct IO on the md300 device
dd if=/dev/zero of=/dev/md300 bs=1M
kill -2 $pid
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] mdadm: Fix Segmentation fault.

majianpeng — 2012-05-25T11:49:40

In function write_init_super1():
If "rv = store_super1(st, di->fd)" return error and the di is the last.
Then the di = NULL && rv > 0, so exec:
if (rv)
    fprintf(stderr, Name ": Failed to write metadata to%s\n",
      di->devname);
will be segmentation fault.

Signed-off-by: majianpeng <majianpeng< at >gmail.com>
---
 super1.c |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/super1.c b/super1.c
index 4f20cc3..37b7a90 100644
--- a/super1.c
+++ b/super1.c
< at >< at > -1242,6 +1242,8 < at >< at > static int write_init_super1(struct supertype *st)
 rv = st->ss->write_bitmap(st, di->fd);
 close(di->fd);
 di->fd = -1;
+if (rv)
+goto error_out;
 }
 error_out:
 if (rv)

Re: very slow file deletion on an SSD

Joe Landman — 2012-05-25T10:49:20

Not to my knowledge.  I'll go back and re-create some files and see.


Hmmm

Re: very slow file deletion on an SSD

Bernd Schubert — 2012-05-25T10:45:38

Hello Joe,

On 05/25/2012 12:37 PM, Joe Landman wrote:

do you have anything enabled, that would make your files to have xattrs?


http://oss.sgi.com/archives/xfs/2011-08/msg00233.html


Cheers,
Bernd
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

very slow file deletion on an SSD

Joe Landman — 2012-05-25T10:37:05

Hi folks:

   Just ran into this (see posted output at bottom).  3.2.14 kernel, MD 
RAID 5, xfs file system.  Not sure (precisely) where the problem is, 
hence posting to both lists.

  [root< at >siFlash ~]# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md22 : active raid5 sdl[0] sds[7] sdx[6] sdu[5] sdk[4] sdz[3] sdw[2] sdr[1]
       1641009216 blocks super 1.2 level 5, 32k chunk, algorithm 2 [8/8] 
[UUUUUUUU]

md20 : active raid5 sdh[0] sdf[7] sdm[6] sdd[5] sdc[4] sde[3] sdi[2] sdg[1]
       1641009216 blocks super 1.2 level 5, 32k chunk, algorithm 2 [8/8] 
[UUUUUUUU]

md21 : active raid5 sdy[0] sdq[7] sdp[6] sdo[5] sdn[4] sdj[3] sdv[2] sdt[1]
       1641009216 blocks super 1.2 level 5, 32k chunk, algorithm 2 [8/8] 
[UUUUUUUU]

md0 : active raid1 sdb1[1] sda1[0]
       93775800 blocks super 1.0 [2/2] [UU]
       bitmap: 1/1 pages [4KB], 65536KB chunk


md2* are SSD RAID5 arrays we are experimenting with.  Xfs file systems 
atop them:

[root< at >siFlash ~]# mount | grep md2
/dev/md20 on /data/1 type xfs (rw)
/dev/md21 on /data/2 type xfs (rw)
/dev/md22 on /data/3 type xfs (rw)

vanilla mount options (following Dave Chinner's long standing advice)

meta-data=/dev/md20              isize=2048   agcount=32, 
agsize=12820392 blks
          =                       sectsz=512   attr=2
data     =                       bsize=4096   blocks=410252304, imaxpct=5
          =                       sunit=8      swidth=56 blks
naming   =version 2              bsize=65536  ascii-ci=0
log      =internal               bsize=4096   blocks=30720, version=2
          =                       sectsz=512   sunit=8 blks, lazy-count=1
realtime =none                   extsz=4096   blocks=0, rtextents=0

[root< at >siFlash ~]# mdadm --detail /dev/md20
/dev/md20:
         Version : 1.2
   Creation Time : Sun Apr  1 19:36:39 2012
      Raid Level : raid5
      Array Size : 1641009216 (1564.99 GiB 1680.39 GB)
   Used Dev Size : 234429888 (223.57 GiB 240.06 GB)
    Raid Devices : 8
   Total Devices : 8
     Persistence : Superblock is persistent

     Update Time : Fri May 25 06:26:23 2012
           State : clean
  Active Devices : 8
Working Devices : 8
  Failed Devices : 0
   Spare Devices : 0

          Layout : left-symmetric
      Chunk Size : 32K

            Name : siFlash.sicluster:20
            UUID : 2f023323:6ec29eb9:a943de06:f6e0c25d
          Events : 296

     Number   Major   Minor   RaidDevice State
        0       8      112        0      active sync   /dev/sdh
        1       8       96        1      active sync   /dev/sdg
        2       8      128        2      active sync   /dev/sdi
        3       8       64        3      active sync   /dev/sde
        4       8       32        4      active sync   /dev/sdc
        5       8       48        5      active sync   /dev/sdd
        6       8      192        6      active sync   /dev/sdm
        7       8       80        7      active sync   /dev/sdf

All the SSDs are on deadline scheduler

[root< at >siFlash ~]# cat /sys/block/sd*/queue/scheduler | uniq
noop [deadline] cfq


All this said, deletes from this unit are taking 1-2 seconds per file ...

[root< at >siFlash ~]# strace -ttt -T rm -f /data/2/test/*
1337941514.040788 execve("/bin/rm", ["rm", "-f", 
"/data/2/test/2.8t-r.97.0", "/data/2/test/2.8t-r.98.0", 
"/data/2/test/2.8t-r.99.0", "/data/2/test/2.9.0"], [/* 40 vars */]) = 0 
<0.000552>
1337941514.041713 brk(0)                = 0x60d000 <0.000031>
1337941514.041927 mmap(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7bc2779000 <0.000032>
1337941514.042113 access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No 
such file or directory) <0.000109>
1337941514.042395 open("/etc/ld.so.cache", O_RDONLY) = 3 <0.000050>
1337941514.042614 fstat(3, {st_mode=S_IFREG|0644, st_size=81118, ...}) = 
0 <0.000102>
1337941514.042928 mmap(NULL, 81118, PROT_READ, MAP_PRIVATE, 3, 0) = 
0x7f7bc2765000 <0.000042>
1337941514.043078 close(3)              = 0 <0.000019>
1337941514.043235 open("/lib64/libc.so.6", O_RDONLY) = 3 <0.000115>
1337941514.043477 read(3, 
"\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\355\301W4\0\0\0"..., 
832) = 832 <0.000039>
1337941514.043647 fstat(3, {st_mode=S_IFREG|0755, st_size=1908792, ...}) 
= 0 <0.000020>
1337941514.043860 mmap(0x3457c00000, 3733672, PROT_READ|PROT_EXEC, 
MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x3457c00000 <0.000085>
1337941514.044065 mprotect(0x3457d86000, 2097152, PROT_NONE) = 0 <0.000034>
1337941514.044191 mmap(0x3457f86000, 20480, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x186000) = 0x3457f86000 <0.000034>
1337941514.044388 mmap(0x3457f8b000, 18600, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x3457f8b000 <0.000085>
1337941514.044592 close(3)              = 0 <0.000058>
1337941514.044763 mmap(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7bc2764000 <0.000039>
1337941514.044893 mmap(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7bc2763000 <0.000020>
1337941514.044981 mmap(NULL, 4096, PROT_READ|PROT_WRITE, 
MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f7bc2762000 <0.000018>
1337941514.045076 arch_prctl(ARCH_SET_FS, 0x7f7bc2763700) = 0 <0.000018>
1337941514.045183 mprotect(0x3457f86000, 16384, PROT_READ) = 0 <0.000023>
1337941514.045270 mprotect(0x345761f000, 4096, PROT_READ) = 0 <0.000019>
1337941514.045350 munmap(0x7f7bc2765000, 81118) = 0 <0.000028>
1337941514.045619 brk(0)                = 0x60d000 <0.000017>
1337941514.045698 brk(0x62e000)         = 0x62e000 <0.000018>
1337941514.045803 open("/usr/lib/locale/locale-archive", O_RDONLY) = 3 
<0.000028>
1337941514.045904 fstat(3, {st_mode=S_IFREG|0644, st_size=99158704, 
...}) = 0 <0.000017>
1337941514.046012 mmap(NULL, 99158704, PROT_READ, MAP_PRIVATE, 3, 0) = 
0x7f7bbc8d1000 <0.000020>
1337941514.046099 close(3)              = 0 <0.000017>
1337941514.046235 ioctl(0, SNDCTL_TMR_TIMEBASE or TCGETS, {B38400 opost 
isig icanon echo ...}) = 0 <0.000020>
1337941514.046373 newfstatat(AT_FDCWD, "/data/2/test/2.8t-r.97.0", 
{st_mode=S_IFREG|0600, st_size=1073741824, ...}, AT_SYMLINK_NOFOLLOW) = 
0 <0.000024>
1337941514.046504 unlinkat(AT_FDCWD, "/data/2/test/2.8t-r.97.0", 0) = 0 
<1.357571>
1337941515.404257 newfstatat(AT_FDCWD, "/data/2/test/2.8t-r.98.0", 
{st_mode=S_IFREG|0600, st_size=1073741824, ...}, AT_SYMLINK_NOFOLLOW) = 
0 <0.000072>
1337941515.404485 unlinkat(AT_FDCWD, "/data/2/test/2.8t-r.98.0", 0) = 0 
<1.608016>
1337941517.012706 newfstatat(AT_FDCWD, "/data/2/test/2.8t-r.99.0", 
{st_mode=S_IFREG|0600, st_size=1073741824, ...}, AT_SYMLINK_NOFOLLOW) = 
0 <0.000082>
1337941517.012957 unlinkat(AT_FDCWD, "/data/2/test/2.8t-r.99.0", 0) = 0 
<1.133890>
1337941518.146983 newfstatat(AT_FDCWD, "/data/2/test/2.9.0", 
{st_mode=S_IFREG|0600, st_size=8589934592, ...}, AT_SYMLINK_NOFOLLOW) = 
0 <0.000023>
1337941518.147145 unlinkat(AT_FDCWD, "/data/2/test/2.9.0", 0) = 0 <0.938754>
1337941519.086125 close(0)              = 0 <0.000102>
1337941519.086357 close(1)              = 0 <0.000061>
1337941519.086540 close(2)              = 0 <0.000021>
1337941519.086694 exit_group(0)         = ?

Anything obvious that we are doing wrong?


Machine may be occupied for a bit.  Might be a few days before we can 
get results back.

Re: Hi! Is "container" more efficient in terms of I/O op. numbers ...

Igor Podlesny — 2012-05-25T02:41:42

[...]

   Well, can it be adopted by allocation sub-array for WIB then, for e. g.?


   Yep, with some kind of updates merging it could be faster, I guess.

   BTW, as it seems the most optimal location of WIB is in the middle
of corresponding datum -- thus
seeks length are always less or equal (datum length 1/2). Have you
considered this layout? :-)


   I hope so. :)

Re: Hi! Is "container" more efficient in terms of I/O op. numbers ...

NeilBrown — 2012-05-25T02:23:52

On Fri, 25 May 2012 09:52:52 +0800 Igor Podlesny <for.poige+linux< at >gmail.com>
wrote:


Thank you for fleshing out your question a bit.  It is always useful to state
your assumptions where asking a question as it makes it easier to understand
and answer the question.

The general metadata isn't updated very often - not often enough to justify
any particular concern for where it is placed.  For a 'reshape' it can be
updated more often, but that is an unusual situation.

bitmap metadata certainly can be updated often, but there is no container
format currently supported which makes use of write-intent bitmaps, so
thinking about containers for bitmaps is not relevant.

If it were, it would make sense to keep the bitmap close to the data that it
described, so having a container arrangement would not be better than
individual arrays.  It maybe be worse depending on the particular details.

Does that make sense?

NeilBrown

Re: Hi! Is "container" more efficient in terms of I/O op. numbers ...

Igor Podlesny — 2012-05-25T01:52:52

   Well, due to disks seeks are expensive, hot data locality is
preferable and valuable thing, isn't it?
   And it makes sense not only for WIB, but other frequent metadata
updates, doesn't it?

The $1,549 per day ZERO traffic system (UPDATE) Recommends Advanced Sports to you

2012-05-25T01:33:02

Email        :sniperxsystem< at >support.com
Friend Name  :Friend
Friend Email :linux-raid< at >vger.kernel.org
comment      :

Listen to this... pretty crazy...

So many people rushed to download this 
$530k/year system yesterday...

That they crashed the ENTIRE server!

=>>http://www.sniperxsystem.com/?code=4fbea9cb964f6<<=

(The site was down ALL day) Pretty crazy. 

... The "ghetto" video alone has sent shockwaves 
through the Clickbank community.

Can you believe THIS guy's one of Clickbanks
biggest super affiliates?

=>>http://www.sniperxsystem.com/?code=4fbea9cb964f6<<=

Talk soon

P.S. This is **BRAND NEW**...

It works and it's made $1,549.87 a DAY for 
the past 739 days in a ROW!

No PPC, no PPV, no CPA, no so-called 'push
button softwares' scams, no 'loopholes'...

Something TOTALLY different.

Check it out (fast, while it's still open):

=>>http://www.sniperxsystem.com/?code=4fbea9cb964f6<<=




--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Hi! Is "container" more efficient in terms of I/O op. numbers ...

NeilBrown — 2012-05-25T00:14:24

On Thu, 24 May 2012 23:30:54 +0800 Igor Podlesny <for.poige+linux< at >gmail.com>
wrote:


I'm not sure you question exactly makes sense, but the answer would be "no"
if it did :-)

NeilBrown

Hi! Is "container" more efficient in terms of I/O op. numbers ...

Igor Podlesny — 2012-05-24T15:30:54

   ... then several "stand-alone" RAIDs on the same HDDs? -- Say, when
using write intent bitmaps.

Re: Sequential writing to degraded RAID6 causing a lot of reading

Patrik Horník — 2012-05-24T12:37:28

What I meant by should was there is theoretical way it can work that
way so it should work that way... :)

I was implicitly referring to whole stripe write.


Actually I have left-symmetric-6 without one of the "regular" drives
not the one with only Qs on it, so it should be similar to degraded
RAID6 with a left-symmetric in this regard.


OK, I did not know that. In my case I have 8 drives RAID6 degraded to
7 drives, so it would be plus to have it implemented the RAID5 way.
But anyway I was thinking the whole-stripe detection should work in
this case.


The RAID5 is 5 drives, the RAID6 arrays are 7 of 8 drives, chunk size
is 64K. I am using command dd if=/dev/zero of=file bs=X count=Y, it
behaves the same for bs between 64K to 1 MB. Actually internal read
speed from every drive is slightly higher that write speed, about cca
10%. The ratio between write speed to the array and write speed to
individual drive is cca 5.5 - 5.7.

I have enough free space on filesystem (ext3) so I guess I should be
hitting whole stripes most of the time.

Patrik

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Activate your free commission shop now Recommends Advanced Sports to you

2012-05-24T12:00:05

Email        :kimtan< at >gmail.com
Friend Name  :Friend
Friend Email :linux-raid< at >vger.kernel.org
comment      :

Hey,

Your commission shop is ready.
Activate your free commission shop now:

==> http://bit.ly/KUXNMS

In a few minutes, your commission shop will be ready, 
attracting buyers from all around the world.

And each shop is designed to suck commissions and sales 
for you automatically. 

This is the first time this revolutionary technique is applied.

So don't delay. Activate your commission shop now:

==> http://bit.ly/KUXNMS

Sincerely,
Kim Tan



--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: md RAID with enterprise-class SATA or SAS drives

Stan Hoeppner — 2012-05-24T06:16:52

Indeed they are.  Well, in a sort folder actually.  I still shouldn't
have missed them.  Please feel free to plant virtual palms to my
forehead if you like. :)

Activate your free commission shop now Recommends Advanced Sports to you

2012-05-24T06:03:16

Email        :kimtan< at >gmail.com
Friend Name  :Friend
Friend Email :linux-raid< at >vger.kernel.org
comment      :

Hey,

Your commission shop is ready.
Activate your free commission shop now:

==> http://bit.ly/KUXNMS

In a few minutes, your commission shop will be ready, 
attracting buyers from all around the world.

And each shop is designed to suck commissions and sales 
for you automatically. 

This is the first time this revolutionary technique is applied.

So don't delay. Activate your commission shop now:

==> http://bit.ly/KUXNMS

Sincerely,
Kim Tan



--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [patch 2/4] raid1: percpu dispatch for write request if bitmap supported

Shaohua Li — 2012-05-24T05:50:15

Can be a separate workqueue sharing for all md arrays.
  

Should be the same I think. For exmaple, blk_finish_plug unplug, plug_cb should
generate several bios, but for the first bio, get_request_wait sleeps and flush
plug list again. later bio hasn't a chance to be queued actually till next time
the task runs. Then unplug loses its meaning.

Thanks,
Shaohua
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Increase minimum chunk size in testsuite to 64kB

NeilBrown — 2012-05-24T05:38:14



Applied, thanks.

And thanks for testing :-)

NeilBrown

Re: [patch 2/4] raid1: percpu dispatch for write request if bitmap supported

NeilBrown — 2012-05-24T05:34:12


A separate set of per-cpu workqueues for each md array doesn't sound like a
good idea to me - if we can possibly avoid it.


My ideas was to only submit the queued writes from a normal call to
blk_finish_plug.  Calls that come from schedule would use the current
approach of signalling the md thread to take over.  There must be a way to
tell the difference - maybe just a new flag to the unplug callback.

NeilBrown

Re: Data Offset

NeilBrown — 2012-05-24T05:20:38

On Thu, 10 May 2012 19:42:10 +0200 Piergiorgio Sartor
<piergiorgio.sartor< at >nexgo.de> wrote:


I certainly won't be easy.  Though if someone did find themselves in that
situation it might motivate me to enhance mdadm in some way to make it easily
fixable.


I'm not sure.  Maybe...

With linux-3.5 and mdadm-3.3 (both unreleased) you will probably be able to
  mdadm --grow --data-offset=5M
and that will happen.  At least for RAID10.  Other levels might follow later.

Should mdadm keep them always the same?  The reason that it doesn't is that I
thought that you could change the data offset by removing each device and
adding it back as a spare with a new data_offset.  Maybe that isn't such a
good idea.

I suspect that I got a chorus of people all saying "please keep data_offset
consistent" - and particular if I received a patch which did that - then I
would probably change mdadm accordingly.

NeilBrown