gmane.linux.raid

Re: raid 10f2 vs 1 on 2 drives

William Thompson — 2012-05-23T11:25:46

Sorry about that (Chief).  Yes, I was refering to hard drives.


I was thinking about how much more head movement there would be to write the
2nd copy of the data.

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

You have been chosen: (Potential Income) Recommends Advanced Sports to you

2012-05-23T08:22:25

Email        :christine< at >yahoo.com
Friend Name  :Friend
Friend Email :linux-raid< at >vger.kernel.org
comment      :

You have been picked to be an exclusive online profits case study.

You will be taught personally from a $2,192,289 online marketer.

This is the same EXACT system.


Never before has this been done.


If you want to be one of the lucky 30 students 

of master Jamie.. be one of 30 lucky actual case studies 

right now by signing up here: 


Christine Sarrosa

The sender of this email receives compensation
when products are purchased. Results will widely vary.
This is an advertisement.


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

CONTACT DR.RUBEN LEE

UCCCON — 2012-05-23T07:57:22

KINDLY FIND ATTACHED MESSAGE AND GET BACK TO ME

[patch 4/4] raid10: percpu dispatch for write request if bitmap supported

Shaohua Li — 2012-05-23T07:26:23

In raid10, all write requests are dispatched in raid10d thread. In fast
storage, the raid10d thread is a bottleneck, because it dispatches request too
slow. Also raid10d thread migrates freely, which makes request completion cpu
not match with submission cpu even driver/block layer has such capability. This
will cause bad cache issue.

If bitmap support is enabled, write requests can only be dispatched after dirty
bitmap is flushed out. After bitmap is flushed, how write requests are
dispatched doesn't impact correctness. A natural idea is to distribute request
dispatch to several threads. With this patch, requests are added to a percpu
list first. After bitmap is flushed, then the percpu list requests will
dispatched in a workqueue. In this way, above bottleneck is removed.

In a 4k randwrite test with a 4 disks setup, below patch can provide about 95%
performance improvements depending on numa binding.

Signed-off-by: Shaohua Li <shli< at >fusionio.com>
---
 drivers/md/raid10.c |   97 ++++++++++++++++++++++++++

[patch 3/4] raid10: directly dispatch write request if no bitmap

Shaohua Li — 2012-05-23T07:26:22

In raid10, all write requests are dispatched in raid10d thread. In fast
storage, the raid10d thread is a bottleneck, because it dispatches request too
slow. Also raid10d thread migrates freely, which makes request completion cpu
not match with submission cpu even driver/block layer has such capability. This
will cause bad cache issue.

If no bitmap, there is no point to queue bio to a thread and dispatch it in the
thread. Directly dispatching bio doesn't impact correctness and removes above
bottleneck.

Multiple threads dispatch requests could potentially reduce request merge and
increase lock contention. For slow stroage, we just worry about request merge.
Caller of .make_request should already have correct block plug set, which will
take care of request merge and locking just like accessing raw device, so we
don't need worry about this too much.

In a 4k randwrite test with a 4 disks setup, below patch can provide 95% ~ 135%
performance improvements depending on numa binding.

Signed-off-by: Shaohua Li <sh

[patch 2/4] raid1: percpu dispatch for write request if bitmap supported

Shaohua Li — 2012-05-23T07:26:21

In raid1, all write requests are dispatched in raid1d thread. In fast storage,
the raid1d thread is a bottleneck, because it dispatches request too slow. Also
raid1d thread migrates freely, which makes request completion cpu not match
with submission cpu even driver/block layer has such capability. This will
cause bad cache issue.

If bitmap support is enabled, write requests can only be dispatched after dirty
bitmap is flushed out. After bitmap is flushed, how write requests are
dispatched doesn't impact correctness. A natural idea is to distribute request
dispatch to several threads. With this patch, requests are added to a percpu
list first. After bitmap is flushed, then the percpu list requests will
dispatched in a workqueue. In this way, above bottleneck is removed.

In a 4k randwrite test with a 2 disks setup, below patch can provide 10% ~ 50%
performance improvements depending on numa binding.

Signed-off-by: Shaohua Li <shli< at >fusionio.com>
---
 drivers/md/raid1.c |   96 +++++++++++++++++++++++++++++++

[patch 1/4] raid1: directly dispatch write request if no bitmap

Shaohua Li — 2012-05-23T07:26:20

In raid1, all write requests are dispatched in raid1d thread. In fast storage,
the raid1d thread is a bottleneck, because it dispatches request too slow. Also
raid1d thread migrates freely, which makes request completion cpu not match
with submission cpu even driver/block layer has such capability. This will
cause bad cache issue.

If no bitmap, there is no point to queue bio to a thread and dispatch it in the
thread. Directly dispatching bio doesn't impact correctness and removes above
bottleneck.

Multiple threads dispatch requests could potentially reduce request merge and
increase lock contention. For slow stroage, we just worry about request merge.
Caller of .make_request should already have correct block plug set, which will
take care of request merge and locking just like accessing raw device, so we
don't need worry about this too much.

In a 4k randwrite test with a 2 disks setup, below patch can provide 20% ~ 50%
performance improvements depending on numa binding.

Signed-off-by: Shaohua Li <shli< at >fu

[patch 0/4] MD: improve raid1/10 write performance for fast storage

Shaohua Li — 2012-05-23T07:26:19

In raid1/10, all write requests are dispatched in a single thread. In fast
storage, the thread is a bottleneck, because it dispatches request too slow.
Also the thread migrates freely, which makes request completion cpu not match
with submission cpu even driver/block layer has such capability. This will
cause bad cache issue. Both these are not a big deal for slow storage.

Switching the dispatching to percpu/perthread based dramatically increases
performance.  The more raid disk number is, the more performance boosts. In a
4-disk raid10 setup, this can double the throughput.

percpu/perthread based dispatch doesn't harm slow storage. This is the way how
raw device is accessed, and there is correct block plug set which can help do
request merge and reduce lock contention.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/2] Add command line argument parsing to 'test' sript

NeilBrown — 2012-05-23T03:39:32


I changed that to
       [0-9]*)
so that I can
   sh test 0
to avoid 10 and 11, or
   sh test 02r1
to just run the raid1 tests from 02.


I removed the trailing spaces here :-)

But with just those fixes I've applied both patches,
Thanks,
NeilBrown

Re: SSSE6 / RAID6

Mathias Burén — 2012-05-23T00:30:16

Aha, that makes sense now. (Neil; I meant SSSE3. Obviously SSSE3 +
RAID6 becomes SSSE6 ;))

Perhaps I can cherry pick these patches and apply them to 3.4.

Mathias
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SSSE6 / RAID6

Jim Kukunas — 2012-05-23T00:26:55

<snip>

The _SSSE3_ RAID6 recovery functions are in Neil's for-next tree. They aren't
in 3.4.

If they were, you'd see something like:

% dmesg | grep ssse3
[    0.167849] raid6: using ssse3x1 recovery algorithm

Re: SSSE6 / RAID6

NeilBrown — 2012-05-23T00:20:51

On Tue, 22 May 2012 22:27:42 +0100 Mathias Burén <mathias.buren< at >gmail.com>
wrote:


SSSE6 ? What is that?  Some other name for SSSE3 ??

The SSSE3 RAID6 code was too late for 3.4. It is in -next and should appear
in 3.5-rc1.

NeilBrown

Re: SSSE6 / RAID6

Mathias Burén — 2012-05-22T23:37:32

$ dmesg|grep -i sss
[    1.003566] sha1_ssse3: Using SSSE3 optimized SHA-1 implementation
$ dmesg|grep -i raid
[    0.240120] raid6: int64x1    333 MB/s
[    0.296660] raid6: int64x2    558 MB/s
[    0.353377] raid6: int64x4    573 MB/s
[    0.410019] raid6: int64x8    660 MB/s
[    0.466731] raid6: sse2x1     578 MB/s
[    0.523360] raid6: sse2x2    1024 MB/s
[    0.579973] raid6: sse2x4    1846 MB/s
[    0.579978] raid6: using algorithm sse2x4 (1846 MB/s)
[    1.017503] sata_mv: Highpoint RocketRAID BIOS CORRUPTS DATA on all
attached drives, regardless of if/how they are configured. BEWARE!
[    1.017509] sata_mv: For data safety, do not use sectors 8-9 on
"Legacy" drives, and avoid the final two gigabytes on all RocketRAID
BIOS initialized drives.
[    1.211831] md: raid6 personality registered for level 6
[    1.211835] md: raid5 personality registered for level 5
[    1.211840] md: raid4 personality registered for level 4
[    5.436217] md/raid:md0: device sdf1 operational as raid disk 4
[    5.436223]

Re: raid 10f2 vs 1 on 2 drives

David Brown — 2012-05-22T22:36:12

No, wear is not going to be significantly different.

You didn't say whether you are talking about hard disks (where location 
makes a difference, but "wear" on the drive motor is insignificant to 
the disk's expected lifetime), or flash disks (where people often worry 
about "wear", though location is irrelevant and wear is also irrelevant 
for most uses of all but the most cheapo disks).

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: SSSE6 / RAID6

H. Peter Anvin — 2012-05-22T22:18:20

It should be printed in the boot log (/var/log/dmesg or similar.)

-hpa

SSSE6 / RAID6

Mathias Burén — 2012-05-22T21:27:42

Hi list,

I just compiled 3.4 for my little Atom box. Is there any way I can
confirm if the RAID6 check is using SSSE6 instructions?

$ zcat /proc/config.gz |grep -i -P '(raid|sss)'
CONFIG_RAID_ATTRS=y
CONFIG_MD_RAID0=m
CONFIG_MD_RAID1=m
CONFIG_MD_RAID10=m
CONFIG_MD_RAID456=y
# CONFIG_MULTICORE_RAID456 is not set
CONFIG_DM_RAID=m
CONFIG_ASYNC_RAID6_TEST=m
CONFIG_ASYNC_RAID6_RECOV=y
CONFIG_CRYPTO_SHA1_SSSE3=y
CONFIG_RAID6_PQ=y

 $ cat /proc/cpuinfo  |grep ssse
flags: fpu vme de tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat
clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm
constant_tsc arch_perfmon pebs bts nopl aperfmperf pni dtes64 monitor
ds_cpl tm2 ssse3 cx16 xtpr pdcm movbe lahf_lm dts

$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4]
md0 : active raid6 sdf1[5] sdh1[8] sdg1[0] sde1[7] sdc1[3] sdd1[4] sdb1[9]
      9751756800 blocks super 1.2 level 6, 64k chunk, algorithm 2
[7/7] [UUUUUUU]
      [>....................]  check =  0.7% (15577860/1950351360)
finish=1918.9min speed

raid 10f2 vs 1 on 2 drives

William Thompson — 2012-05-22T19:33:40

I understand that raid 10 f2 is slower on writes due to the location of the
2nd copy.  My question is, if lots of writes are performed, could this
layout wearout the drives quicker than raid 1?
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/2] Add command line argument parsing to 'test' sript

2012-05-22T16:55:30

From: Jes Sorensen <Jes.Sorensen< at >redhat.com>

This adds more generic command line argument parsing to the test
script. It also introduces a couple of new options, while preserving
the old '<prefix>' and 'setup' arguments. The new options are
--disable-multipath and --tests=<test1>,<test2>,...

Signed-off-by: Jes Sorensen <Jes.Sorensen< at >redhat.com>
---
 test |   88 ++++++++++++++++++++++++++++++++++++++++++++++++-----------------
 1 files changed, 65 insertions(+), 23 deletions(-)

diff --git a/test b/test
index 0679983..9428b20 100755
--- a/test
+++ b/test
< at >< at > -8,9 +8,6 < at >< at > then echo >&2 "test: testing can only be done as 'root'."
 fi
 
 prefix='[0-9][0-9]'
-if [ -n "$1" ]
-then prefix=$1
-fi
 
 dir=`pwd`
 mdadm=$dir/mdadm
< at >< at > -94,10 +91,6 < at >< at > ulimit -c unlimited
 echo 2000 > /proc/sys/dev/raid/speed_limit_max
 echo 0 > /sys/module/md_mod/parameters/start_ro
 
-if [ " $1" = " setup" ]
-then trap 0 ; exit 0
-fi
-
 # mdadm always adds --quiet, and we want to see any unexpected messages
 mdadm() {
     rm -f $target

[PATCH 1/2] Check for multipath module before running multipath tests

2012-05-22T16:55:29

From: Jes Sorensen <Jes.Sorensen< at >redhat.com>

Some systems do not ship the md multipath module. If not available
simply skip any multipath tests.

Signed-off-by: Jes Sorensen <Jes.Sorensen< at >redhat.com>
---
 test              |    6 ++++++
 tests/00multipath |    5 +++++
 2 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/test b/test
index 4f0979e..0679983 100755
--- a/test
+++ b/test
< at >< at > -19,6 +19,12 < at >< at > then
    echo >&2 "test: $mdadm isn't usable."
 fi
 
+# Check whether to run multipath tests
+modprobe multipath 2> /dev/null
+if grep -s 'Personalities : .*multipath' > /dev/null /proc/mdstat ; then
+    MULTIPATH="yes"
+fi
+
 # assume md0, md1, md2 exist in /dev
 md0=/dev/md0 md1=/dev/md1 md2=/dev/md2
 mdp0=/dev/md_d0
diff --git a/tests/00multipath b/tests/00multipath
index bc0429f..84e4d69 100644
--- a/tests/00multipath
+++ b/tests/00multipath
< at >< at > -2,6 +2,11 < at >< at >
 #
 # create a multipath, and fail and stuff
 
+if [ "$MULTIPATH" != "yes" ]; then
+  echo -ne 'skipping... '
+  exit 0
+fi
+
 mdadm -CR

[PATCH 0/2] Test suite enhancements

2012-05-22T16:55:28

From: Jes Sorensen <Jes.Sorensen< at >redhat.com>

Hi,

Two fixes for the testsuite. The first is just to allow it to run on
systems which do not include the multipath module. The second adds
command line arguments to 'test'. In particular it makes it possible
to specify explicit tests to run. I believe I managed to preserve the
behavior for old style arguments.

Cheers,
Jes


Jes Sorensen (2):
  Check for multipath module before running multipath tests
  Add command line argument parsing to 'test' sript

 test              |   94 ++++++++++++++++++++++++++++++++++++++++-------------
 tests/00multipath |    5 +++
 2 files changed, 76 insertions(+), 23 deletions(-)

You have been chosen: (Potential Income) Recommends Advanced Sports to you

2012-05-22T14:08:51

Email        :christine< at >yahoo.com
Friend Name  :Friend
Friend Email :linux-raid< at >vger.kernel.org
comment      :

You have been picked to be an exclusive online profits case study.

You will be taught personally from a $2,192,289 online marketer.

This is the same EXACT system.


Never before has this been done.


If you want to be one of the lucky 30 students 

of master Jamie.. be one of 30 lucky actual case studies 

right now by signing up here: 


Christine Sarrosa

The sender of this email receives compensation
when products are purchased. Results will widely vary.
This is an advertisement.


--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html