gmane.linux.raid

Re: raid10 physical layout?

Roy Sigurd Karlsbakk — 2013-05-25T13:31:33

You also may want to consider using LVM instead of md RAID-10, since the latter isn't very flexible - you can't add new drives or otherwise grow raid-10 (atm).

Vennlige hilsener / Best regards

roy
--
Roy Sigurd Karlsbakk
(+47) 98013356
roy< at >karlsbakk.net
http://blogg.karlsbakk.net/
GPG Public key: http://karlsbakk.net/roysigurdkarlsbakk.pubkey.txt
--
I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et elementært imperativ for alle pedagoger å unngå eksessiv anvendelse av idiomer med xenotyp etymologi. I de fleste tilfeller eksisterer adekvate og relevante synonymer på norsk.
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: "Missing" RAID devices

Stan Hoeppner — 2013-05-25T01:42:01

Yet another invalid, nonsensical argument...


And you might find a polar bear or two living in the tropics.


The World According to Keld.  Most laptops can only house one drive.  If
they held two or more their on battery time would be useless.  Most
desktops are sold with only one drive.  Yes, in a perfect world everyone
would be redundant.


Properly configured servers don't swap.  When they need to swap it's
typically because something has gone wrong.  When this happens they need
to free pages as quickly as possible.  Once the problem no longer
exists, the speed with which pages are brought back from swap isn't
critical.

Please put down the shovel.  Your arguments keep digging you into a
deeper hole.  I'm not sure if you'll ever be able to climb out.

Re: Moving RAID1 disks between systems

Jim Santos — 2013-05-24T21:58:21

Is there a way to rename md devices?  It would be useful in general and
maybe necessary if assembling the arrays from the other system causes
md device name collisions
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: "Missing" RAID devices

2013-05-24T19:22:58

And the cost to select, buy and install RAM is much more than USD 10.
And some systems dont have room for more RAM, etc.


Some laptop users and desktop users run raid10. I think all laptop
and desktop users should have at least 2 disks and run mirrorred raid on them.
Both for security and for performance.

Server performance for properly configured servers will benefit from
snappy swap read performance. Write performance of swap would normally 
not be noticable - not a bottleneck.

best regards
keld
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: raid10 physical layout?

Phil Turmel — 2013-05-24T19:13:41


Just alternate the manufacturers when you list the devices in the
"--create" operation.  MD lays out raid10,f2 redundant data on adjacent
disks.  (Wraps around.)

Phil
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: "Missing" RAID devices

Stan Hoeppner — 2013-05-24T19:05:44

If a single user system has multiple drives configured in RAID10 and
productivity applications are being swapped, then the user should be
smacked in the head.  2GB DIMMs are $10.  Any hard drive is $50+ but
usually much more.

This is not a valid argument.


Neither is this.  Laptop users don't run RAID10.  And server swap
performance is all about page write, not read, as I previously stated.

raid10 physical layout?

Matt Garman — 2013-05-24T18:49:30

I'm looking into building a six-disk raid10.  The six disks are
comprised of three from one manufacturer, and three from another.

My intuition says that for better statistical reliability, the
redundant data copies ought to fall on disks of different
manufacturers.  I know I could achieve this easily by building a
"classic" RAID1+0 system: three RAID-1 sets built by
manufacturerA+manufacturerB pairs, then striping those mirrored sets
in RAID-0.

But based on what I've been reading, looks like mdadm's more
sophisticated raid10 layout schemes give better performance.  For
example, can I create a raid10,f2 set in such a way as to meet my
"redundant copies on different manufacturer" criteria?

Thanks,
Matt

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Moving RAID1 disks between systems

Drew — 2013-05-24T18:30:05

Hi Jim,

You should have no troubles reassembling the array on a new machine.
From what I understand, md raid doesn't depend on device names for
assembling arrays, it has some internal UUID's to distinguish which
array a member disk belongs to.

mdadm --assemble --scan is pretty much all you need (and won't break
anything in the process)

Re: "Missing" RAID devices

2013-05-24T18:03:30

Agreed


Yes, the kernel will handle striping, but not mirrored, if you do not employ raid.. 


yes, it is possible, and why not do it, swap is mostly a very small
part of the total disk space, so that seems to be very cheap, and then also
giving identical disk layout in many situations.


I think it is serious that a process, or a number of processes fail because of failing
disks. And it does not cost much disk space to prevent against this. It does
cost double/triple write IO, but that is probably worth it too.

I do think having a uniform disk space with raid0 reading property does
speed up the reading. The kernel cannot evenly spread IO over the disks,
as the chunks it needs to read may be different in size. raid10,far automatically
does this even spread. And if you need mirrored raid, then no other mirrored
raid types give you raid0 read speed.



Raid1 is only half as  fast as raid10,far for single reads..


The IO scheduling thakes care of latency problems, grouping the
right tracks together for the write tasks for the far layout.

yes for far layout and small partitions like swap, the difference
between the speed of inner and outer tracks are insignificant.


I have examples of large swaps like firefox and flash

raid1 and raid10,f2 are about the same for sequential write, which is what
is used for swap write io. Single read speed is far better for the far layout.
So why choose the slower raid1?
https://raid.wiki.kernel.org/index.php/Performance

best regards
keld
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Самое подходящее предложение для обычников нашего сайта

www.vikusik0912 — 2013-05-24T16:48:19

самые большие скидки на нынешний час http://goo.gl/xKRsP?/LGSceA
Молвил: <<Ты бы, сынка, лучше Мы рады вам!

Re: "Missing" RAID devices

2013-05-24T17:15:25

In my experience read performance from swap is critical, at least 
on single user systems. Eg swapping in firefox  or libreoffice 
may take quite some time and there raid10,far helps by almost halfing
the time for the swapping in. writes are not important, as long as you are not trashing.
In general halfing the swapping in with raid10,far is nice for a process, but 
for small processes it is not noticable for a laptop user or a 
server user, say http or ftp.

best regards
keld
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Moving RAID1 disks between systems

Jim Santos — 2013-05-24T16:53:31

Hi,

Let's say that two people each have a system with a RAID1 array made
up of two disks
(/dev/sda & /dev/sdb).  Now let's say that one of the systems
experiences a failure that
keeps it from being able to be started (e.g. blown motherboard).  Is
there a way to get the
data off of the failed system's disks after putting them in the
working system?  It seems to
me that the disk device names stored in the superblock are going to
conflict with the device
names of the existing disks.  But then again, I'm not a SW RAID expert ;-)

TIA
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Attempt to change raid1 to raid0 results in division error in kernel

NeilBrown — 2013-05-24T11:27:03

On Thu, 23 May 2013 11:44:00 +0100 Robert Goliasz
<r.goliasz< at >digital-science.com> wrote:

....

Fixed by commit 24b961f811a3e790a9b93604d25 which is in linux-3.4


http://git.kernel.org/cgit/linux/kernel/git/torvalds/linux.git/commit/?id=24b961f811a3e790a9b93604d25

NeilBrown

Re: Attempt to change raid1 to raid0 results in division error in kernel

Robert Goliasz — 2013-05-24T09:39:40

Hi Stan,

Thanks for your reply.

The point of my thread was to point out that a userland tool shouldn't
be able to cause a kernel to crash. It also shouldn't segfault when given
incorrect arguments.

If conversion from raid1 to raid0 is supported but I didn't specify
the right arguments, I should get an appropriate error message. If it's
not supported, then I should also get an appropriate error message.
My machine shouldn't crash because of me giving wrong arguments to mdadm.

Another, smaller, issue is that if conversion from raid 1 to raid 0 is
not supported then perhaps the mdadm manpage should be clearer about it,
rather saying "Currently supported growth options [...] [are] changing the
RAID level between 0, 1, 5, and 6"


This doesn't seem to be correct, I'm able to at least create a RAID0 stripe
without specifying the chunk size explicitly:

  root< at >gbl-macbook:~# mdadm --create /dev/md1 -l 0 -n 2 /dev/vg0/v1 /dev/vg0/v2
  mdadm: /dev/vg0/v1 appears to be part of a raid array:
      level=raid1 devices=3 ctime=Thu May 23 11:13:33 2013
  Continue creating array? y
  mdadm: Defaulting to version 1.2 metadata
  mdadm: array /dev/md1 started.
  root< at >gbl-macbook:~# cat /proc/mdstat 
  Personalities : [raid0] 
  md1 : active raid0 dm-4[1] dm-3[0]
        203776 blocks super 1.2 512k chunks
        
  unused devices: <none>
  root< at >gbl-macbook:~# 



I knew exactly what I was doing. I was testing whether conversion from raid1
to raid0 was supported. I got an unexpected result in the form of a kernel
crash. After discussing on IRC I was asked to report this here.

I've concluded that it's not supported or broken and will not attempt to run
this on a production machine.

Best size partition for a mdadm array

Andrea Bolandrina — 2013-05-24T09:25:11

Hi All,

In choosing the right size for a mdadm array the manual sais:
"Sometimes a replacement drive can be a little smaller than the
original drives though this should be minimised by IDEMA standards.
Such a replacement drive will be rejected by md.
To guard against this it can be useful to set the initial size
slightly smaller than the smaller device with the aim that it will
still be larger than any replacement."

I want to do a 5x2TB md (raid 6) array .
Could you advise me how "slightly smaller" should my partitions be?

100MB smaller than the actual size? 1GB?

Thank in advance for the attention.

Regards,
Andrea
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: "Missing" RAID devices

David Brown — 2013-05-24T09:23:30

I think you are getting a number of things wrong here.  For general
usage, especially on a two disk system, raid10,f2 is very often an
excellent choice of setup - it gives you protection (two copies of
everything) and fastreads (you get striped read performance, and always
from the faster outer half of the disk).  You pay a higher write latency
compared to plain raid1, but with typical usage figures of 5 reads per
write, that's fine.  And normally you don't have to wait for writes to
finish anyway.

But swap is different in many ways.

First, the read/write ratio for swap is much closer to 1 - it can even
be lower than 1.  (Things like startup code for programs can get pushed
to swap and never read again, as can leaked memory from buggy programs.)

Secondly, write latency is a big factor - data is pushed to swap to free
up memory for other usage, and that has to wait until the write is complete.

Thirdly, the kernel will handle striping of multiple swap partitions
automatically.  And it will do it in a way that is optimal for swap
usage, rather than the chunk sizes used by a striped raid system.  (More
often, the kernel wants parallel access to different parts of swap,
rather than single large reads or writes.)


One thing that seems to be slightly confused here in this thread is the
mixup between the number of mirror copies and the number of drives in
raid10 setups.  With md raid, you can have as many mirrors as you like
over as many drives as you like, though you need at least as many
partitions as mirrors (and it seldom makes sense to have more mirrors
than drives).  For example, if you have 3 disks, you can use "far3"
layout to get three copies of your data - one copy on each disk.  But
you can also use "far2", and get two copies of your data.  See
<http://en.wikipedia.org/wiki/Non-standard_RAID_levels#Linux_MD_RAID_10>
for some pictures.

With plain raid1, if you use 3 drives you get three copies.

It seems unlikely to me that you would need the "safe against two disk
failure" protection of 3-way mirrors on swap, but it is possible.


Back to swap.

If you don't need protection for your swap (swap should not often be in
use, and a dead disk will lead to crashes on swapped-out processes but
should not cause more problems than that), put a small partition on each
disk, and add them all to swap.  The kernel will handle striping of the
swap partitions.  There is nothing you can do to make it faster.

When you want protection, raid1 is your best choice.  Make small
partitions on each disk, then pair them up as a number of raid1 pairs,
and add each of these as swap.  Your system will survive any disk
failure, or multiple failures as long as they are from different pairs.
 Again, there is nothing you can do to make it faster.


The important factor here is to minimise write latency.  You do that by
keeping the layers as simple as possible - raid1 is simpler and faster
than raid10 on two disks.  With small partitions, head movement and the
bandwidth differences between inner and outer tracks makes no
difference, so "far" layout is no benefit.

Theoretically, a set of raid10,f2 pairs rather than raid1 pairs would
allow faster reading of large chunks of swap - assuming, of course, that
the rest of the system supports such large I/O bandwidth.  But such
large streaming reads do not often happen with swap - more commonly, the
kernel will jump around in its accesses.  Large reads that use all
spindles are good for the throughput for large streamed reads, but they
also block all disks and increase the latency for random accesses which
are the common case for swap.


I'm a great fan of raid10,f2 - I think it is an optimal choice for many
uses, and shows a power and flexibility of Linux's md system that is
well above what you can get with hardware raid (or software raid on
other OS's).  But for swap, you want raid1.




--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: "Missing" RAID devices

Stan Hoeppner — 2013-05-24T07:37:01

On most of today's systems, read performance is largely irrelevant WRT
swap performance.  However write performance is critical.  None of the
md/RAID10 layouts are going to increase write throughput over RAID1
pairs.  And all the mirrored RAIDs will be 2x slower than interleaved
swap across direct disk partitions.

Re: "Missing" RAID devices

2013-05-24T06:32:28

I think it is you who are not fully aquainted with Linux MD. Linux 
MD RAID10,far3 offers improved performance in single read, which is an
advantage for swap, when you are swapping in. Thinkk about it and try it out for yourself.
Especially if we are talking 3 drives (far3), but also when you are
talking more drives and only 2 copies. You don't get raid0 read performance in Linux
on a combination of raid1 and raid0.

best regards
keld
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: "Missing" RAID devices

Stan Hoeppner — 2013-05-24T03:45:56

...

As I mention above, none of the md/RAID10 layouts will yield any added
performance benefit for swap partitions.  And I state the reason why.
If you think about this for a moment you should reach the same conclusion.

Re: Attempt to change raid1 to raid0 results in division error in kernel

Stan Hoeppner — 2013-05-24T01:40:13

I don't even know if this is possible.

...

If it is possible, this command line won't work because you didn't
specify a chunk size for the RAID0 stripe.  Every striped array type
requires a chunk size.

...


Well of course.  Experimenting with mdadm without knowing what you're
doing will often result in lost/corrupted arrays and other forms of damage.

At this point you should simply start a new thread and ask:

"How do I convert a RAID1 array to RAID0?"

or

"How do I convert a RAID1 array to RAID10?"

instead of working backwards from your mistakes here.

Attempt to change raid1 to raid0 results in division error in kernel

Robert Goliasz — 2013-05-23T10:44:00

Hi,

When attempting to change raid level from 1 to 0 (I was doing some testing
trying to convert a raid1 array to a raid10 one), I got a division error.

I'm on Debian sid, output of uname -a is:

  Linux gbl-macbook 3.2.0-4-amd64 #1 SMP Debian 3.2.35-2 x86_64 GNU/Linux

I had the following raid set up (using devices on LVM):

  gbl-macbook# cat /proc/mdstat 
  Personalities : [raid1] [raid0] [raid10] 
  md0 : active raid1 md1[2] md2[1]
        409280 blocks super 1.2 [2/2] [UU]
        
  md2 : active raid0 dm-8[1] dm-7[0]
        1022976 blocks super 1.2 512k chunks
        
  md1 : active raid0 dm-6[1] dm-5[0]
        1022976 blocks super 1.2 512k chunks
        
  unused devices: <none>

/dev/md0 was a mounted ext4 filesystem.

I attempted to run this:

  gbl-macbook# mdadm --grow /dev/md0 -l 0
  zsh: segmentation fault  mdadm --grow /dev/md0 -l 0
  gbl-macbook# 

All my terminals have shown the following alert:

  Message from syslogd< at >gbl-macbook at May 23 11:26:11 ...
   kernel:[569106.802042] divide error: 0000 [#1] SMP 
  
  Message from syslogd< at >gbl-macbook at May 23 11:26:11 ...
   kernel:[569106.802393] Stack:
  
  Message from syslogd< at >gbl-macbook at May 23 11:26:11 ...
   kernel:[569106.802430] Call Trace:
  
  Message from syslogd< at >gbl-macbook at May 23 11:26:11 ...
   kernel:[569106.802519] Code: 4c 24 18 49 8b 6c 24 18 48 89 4c 24 18 eb 6b 48 8b 7d 30 48 8d 74 24 38 e8 d4 16 af e0 49 63 8c 24 bc 00 00 00 48 8b 45 10 31 d2 <48> f7 f1 48 0f af c1 48 89 45 10 4d 8b 6c 24 18 eb 2f 49 8b 7d 
  
Here's the relevant output from dmesg:

  [569106.729859] RAID1 conf printout:
  [569106.729870]  --- wd:1 rd:2
  [569106.729878]  disk 0, wo:0, o:1, dev:md1
  [569106.734368] md: unbind<md2>
  [569106.795283] md: export_rdev(md2)
  [569106.802042] divide error: 0000 [#1] SMP 
  [569106.802053] CPU 0 
  [569106.802057] Modules linked in: raid10 raid0 ext4 jbd2 mbcache raid1 md_mod b43 bcma asix usbnet mii nls_cp437 vfat fat nls_utf8 hfsplus usb_storage cdc_acm snd_usb_audio snd_usbmidi_lib ip6table_filter ip6_tables iptable_filter ip_tables ebtable_nat ebtables x_tables parport_pc ppdev lp parport rfcomm bnep pci_stub cpufreq_stats vboxpci(O) cpufreq_userspace cpufreq_powersave cpufreq_conservative vboxnetadp(O) vboxnetflt(O) vboxdrv(O) binfmt_misc uinput nfsd nfs nfs_acl auth_rpcgss fscache lockd sunrpc loop firewire_sbp2 fuse kvm_intel kvm uvcvideo videodev v4l2_compat_ioctl32 media snd_hda_codec_hdmi btusb bluetooth crc16 bcm5974 snd_hda_codec_cirrus joydev snd_hda_intel snd_hda_codec snd_pcm_oss snd_hwdep snd_mixer_oss snd_pcm snd_page_alloc snd_seq_midi snd_seq_midi_event snd_rawmidi arc4 snd_seq snd_seq_device snd_timer coretemp mac80211 cfg80211 rfkill ssb rng_core applesmc pcmcia i2c_i801 pcmcia_core crc32c_intel snd iTCO_wdt acpi_cpufreq soundcore ghash_clmulni_intel aesni_intel aes_x86_64 aes_generic evdev i2c_core iTCO_vendor_support mperf cryptd input_polldev pcspkr fglrx(P) battery video ac apple_bl(O) button processor power_supply thermal_sys xfs hid_microsoft dm_mod sg sr_mod sd_mod cdrom crc_t10dif ata_generic hid_apple usbhid hid uhci_hcd ata_piix libata firewire_ohci scsi_mod firewire_core crc_itu_t sdhci_pci ehci_hcd sdhci mmc_core tg3 usbcore libphy usb_common [last unloaded: bcma]
  [569106.802283] 
  [569106.802290] Pid: 30710, comm: mdadm Tainted: P           O 3.2.0-4-amd64 #1 Debian 3.2.35-2 Apple Inc. MacBookPro8,2/Mac-94245A3940C91C80
  [569106.802302] RIP: 0010:[<ffffffffa065745b>]  [<ffffffffa065745b>] create_strip_zones+0x71/0x491 [raid0]
  [569106.802320] RSP: 0018:ffff8801a8a77d48  EFLAGS: 00010246
  [569106.802325] RAX: 00000000001f3600 RBX: ffff88012ccf5920 RCX: 0000000000000000
  [569106.802331] RDX: 0000000000000000 RSI: fffffffffffffffb RDI: ffff88025151540c
  [569106.802337] RBP: ffff880254d2f000 R08: 00000000fffffffd R09: ffff88025151ffff
  [569106.802343] R10: ffffffff81600000 R11: ffffffff81600000 R12: ffff88042f058c00
  [569106.802349] R13: ffff8801beb09000 R14: 0000000000000005 R15: ffff88042f058c18
  [569106.802356] FS:  00007f6027661700(0000) GS:ffff88046fa00000(0000) knlGS:0000000000000000
  [569106.802363] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
  [569106.802368] CR2: 000000000041f820 CR3: 00000001916d0000 CR4: 00000000000406f0
  [569106.802375] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
  [569106.802381] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
  [569106.802388] Process mdadm (pid: 30710, threadinfo ffff8801a8a76000, task ffff880116fee380)
  [569106.802393] Stack:
  [569106.802397]  00000000fffffff4 0000000000000001 0000000000000001 ffff88042f058c18
  [569106.802408]  0000000000000000 ffff8801a8a77e10 ffff8801a8a77e28 ffff88010031646d
  [569106.802419]  30646961722d646d ffff880100000000 ffff88046fdfcc08 0000004081109b2e
  [569106.802430] Call Trace:
  [569106.802442]  [<ffffffffa0657b11>] ? raid0_takeover+0x1c3/0x1e5 [raid0]
  [569106.802456]  [<ffffffffa066d409>] ? level_store+0x21b/0x52b [md_mod]
  [569106.802470]  [<ffffffffa066ec77>] ? md_attr_store+0x9a/0xbc [md_mod]
  [569106.802483]  [<ffffffff8114e4e7>] ? sysfs_write_file+0xe0/0x11c
  [569106.802494]  [<ffffffff810f9cc5>] ? vfs_write+0xa2/0xe9
  [569106.802502]  [<ffffffff810f9ea2>] ? sys_write+0x45/0x6b
  [569106.802515]  [<ffffffff81352012>] ? system_call_fastpath+0x16/0x1b
  [569106.802519] Code: 4c 24 18 49 8b 6c 24 18 48 89 4c 24 18 eb 6b 48 8b 7d 30 48 8d 74 24 38 e8 d4 16 af e0 49 63 8c 24 bc 00 00 00 48 8b 45 10 31 d2 <48> f7 f1 48 0f af c1 48 89 45 10 4d 8b 6c 24 18 eb 2f 49 8b 7d 
  [569106.802594] RIP  [<ffffffffa065745b>] create_strip_zones+0x71/0x491 [raid0]
  [569106.802604]  RSP <ffff8801a8a77d48>
  [569106.802688] ---[ end trace dc1acf1d4fa82dd5 ]---

Afterwards, my mounted filesystem (/dev/md0) disappeared (it's no longer
mounted), and all operations related to software raid seem to fail:

  gbl-macbook# umount /dev/md0
  ^C^C^C^C
  <hangs forever>

  11:41:38 goblin< at >gbl-macbook:~ % cat /proc/mdstat 
  <hangs forever but is Ctrl-Cable>

I haven't attempted to re-produce this yet.