http://blog.gmane.org/gmane.linux.file-systems.zfs.user hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8563 <pre>Hi, I have an Ubuntu 13.04 running on top of a ZFS pool made from a mirror or 2x 1 TB SATA disks. The machine is working fine and has always been, perfectly stable, so I assume that the hardware, CPU and RAM are OK. The ZFS pool itself is a bit more than 2 years old, AFAIR. SMART shows that all the HDs are healthy and have recorded no bad or relocated sectors ever. As the disks are getting a bit old however, I've started to scrub the pool. To my surprise, scrub has found ~70 CKSUM errors on one disk the 1st time - and could "fix" it successfully. - Then it ran once without finding any error. - Then it ran once and reported around ~50 CKSUM errors on each disk, still telling that it had "fixed" this OK. Still, the disks SMART don't show any recorded error whatsoever, no sector got either reallocated or pending reallocation, ever. Disks pass all the SMART offline tests perfectly well. Given that SATA disks include error detection and correction code, and that the SATA link is end-to-end CRC'd, how comes that ZFS scrub could detect and "fix" CKSUM errors, where the disk itself and SATA subsystem never detected any ? I'm at the point where I'm VERY puzzled. Should I suspect my disks to be slowly dying without telling ? Should I suspect bad RAM ? Should I suspect a weird bug in the ZFS code ? </pre> Swâmi Petaramesh 2013-05-23T07:59:13 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8557 <pre>Is it possible to merge split mirrors? I'm thinking of using a rotating mirror system to store off-site backups transparently. Split off one of a four way mirror and ship it away while the other three chug away, and rotate until I have two offline copies and two active disks putting the oldest offline back in, letting it catch up, then taking the oldest online back off and moving it off-site. Does zpool split suit my needs, or am I better off using the older Zpool offline mechanism? </pre> Schlacta, Christ 2013-05-22T18:31:28 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8546 <pre>I need to write a script that initiates a scrub, waits for it to finish, then upon completion resumes running the script. I can't seem to find a way to do this easily; zpool status (pool) always returns a zero even when a scrub is in progress. Has anyone already done something like this? I suppose I could fire off a cron job once a minute looking at zpool status and parse out what I need, but that feels... ugly. Thanks! </pre> LosingMy ZFS 2013-05-22T11:54:13 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8536 <pre>I switched from OpenIndiana (OI) to zfs on linux. I exported the pool under OI then installed ZOL and imported the pool. All went well except for the cache device which shows up as follows: cache c2t50026B72240C555Dd0s0 UNAVAIL 0 0 0 Of course he cxtx... names are not used under linux. I don't know why it didn't pick up the new device name. If I try to remove the cache device I get: [root]# zpool remove tank c2t50026B72240C555Dd0s0 cannot remove c2t50026B72240C555Dd0s0: no such device in pool How can I remove this cache device? Thanks in advance. </pre> rparachoniak-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org 2013-05-22T00:18:15 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8534 <pre>Using Virtual disks in vmware, I made a ZFS pool. I later rebuilt the system, but forgot what kind of pool I created using the disks (mirror, raid, raidz). Is it possible for the ZFS commands to look at the disks, and tell me, so I can recreate it? </pre> flyddw-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org 2013-05-21T23:51:24 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8527 <pre>Does this error cause any issues? Couldn't exec /usr/lib/rpm/elfdeps: No such file or directory Everything seems to build and run ok, but I do have some strange problems with ZFS, but it might not be related to that error. Thanks, Ryan </pre> Ryan How 2013-05-21T05:03:43 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8518 <pre>Basically: 1. Make sure that all the crypto modules are included in the initrd 2. Include the whole /boot/zfs directory to the initrd, not just the cache 3. Make sure that all crypto modules are loaded before running: 4. Run 'zfs key -l ZFS_BOOTFS' just before mounting filesystem(s) Maybe we should triple check that the module isn't loaded first, but it doesn't seem to hurt to just modprobe a module that's already loaded... To make this work, the wrapper key must be in /boot/zfs at creation time. At least to make everything 'automatic'. It doesn't seem to be nessesary for grub to support this (although it is in the latest version), as long as the wrapper key is included in the initrd. It should probably also work if keysource=passphrase,prompt but I haven't double checked that. No reason why it shouldn't though... I've just tried this on /, /usr, /home and /var on separate encrypted ZFS. The /boot fs is a separate ext4 partition. Next step is to make /boot encrypted as well.... And maybe have an option for the key file to be on external storage (such as USB stick or what have you). But this is the first step at least... !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! PLEASE NOTE, that my repo here includes the crypt stuff from zfsrouge, so you might not want to pull the whole thing, just cherry-pick the relevant commits! !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! The following changes since commit 44e87cba201b5faf9ab25e1f90284efb13140169: zfs-initramfs depends on zfs-grub | zfs-grub-pc | zfs-grub-pc-bin. (2013-05-20 03:59:23 +0200) are available in the git repository at: https://github.com/FransUrbo/debian-zfs master Turbo Fredriksson (1): Support booting on crypted root by adding support to initramfs hook and script. debian/changelog | 8 ++++- .../usr/share/initramfs-tools/hooks/zfs | 8 +++-- .../usr/share/initramfs-tools/scripts/zfs | 39 +++++++++++++++++++- 3 files changed, 50 insertions(+), 5 deletions(-) </pre> Turbo Fredriksson 2013-05-20T13:27:35 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8486 <pre>I have (and had) autoexpand=on, but still get expandsize=16.0E. I replaced three 1.5TB disks with tree 3TB disks about a month ago, and i did get extra space (I'm not 100% sure I got the size I had expected/should but...). How much is this actually, and how do I know where this available space is? I found Illumos #1948 (ZoL pull #908), but that's closed and don't seem to have been merged into ZoL. In that, it seems that '16E' just means undefined.. Do I actually have free/unallocated space or not? </pre> Turbo Fredriksson 2013-05-19T11:42:41 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8470 <pre>I just started a zpool scrub on a pool with a cache device, and a strange thought occured to me... I don't believe a zpool scrub should touch cache at all. I may be wrong, but it seems we want to read all the data from the disk. Reading data from the cache bypasses reading it from disk... Please feel encouraged to correct me if I'm wrong, or consider this a bug report if I'm not wrong. </pre> Christ Schlacta 2013-05-18T18:14:35 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8466 <pre>Hi, before i am going to mess up my configuration ;-) i would like to understand a little bit more: I know, that grub2 is able to boot off a stored ISO-image from disk using casper (and i did it by providing a config file in grub.d). While not using zfs root, most of my ISO's are stored in a zfs pool. Only for booting off the ISO i copied the image over to ext4. IIRC grub2 should be able to read from a zfs pool (given it is not compressed). But i have not yet managed to get it to use my zfs filesystem stored ISO's directly (maybe my understanding of the boot process and grub2 is insufficient?). just in case anyone has done just this, i'd be interested to get their working grub.d script as a starting point. - or further explanation - or both ;-) Thank you UbuntuNewbie </pre> ChessDoter 2013-05-18T09:14:39 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8463 <pre>With all the ZFS problems I'm having, it seems that it must be a problem with my kernel or something, coz otherwise I'm sure there would be a lot more people posting issues on the mailing list and a lot less people using ZOL! They all almost seem related to I/O scheduling and buffering (well the symptoms of it, not sure what the actual problem is) One of the problems I'm having is when copying a file (cp fileA fileB), it basically starves everything else. Then my VMs (running as file backed storage directly on ZFS) get corrupt filesystems reporting that the disk controller was not responding (I'm guessing because they are trying to write, get to response from the system, so the guest things the virtual disk is faulty) Does anyone else get anything like this? When copying as well, all the zfs commands are slow to respond (eg. zpool iostat takes a few seconds to spit out any info) Not using dedup or compression, machine has 32GB ram of which 6GB is currently free (after running for a few hours) I'm not really fussed if performance is slow (but performance is actually quite good), but if I am doing more than 1 disk related thing at a time it is kinda useless. Thanks, Ryan </pre> Ryan How 2013-05-18T04:29:32 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8454 <pre>Ugh, banging my head here. Running (k)ubuntu 12.04.2 LTS. Have two pools set up, root_pool and data_pool. data_pool is a raidz across 5x4TB drives. root_pool mounts at startup just fine, but data_pool will never automount, and I can't for the life of me figure out why. Right after boot, here's what I have: root&lt; at &gt;kubuntu:/home/vince# zpool status -v pool: root_pool state: ONLINE scan: none requested config: NAME STATE READ WRITE CKSUM root_pool ONLINE 0 0 0 scsi-SATA_SanDisk_SDSSDP0130804401449-part3 ONLINE 0 0 0 errors: No known data errors root&lt; at &gt;kubuntu:/home/vince# zpool status -v data_pool cannot open 'data_pool': no such pool But I can *always* import the pool OK (have to use -f, don't know why): root&lt; at &gt;kubuntu:/home/vince# zpool import -d /dev/disk/by-id data_pool cannot import 'data_pool': pool may be in use from other system use '-f' to import anyway root&lt; at &gt;kubuntu:/home/vince# zpool import -f -d /dev/disk/by-id data_pool root&lt; at &gt;kubuntu:/home/vince# zpool list data_pool NAME SIZE ALLOC FREE CAP DEDUP HEALTH ALTROOT data_pool 18.1T 1.38M 18.1T 0% 1.00x ONLINE - Doing a "zfs get all data_pool" says the pool canmount, has a mount point, and basically everything looks hunky dory. I know what you're thinking, stale cache file, right? Nope. Rebuilt it, checked the date/time on it and it's the new time. Also, doing a strings on /etc/zfs/zpool.cache tells me that there's SOMETHING about data_pool (and root_pool) in it: root&lt; at &gt;kubuntu:/home/vince# strings /etc/zfs/zpool.cache data_pool version name data_pool state pool_guid hostname kubuntu vdev_children vdev_tree type root guid children type raidz guid nparity metaslab_array metaslab_shift ashift asize is_log create_txg children type disk guid path whole_disk create_txg type disk guid path whole_disk create_txg type disk guid path whole_disk create_txg type disk guid path whole_disk create_txg type disk guid path whole_disk create_txg features_for_read root_pool version name root_pool state pool_guid Je&lt; at &gt;( hostname kubuntu vdev_children vdev_tree type root guid Je&lt; at &gt;( children type disk guid `)bt path ;/dev/disk/by-id/scsi-SATA_SanDisk_SDSSDP0130804401449-part3 whole_disk metaslab_array metaslab_shift ashift asize is_log create_txg features_for_read I'm at a loss here. No error messages in dmesg, nothing in syslog. I feel like I'm missing something huge but am at a loss here. zdb has nothing much (history of creation and imports, that's it.) All suggestions (no matter how simple) appreciated! Thanks! </pre> LosingMyZFS 2013-05-17T18:09:27 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8400 <pre>Hi :) I have this bunch of small sample files (50+ gigs worth) that I've previously shared via NFS to my Hackintosh where I play around with making music. The FS I created for it is a regular zfs one, just with gzip-9 compression added to the mix. The zpool is quite massive, so my recordsize is 128k, which I've assumed meant that I was wasting a bunch of space on smaller files. But now I'm not so sure. I've been playing around with SCST and ZVOLs and wanted to move all these samples onto a ZVOL formatted with NTFS (4k sectors) (so I can also mount it in Windows). I created the zvol with gzip-9 compression as well, and I was kind of expecting to see the end result being less space consumed than with the regular ZFS fs. Not so. In fact, I created this ZVOL with -V 60G, but references, usedbydataset, and written all say 64.5 GB, whereas the regular ZFS FS uses 52.6 GB. What am I missing here? Probably a lot, so if you can, please educate me! I just don't get it! My properties on the zvol and filesystem: NAME PROPERTY VALUE SOURCE titanic/zvol-composer type volume - titanic/zvol-composer creation Sun May 12 11:56 2013 - titanic/zvol-composer used 64.5G - titanic/zvol-composer available 343G - titanic/zvol-composer referenced 64.5G - titanic/zvol-composer compressratio 1.05x - titanic/zvol-composer reservation none default titanic/zvol-composer volsize 60G local titanic/zvol-composer volblocksize 32K - titanic/zvol-composer checksum on default titanic/zvol-composer compression gzip-9 local titanic/zvol-composer readonly off default titanic/zvol-composer copies 1 default titanic/zvol-composer refreservation none local titanic/zvol-composer primarycache all inherited from titanic titanic/zvol-composer secondarycache all default titanic/zvol-composer usedbysnapshots 0 - titanic/zvol-composer usedbydataset 64.5G - titanic/zvol-composer usedbychildren 0 - titanic/zvol-composer usedbyrefreservation 0 - titanic/zvol-composer logbias latency default titanic/zvol-composer dedup off default titanic/zvol-composer mlslabel none default titanic/zvol-composer sync standard default titanic/zvol-composer refcompressratio 1.05x - titanic/zvol-composer written 64.5G - titanic/zvol-composer snapdev hidden default NAME PROPERTY VALUE SOURCE titanic/Systems/Dex type filesystem - titanic/Systems/Dex creation Wed Apr 24 19:07 2013 - titanic/Systems/Dex used 52.6G - titanic/Systems/Dex available 343G - titanic/Systems/Dex referenced 52.6G - titanic/Systems/Dex compressratio 1.07x - titanic/Systems/Dex mounted yes - titanic/Systems/Dex quota none default titanic/Systems/Dex reservation none default titanic/Systems/Dex recordsize 128K default titanic/Systems/Dex mountpoint /titanic/Systems/Dex inherited from titanic titanic/Systems/Dex sharenfs off inherited from titanic titanic/Systems/Dex checksum on default titanic/Systems/Dex compression gzip-9 local titanic/Systems/Dex atime off inherited from titanic titanic/Systems/Dex devices on default titanic/Systems/Dex exec on default titanic/Systems/Dex setuid on default titanic/Systems/Dex readonly off default titanic/Systems/Dex zoned off default titanic/Systems/Dex snapdir hidden default titanic/Systems/Dex aclinherit restricted default titanic/Systems/Dex canmount on default titanic/Systems/Dex xattr on default titanic/Systems/Dex copies 1 default titanic/Systems/Dex version 4 - titanic/Systems/Dex utf8only off - titanic/Systems/Dex normalization none - titanic/Systems/Dex casesensitivity sensitive - titanic/Systems/Dex vscan off default titanic/Systems/Dex nbmand off default titanic/Systems/Dex sharesmb off default titanic/Systems/Dex refquota none default titanic/Systems/Dex refreservation none default titanic/Systems/Dex primarycache all inherited from titanic titanic/Systems/Dex secondarycache all default titanic/Systems/Dex usedbysnapshots 0 - titanic/Systems/Dex usedbydataset 52.6G - titanic/Systems/Dex usedbychildren 0 - titanic/Systems/Dex usedbyrefreservation 0 - titanic/Systems/Dex logbias latency default titanic/Systems/Dex dedup off default titanic/Systems/Dex mlslabel none default titanic/Systems/Dex sync standard default titanic/Systems/Dex refcompressratio 1.07x - titanic/Systems/Dex written 52.6G - titanic/Systems/Dex snapdev hidden default </pre> Daniel Smedegaard Buus 2013-05-17T15:05:10 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8395 <pre>Hello, list I currently have a whitebox running ZFS under Solaris 11 as a NFS server for VMware. It has a pool made of 7.2K SATA disks + a couple SSD for ZiL and L2ARC and it performs reasonable well. However we need more space/performance and also replace the hardware with enterprise grade gear. I'm also leaning towards ZoL, as we are a Linux shop and that Solaris is the odd ball. I'm following this discussing list for a while and I'm feeling comfortable with the current status of ZoL (good job, btw). Here is my plan for the hardware: Dell PE R720xd, 96GB of RAM. - CentOS 6 x64 booting of a RAID1 (Perc H710) - 9207-8i controller - 11 x OCZ-VERTEX3 480GB - 13 x SAMSUNG 840 Pro Series MZ-7PD512BW 512GB - 2x Intel Ethernet I350 QP 1Gb Now, any thoughts on the best configuration for this setup? My initial idea would be to create a single volume with two raid-z arrays: 1 for the 11 OCZ and 1 for 12 Samsung. Then 1 Samsung as a hot-spare. I would create zvols from that pool and export them via iSCSI to the VMware servers. I would appreciate any input you guys have. Regards, Eri Ramos Bastos </pre> Eri Ramos Bastos 2013-05-17T14:23:44 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8394 <pre></pre> 张飞 2013-05-17T13:01:34 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8383 <pre>I'm curious to see if anyone is doing ZoL on the Dell T320 series servers. They have an option called 'embedded SATA' that links their 8 drive front enclosure to the Intel X79 mainboard. The specification sheet (attached) says that the controller is only valid for Windows but I'm assuming its something LSI based that will work acceptably in Linux. Alternatively, has anyone been using the H710P controller (LSISA2208) that Dell sells? I remember PERC requiring some hacks to expose each drive as a single unit (which I assume is what works best for ZFS). Any experience would be helpful! </pre> Andrew Dunn 2013-05-16T21:42:48 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8377 <pre>Hi, everybody, I've decided to switch my primary laptop over to zfs on /home. I've got a zpool made of 3 LUKS devices, one for the spinning rust, and one each for partitions on the mSATA part to serve as L2ARC and ZIL. It's a laptop, so until native encryption lands, that's just how it has to be. Everything works great except for the final mount of /home at boot time. The zpool comes up fine (all members present and OK) but there's no mount until I export/import it. As a workaround, I enabled an rc.local service under systemd and put an export/import in there, and that's fine as a workaround, but I'd like to figure out why it's not working as intended. I don't have any error messages to go on, so could somebody please point me in the right direction for figuring out where the mount process isn't happening? Thanks, -Bill </pre> Bill McGonigle 2013-05-16T16:10:24 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8376 <pre>I understand that the current version of grub isn't able to deal with root pools that have compression enabled.... but, is it safe to make a child filesystem *in* the root pool and turn on compression there?</pre> bp-/tNbCOJADEfR7s880joybQ< at >public.gmane.org 2013-05-16T15:57:40 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8374 <pre>If DKMS on CentOS 6.4 is working, could someone help me with the steps and order of steps to update the Linux kernel? I had ZFS on Linux version 0.6.1, along with the spl packages and dkms packages for both installed. Yesterday, I updated the kernel to 2.6.32-358.6.1.el6.x86_64 but after installation and reboot, when trying to run 'zpool status' I got the errors: Failed to load ZFS module stack. Load the module manually by running 'insmod &lt;location&gt;/zfs.ko' Failed to load ZFS module stack. Load the module manually by running 'insmod &lt;location&gt;/zfs.ko' The only fix I could put together was: 'yum remove -y zfs spl dkms' and then 'yum install -y zfs'. Any help would be much appreciated. Mike </pre> mikelape-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org 2013-05-16T15:08:05 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8372 <pre>Hello, I require help interpreting some ZFS write benchmarks of a SSD I did with fio. I'm testing on a Dell PE850 with 64 AMD Opteron 6272 (2.1GHz) cores and 256GB RAM. It looks to me as if for some reason ZFS is not able to get more than ~4k IOPS on my system. Read performance is great in all tests (~100k IOPS), but write performance with ZFS seems to be capped at ~4k IOPS. I did my tests with 4x Intel SSDs as well as 4x OCS Revodrive PCIe Cards. In all tests write performance never went above 4k IOPS, whether I was testing a single SSD, a Raid10 or a RaidZ configuration. To reduce complexity I am pasting the results of benchmarking a single Intel SSD, once with ext4, once with ZFS. My test command: fio --bs=4k --size=1g --direct=0 --runtime=30 --rw=randwrite --numjobs=10 --group_reporting --time_based --name=testfile Results: ext4, direct=1 write: io=2528.1MB, bw=86317KB/s, iops=21579 , runt= 30001msec ext4 direct=0 write: io=66526MB, bw=2217.6MB/s, iops=567688 , runt= 30000msec zfs direct=0 write: io=478328KB, bw=15529KB/s, iops=3882 , runt= 30802msec I did the test several times with different block sizes and concurrency. It always caps around 4-5k IOPS. I also experimented with setting recordsize=4k and ashift=12 although I have no idea if that does make any sense for SSDs. It didn't seem to have any impact, neither positive nor negative. What in my system configuration could be the limiting factor that doesn't allow ZFS to go above 4k IOPS? Thanks, </pre> Lukas Loesche 2013-05-16T10:15:03 http://comments.gmane.org/gmane.linux.file-systems.zfs.user/8366 <pre>Hi, What is the status of supporting O_DIRECT on ZFS? I found a 2 years old discussion, where it is said, that it is not supported yet. I have this problem when opening a file ('open(...., O_DIRECT, ...)') stored on ZFS. Is it still an open issue, or is my problem of a different nature. Thanks, </pre> Radoslaw Garbacz 2013-05-15T19:10:35 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.linux.file-systems.zfs.user