http://permalink.gmane.org/gmane.comp.file-systems.btrfs hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17571 <pre>On Fri, May 25, 2012 at 10:27 PM, Peter Maloney &lt;peter.maloney&lt; at &gt;brockmann-consult.de&gt; wrote: I've run it only once after creating all snapshots. My expectation is that in both cases the result is the same. If all snapshots have the file /foo/bar, then each individual snapshotted copy of it would have a different atime and thus an own metadata block for it. As this happens with all files, no matter how i iterated the files, then nearly all metadata blocks get their own copy. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Alexander Block 2012-05-25T20:42:21 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17570 <pre> I take it that you not only count I/O-errors, but also corrupted blocks and errors generated by misdirected writes. These are informations that are not available to the block layer. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Arne Jansen 2012-05-25T20:41:34 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17567 <pre>It would be helpful if already the generic block layer would offer device error counters. Then btrfs could read them, add own counters for its checksum detected errors, and store everything persistently in the filesystem. The goal is to replace disks that have an increased error rate with spare disks, and the goal is to repair this degenerated RAID state quickly. On 05/25/2012 17:18, Christoph Hellwig wrote: [...] -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Stefan Behrens 2012-05-25T17:49:47 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17566 <pre>On Fri, May 25, 2012 at 9:25 AM, Alexander Block &lt;ablock84&lt; at &gt;googlemail.com&gt; wrote: I think you're working from a incorrect understanding: the atime update (metadata) will not cause the data to be de-cow'ed. Even writing to the file will only decow the portion that was written. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> cwillu 2012-05-25T17:39:24 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17564 <pre>Snapshots are by default r/w but can be created r/o explicitly. But that doesn't matter for the normal use case where you snapshot / and continue working on /. After snapshotting, all metadata is shared between the two subvolumes, but when a metadata block in one of both subvolume changes (no matter which one), this one metadata block get's cowed and unshared and uses up more space. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Alexander Block 2012-05-25T16:38:54 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17561 <pre> Just mount with -o noatime, there's no chance of turning something like that on by default since it will break some applications (notably mutt). Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Josef Bacik 2012-05-25T15:42:50 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17559 <pre>On Fri, May 25, 2012 at 5:15 PM, Alexander Block &lt;ablock84&lt; at &gt;googlemail.com&gt; wrote: An additional note. The RO subvolume + atime patch that I sent out recently may reduce the problems in the described worst case if RO snapshots were used. It would however still have the same problems in the case / is snapshotted and you continue working on / (which is the typical usage for snapshots i think). -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Alexander Block 2012-05-25T15:25:21 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17557 <pre>Hello, I would like to start a discussion on atime in Btrfs (and possibly other filesystems with snapshot support). As atime is updated on every access of a file or directory, we get many changes to the trees in btrfs that as always trigger cow operations. This is no problem as long as the changed tree blocks are not shared by other subvolumes. Performance is also not a problem, no matter if shared or not (thanks to relatime which is the default). The problems start when someone starts to use snapshots. If you for example snapshot your root and continue working on your root, after some time big parts of the tree will be cowed and unshared. In the worst case, the whole tree gets unshared and thus takes up the double space. Normally, a user would expect to only use extra space for a tree if he changes something. A worst case scenario would be if someone took regular snapshots for backup purposes and later greps the contents of all snapshots to find a specific file. This would touch all inodes in all trees an</pre> Alexander Block 2012-05-25T15:15:40 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17556 <pre>The linux-joystick mailing list has been superseded by the linux-input list run at vger.kernel.org. See http://vger.kernel.org/vger-lists.html for information on the new list server (or consult your local oracle). Yours virtually, Martin Mares -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Martin Mares 2012-05-25T14:59:24 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17555 <pre>If cow_file_range_inline fails with ENOSPC we abort the transaction which isn't very nice. This really shouldn't be happening anyways but there's no sense in making it a horrible error when we can easily just go allocate normal data space for this stuff. Thanks, Signed-off-by: Josef Bacik &lt;josef&lt; at &gt;redhat.com&gt; --- fs/btrfs/inode.c | 5 ++++- 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index cd51968..46d8732 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c &lt; at &gt;&lt; at &gt; -257,10 +257,13 &lt; at &gt;&lt; at &gt; static noinline int cow_file_range_inline(struct btrfs_trans_handle *trans, ret = insert_inline_extent(trans, root, inode, start, inline_len, compressed_size, compress_type, compressed_pages); -if (ret) { +if (ret &amp;&amp; ret != -ENOSPC) { btrfs_abort_transaction(trans, root, ret); return ret; +} else if (ret == -ENOSPC) { +return 1; } + btrfs_delalloc_release_metadata(inode, end + 1 - start); btrfs_drop_extent_cache(inode, start, aligned_end -</pre> Josef Bacik 2012-05-25T14:10:30 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17554 <pre>Ceph was hitting this race where we would remove an inode from the per-root orphan list before we would release the space we had reserved for the inode. We actually don't need a list or anything, we just need to make sure the root doesn't try to free up the orphan reserve until after the inodes have released their reservations. So use an atomic counter instead of a list on the root and only decrement the counter after we've released our reservation. I've tested this as well as several others and we no longer see the warnings that you would see while running ceph. Thanks, Btrfs: fix how we deal with the orphan block rsv Ceph was hitting this race where we would remove an inode from the per-root orphan list before we would release the space we had reserved for the inode. We actually don't need a list or anything, we just need to make sure the root doesn't try to free up the orphan reserve until after the inodes have released their reservations. So use an atomic counter instead of a list on the root and on</pre> Josef Bacik 2012-05-25T14:10:07 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17553 <pre>Miao pointed this out while I was working on an orphan problem that messing with a bitfield where different ranges are protected by different locks doesn't work out right. Turns out we've been doing this forever where we have different parts of the bit field protected by either no lock at all or different locks which could cause all sorts of weird problems including the issue I was hitting. So instead make a runtime_flags thing that we use the normal bit operations on that are all atomic so we can keep having our no/different locking for the different flags and then make force_compress it's own thing so it can be treated normally. Thanks, Signed-off-by: Josef Bacik &lt;josef&lt; at &gt;redhat.com&gt; --- fs/btrfs/btrfs_inode.h | 30 ++++++++++++++++-------------- fs/btrfs/delayed-inode.c | 4 ++-- fs/btrfs/disk-io.c | 3 ++- fs/btrfs/extent-tree.c | 11 ++++++----- fs/btrfs/file.c | 12 ++++++------ fs/btrfs/inode.c | 28 ++++++++++++---------------- 6 files changed, 44 insertion</pre> Josef Bacik 2012-05-25T14:09:45 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17552 <pre>"btrfs device stats" is used to retrieve and print the device stats. "btrfs device stats -z" is used to atomically retrieve, reset and print the stats. Signed-off-by: Stefan Behrens &lt;sbehrens&lt; at &gt;giantdisaster.de&gt; --- cmds-device.c | 118 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++ ctree.h | 6 +++ ioctl.h | 33 ++++++++++++++++ man/btrfs.8.in | 14 +++++++ print-tree.c | 6 +++ 5 files changed, 177 insertions(+) diff --git a/cmds-device.c b/cmds-device.c index db625a6..7621cc0 100644 --- a/cmds-device.c +++ b/cmds-device.c &lt; at &gt;&lt; at &gt; -246,11 +246,129 &lt; at &gt;&lt; at &gt; static int cmd_scan_dev(int argc, char **argv) return 0; } +static const char * const cmd_dev_stats_usage[] = { +"btrfs device stats [-z] &lt;path&gt;|&lt;device&gt;", +"Show current device IO stats. -z to reset stats afterwards.", +NULL +}; + +static int cmd_dev_stats(int argc, char **argv) +{ +char *path; +struct btrfs_ioctl_fs_info_args fi_args; +struct btrfs_ioctl_dev_info_args *di_args = NULL; +int ret; +int fdmnt; +int</pre> Stefan Behrens 2012-05-25T14:07:18 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17551 <pre>"btrfs device stats" is used to retrieve and print the device stats. "btrfs device stats -z" is used to atomically retrieve, reset and print the stats. In order to share two utility functions between scrub and the dev stats code, these two functions are moved to utils.c and renamed. Since these functions are using open_file_or_dir(), and since the linking against utils.o and common.o was different, open_file_or_dir() was moved from common.c to utils.c. And since that function makes use of the function dirfd(3), the required XOPEN version was raised from 6 to 7. Changes v1-&gt;v2: - Remove a verbose printf() - Cast u64 to unsigned long long for printf() - Update the man page Changes v2-&gt;v3: - Rebase on Chris' current master branch - Split the patch into three seperate patches because after rebasing, open_file_or_dir() was moved and additional changes had been necessary Changes v3-&gt;v4: - Add padding at end of ioctl structure Changes v4-&gt;v5: - The statistic members in the ioctl are now organized as an array</pre> Stefan Behrens 2012-05-25T14:07:15 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17550 <pre>This is a preparation step to add support for device stats. The definition of the function open_file_or_dir() is moved from common.c to utils.c in order to be able to share some common code between scrub and the device stats in the following step. That common code uses open_file_or_dir(). Since open_file_or_dir() makes use of the function dirfd(3), the required XOPEN version was raised from 6 to 7. Signed-off-by: Stefan Behrens &lt;sbehrens&lt; at &gt;giantdisaster.de&gt; --- Makefile | 4 ++-- btrfsctl.c | 28 ---------------------------- cmds-balance.c | 1 + cmds-inspect.c | 1 + cmds-subvolume.c | 1 + commands.h | 3 --- common.c | 46 ---------------------------------------------- utils.c | 31 +++++++++++++++++++++++++++++-- utils.h | 3 +++ 9 files changed, 37 insertions(+), 81 deletions(-) diff --git a/Makefile b/Makefile index 79818e6..fe2b432 100644 --- a/Makefile +++ b/Makefile &lt; at &gt;&lt; at &gt; -39,8 +39,8 &lt; at &gt;&lt; at &gt; all: version $(progs) manpages version: </pre> Stefan Behrens 2012-05-25T14:07:16 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17549 <pre>Two convenient utility functions that have so far been local to scrub are moved to utils.c. They will be used in the device stats code in a following commit. Signed-off-by: Stefan Behrens &lt;sbehrens&lt; at &gt;giantdisaster.de&gt; --- cmds-scrub.c | 72 ++-------------------------------------------------------- utils.c | 66 +++++++++++++++++++++++++++++++++++++++++++++++++++++ utils.h | 4 ++++ 3 files changed, 72 insertions(+), 70 deletions(-) diff --git a/cmds-scrub.c b/cmds-scrub.c index c4503f4..37a9890 100644 --- a/cmds-scrub.c +++ b/cmds-scrub.c &lt; at &gt;&lt; at &gt; -967,74 +967,6 &lt; at &gt;&lt; at &gt; static struct scrub_file_record *last_dev_scrub( return NULL; } -static int scrub_device_info(int fd, u64 devid, - struct btrfs_ioctl_dev_info_args *di_args) -{ -int ret; - -di_args-&gt;devid = devid; -memset(&amp;di_args-&gt;uuid, '\0', sizeof(di_args-&gt;uuid)); - -ret = ioctl(fd, BTRFS_IOC_DEV_INFO, di_args); -return ret ? -errno : 0; -} - -static int scrub_fs_info(int fd, char *path, -struct btrfs_ioctl_fs_info_args *fi_ar</pre> Stefan Behrens 2012-05-25T14:07:17 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17548 <pre>The device statistics are written into the device tree with each transaction commit. Only modified statistics are written. When a filesystem is mounted, the device statistics for each involved device are read from the device tree and used to initialize the counters. Signed-off-by: Stefan Behrens &lt;sbehrens&lt; at &gt;giantdisaster.de&gt; --- fs/btrfs/ctree.h | 38 +++++++++++ fs/btrfs/disk-io.c | 7 ++ fs/btrfs/print-tree.c | 3 + fs/btrfs/transaction.c | 4 ++ fs/btrfs/volumes.c | 176 ++++++++++++++++++++++++++++++++++++++++++++++++ fs/btrfs/volumes.h | 4 ++ 6 files changed, 232 insertions(+) diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h index ec42a24..3e5ab89 100644 --- a/fs/btrfs/ctree.h +++ b/fs/btrfs/ctree.h &lt; at &gt;&lt; at &gt; -823,6 +823,14 &lt; at &gt;&lt; at &gt; struct btrfs_csum_item { u8 csum; } __attribute__ ((__packed__)); +struct btrfs_dev_stats_item { +/* + * grow this item struct at the end for future enhancements and keep + * the existing values unchanged + */ +__le64 values[BTRFS_DEV_STAT_VA</pre> Stefan Behrens 2012-05-25T14:06:10 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17547 <pre>The goal is to detect when drives start to get an increased error rate, when drives should be replaced soon. Therefore statistic counters are added that count IO errors (read, write and flush). Additionally, the software detected errors like checksum errors and corrupted blocks are counted. Signed-off-by: Stefan Behrens &lt;sbehrens&lt; at &gt;giantdisaster.de&gt; --- fs/btrfs/disk-io.c | 13 ++++--- fs/btrfs/extent_io.c | 18 ++++++++-- fs/btrfs/ioctl.h | 19 ++++++++++ fs/btrfs/scrub.c | 65 +++++++++++++++++++++++++--------- fs/btrfs/volumes.c | 94 ++++++++++++++++++++++++++++++++++++++++++++++++-- fs/btrfs/volumes.h | 45 ++++++++++++++++++++++++ 6 files changed, 230 insertions(+), 24 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index a7ffc88..da4d33f 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c &lt; at &gt;&lt; at &gt; -2556,18 +2556,19 &lt; at &gt;&lt; at &gt; recovery_tree_root: static void btrfs_end_buffer_write_sync(struct buffer_head *bh, int uptodate) { -char b[BDEVNAME_SIZE]; - if (uptod</pre> Stefan Behrens 2012-05-25T14:06:08 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17546 <pre>Changes v1-v2: - Remove restriction that BTRFS_IOC_GET_DEVICE_STATS is a privileged operation - Cast u64 to unsigned long long for printf() Changes v2-v3: - Rebased on Chris' current master Changes v3-v4: - Add padding at end of ioctl structure Changes v4-v5: - The statistic members in the ioctl are now organized as an array of 64 bit values. Symbolic names for the array indexes are defined in an enum, which also defines the max value. This change makes it easier to add new statistic members in the future - Give ins_len = -1 to btrfs_search_slot() when an item might get deleted - Introduce a helper function for the repeated sequence stat_int() + dirty = 1 + stat_print() - Introduce a helper function for the code that shares the bio bi_private member for two pieces of information The goal is to detect when drives start to get an increased error rate, when drives should be replaced soon. Therefore statistic counters are added that count IO errors (read, write and flush). Additionally, the sof</pre> Stefan Behrens 2012-05-25T14:06:07 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17545 <pre>An ioctl interface is added to get the device statistic counters. A second ioctl is added to atomically get and reset these counters. Signed-off-by: Stefan Behrens &lt;sbehrens&lt; at &gt;giantdisaster.de&gt; --- fs/btrfs/ioctl.c | 26 ++++++++++++++++++++++++++ fs/btrfs/ioctl.h | 14 ++++++++++++++ fs/btrfs/volumes.c | 34 ++++++++++++++++++++++++++++++++++ fs/btrfs/volumes.h | 3 +++ 4 files changed, 77 insertions(+) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/ioctl.c index 14f8e1f..2f5072d 100644 --- a/fs/btrfs/ioctl.c +++ b/fs/btrfs/ioctl.c &lt; at &gt;&lt; at &gt; -3042,6 +3042,28 &lt; at &gt;&lt; at &gt; static long btrfs_ioctl_scrub_progress(struct btrfs_root *root, return ret; } +static long btrfs_ioctl_get_dev_stats(struct btrfs_root *root, + void __user *arg, int reset_after_read) +{ +struct btrfs_ioctl_get_dev_stats *sa; +int ret; + +if (reset_after_read &amp;&amp; !capable(CAP_SYS_ADMIN)) +return -EPERM; + +sa = memdup_user(arg, sizeof(*sa)); +if (IS_ERR(sa)) +return PTR_ERR(sa); + +ret = btrfs_get_dev_stats(root, sa, reset_aft</pre> Stefan Behrens 2012-05-25T14:06:09 http://permalink.gmane.org/gmane.comp.file-systems.btrfs/17544 <pre>Before the update_time inode operation was indroduced, it was not possible to prevent updates of atime on RO subvolumes. btrfs_update_time does now check if the root is RO and skip updating of atime. This patch requires the update_time patches from Josef Bacik. Signed-off-by: Alexander Block &lt;ablock84&lt; at &gt;googlemail.com&gt; --- fs/btrfs/inode.c | 26 +++++++++++++++++++++----- fs/inode.c | 3 +++ 2 files changed, 24 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/inode.c b/fs/btrfs/inode.c index 54ae3df..b48db5a 100644 --- a/fs/btrfs/inode.c +++ b/fs/btrfs/inode.c &lt; at &gt;&lt; at &gt; -4470,14 +4470,30 &lt; at &gt;&lt; at &gt; int btrfs_dirty_inode(struct inode *inode) static int btrfs_update_time(struct inode *inode, struct timespec *now, int flags) { -if (flags &amp; S_VERSION) +struct btrfs_root *root = BTRFS_I(inode)-&gt;root; +int did_update = 0; + +if (flags &amp; S_VERSION) { inode_inc_iversion(inode); -if (flags &amp; S_CTIME) +did_update = 1; +} +if (flags &amp; S_CTIME) { inode-&gt;i_ctime = *now; -if (flags &amp; S_MTIME) +</pre> Alexander Block 2012-05-25T12:50:43 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.file-systems.btrfs