gmane.comp.file-systems.btrfs

kernel BUG on balance, 3.10-rc2 skinny

Fredrik Rinnestam — 2013-05-23T21:16:46

Filesystem was created fairly recently under kernel 3.9 iirc.

Data, RAID0: total=2.51TB, used=2.49TB
System, RAID1: total=32.00MB, used=188.00KB
System: total=4.00MB, used=0.00
Metadata, RAID1: total=5.00GB, used=3.72GB

Total devices 2 FS bytes used 2.49TB
devid    1 size 2.73TB used 1.26TB path /dev/sdd
devid    2 size 2.73TB used 1.26TB path /dev/sdc

mount:
/dev/sdd on /home/fredrik type btrfs (rw,noatime,compress=lzo,space_cache)

log attached

[PATCH] xfstests: btrfs/311: sparse copy between different filesystems/mountpoints on btrfs

Eric Sandeen — 2013-05-23T17:06:30

From: Koen De Wit <koen.de.wit< at >oracle.com>

# Check if creating a sparse copy ("reflink") of a file on btrfs
# expectedly fails when it's done between different filesystems or
# different mount points of the same filesystem.
#
# For both situations, these actions are executed:
#    - Copy a file with the reflink=auto option.
#      A normal copy should be created.
#    - Copy a file with the reflink=always option. Should result in error,
#      no file should be created.

[sandeen: mostly cosmetic changes]

Signed-off-by: Koen De Wit <koen.de.wit< at >oracle.com>
Signed-off-by: Eric Sandeen <sandeen< at >redhat.com>
---

Originally submitted as:
xfstests: 301: sparse copy between different filesystems/mountpoints on btrfs

diff --git a/tests/btrfs/311 b/tests/btrfs/311
new file mode 100755
index 0000000..9256e7a
--- /dev/null
+++ b/tests/btrfs/311
< at >< at > -0,0 +1,106 < at >< at >
+#! /bin/bash
+# FS QA Test No. 311
+#
+# Check if creating a sparse copy ("reflink") of a file on btrfs
+# expectedly fails when it's done between different filesystems or
+# different mount points of the same filesystem.
+#
+# For both situations, these actions are executed:
+#    - Copy a file with the reflink=auto option.
+#      A normal copy should be created.
+#    - Copy a file with the reflink=always option. Should result in error,
+#      no file should be created.
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2013, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    umount $SCRATCH_MNT &>/dev/null
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+
+_require_scratch
+_require_cp_reflink
+
+SOURCE_DIR=$TEST_DIR/test-$seq
+CROSS_DEV_DIR=$SCRATCH_MNT/test-$seq
+# mount point & target for twice-mounted device
+TEST_DIR2=$TEST_DIR/mount2
+DUAL_MOUNT_DIR=$SCRATCH_MNT/test-bis-$seq
+
+rm -rf $SOURCE_DIR
+mkdir $SOURCE_DIR
+
+rm -f $seqres.full
+
+_scratch_mkfs
+$XFS_IO_PROG -f -c 'pwrite -S 0x61 0 9000' $SOURCE_DIR/original >> $seqres.fll
+
+_filter_testdirs()
+{
+_filter_test_dir | _filter_scratch
+}
+
+_create_reflinks_to()
+{
+    # auto reflink, should fall back to non-reflink
+    rm -rf $1; mkdir $1
+    echo "reflink=auto:"
+    cp --reflink=auto $SOURCE_DIR/original $1/copy
+    md5sum $SOURCE_DIR/original | _filter_testdirs
+    md5sum $1/copy | _filter_testdirs
+
+    # always reflink, should fail outright
+    rm -rf $1; mkdir $1
+    echo "reflink=always:"
+    cp --reflink=always $SOURCE_DIR/original $1/copyfail 2>&1 | _filter_testdirs
+
+    # The failed target actually gets created by cp:
+    ls $1/copyfail | _filter_testdirs
+}
+
+echo "test reflinks across different devices"
+_scratch_mount
+_create_reflinks_to $CROSS_DEV_DIR
+_scratch_unmount
+
+echo "test reflinks across different mountpoints of same device"
+mount $TEST_DEV $SCRATCH_MNT || _fail "Couldn't double-mount $TEST_DEV"
+_create_reflinks_to $DUAL_MOUNT_DIR
+umount $SCRATCH_MNT
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/311.out b/tests/btrfs/311.out
new file mode 100644
index 0000000..210727b
--- /dev/null
+++ b/tests/btrfs/311.out
< at >< at > -0,0 +1,15 < at >< at >
+QA output created by 311
+test reflinks across different devices
+reflink=auto:
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-311/original
+42d69d1a6d333a7ebdf64792a555e392  SCRATCH_MNT/test-311/copy
+reflink=always:
+cp: failed to clone `SCRATCH_MNT/test-311/copyfail': Invalid cross-device link
+SCRATCH_MNT/test-311/copyfail
+test reflinks across different mountpoints of same device
+reflink=auto:
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-311/original
+42d69d1a6d333a7ebdf64792a555e392  SCRATCH_MNT/test-bis-311/copy
+reflink=always:
+cp: failed to clone `SCRATCH_MNT/test-bis-311/copyfail': Invalid cross-device link
+SCRATCH_MNT/test-bis-311/copyfail
diff --git a/tests/btrfs/group b/tests/btrfs/group
index bd624c4..c897118 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
< at >< at > -12,3 +12,4 < at >< at >
 308 auto quick reflink
 309 auto quick reflink
 310 auto quick reflink
+311 auto quick reflink

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] xfstests: btrfs/309: sparse copy of a directory tree on btrfs

Eric Sandeen — 2013-05-23T16:43:32

# Tests file clone functionality of btrfs ("reflinks") on directory trees.
#   - Create directory and subdirectory, each having one file
#   - Create 2 recursive reflinked copies of the tree
#   - Modify the original files
#   - Modify one of the copies

[sandeen: mostly cosmetic changes]

Signed-off-by: Koen De Wit <koen.de.wit< at >oracle.com>
Signed-off-by: Eric Sandeen <sandeen< at >redhat.com>
---

Originally submitted as
xfstests: 298: sparse copy of a directory tree on btrfs

diff --git a/tests/btrfs/309 b/tests/btrfs/309
new file mode 100755
index 0000000..b3927ba
--- /dev/null
+++ b/tests/btrfs/309
< at >< at > -0,0 +1,98 < at >< at >
+#! /bin/bash
+# FS QA Test No. btrfs/309
+#
+# Tests file clone functionality of btrfs ("reflinks") on directory trees.
+#   - Create directory and subdirectory, each having one file
+#   - Create 2 recursive reflinked copies of the tree
+#   - Modify the original files
+#   - Modify one of the copies
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2013, Oracle and/or its affiliates.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-----------------------------------------------------------------------
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1    # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+    cd /
+    rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. common/rc
+. common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+
+_require_xfs_io_fiemap
+_require_cp_reflink
+
+TESTDIR1=$TEST_DIR/test-$seq
+rm -rf $TESTDIR1
+mkdir $TESTDIR1
+
+_checksum_files() {
+    for F in original/file1 original/subdir/file2 \
+                 copy1/file1 copy1/subdir/file2 \
+                 copy2/file1 copy2/subdir/file2
+        do
+            md5sum $TESTDIR1/$F | _filter_test_dir
+        done
+}
+
+rm -f $seqres.full
+
+mkdir $TESTDIR1/original
+mkdir $TESTDIR1/original/subdir
+
+echo "Create the original files and reflink dirs"
+$XFS_IO_PROG -F -f -c 'pwrite -S 0x61 0 9000' $TESTDIR1/original/file1  >> $seqres.full 2>&1
+$XFS_IO_PROG -F -f -c 'pwrite -S 0x62 0 11000' $TESTDIR1/original/subdir/file2  >> $seqres.full 2>&1
+cp --recursive --reflink $TESTDIR1/original $TESTDIR1/copy1
+cp --recursive --reflink $TESTDIR1/copy1 $TESTDIR1/copy2
+
+_verify_reflink $TESTDIR1/original/file1 $TESTDIR1/copy1/file1
+_verify_reflink $TESTDIR1/original/subdir/file2 $TESTDIR1/copy1/subdir/file2
+_verify_reflink $TESTDIR1/original/file1 $TESTDIR1/copy2/file1
+_verify_reflink $TESTDIR1/original/subdir/file2 $TESTDIR1/copy2/subdir/file2
+
+echo "Original md5sums:"
+_checksum_files
+
+echo "Overwrite original/file1 and original/subdir/file2 with new data"
+$XFS_IO_PROG -c 'pwrite -S 0x63 0 13000' $TESTDIR1/original/file1  >> $seqres.full 2>&1
+$XFS_IO_PROG -c 'pwrite -S 0x64 5000 1000'  $TESTDIR1/original/subdir/file2 >> $seqres.full 2>&1
+echo "md5sums now:"
+_checksum_files
+
+echo "Overwrite copy1/file1 and copy1/subdir/file2 with new data"
+$XFS_IO_PROG -c 'pwrite -S 0x65 0 9000' $TESTDIR1/copy1/file1  >> $seqres.full 2>&1
+$XFS_IO_PROG -c 'pwrite -S 0x66 5000 25000' $TESTDIR1/copy1/subdir/file2  >> $seqres.full 2>&1
+echo "md5sums now:"
+_checksum_files
+
+# success, all done
+status=0
+exit
diff --git a/tests/btrfs/309.out b/tests/btrfs/309.out
new file mode 100644
index 0000000..93197d8
--- /dev/null
+++ b/tests/btrfs/309.out
< at >< at > -0,0 +1,25 < at >< at >
+QA output created by 309
+Create the original files and reflink dirs
+Original md5sums:
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-309/original/file1
+ca390545f0aedb54b808d6128c56a7dd  TEST_DIR/test-309/original/subdir/file2
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-309/copy1/file1
+ca390545f0aedb54b808d6128c56a7dd  TEST_DIR/test-309/copy1/subdir/file2
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-309/copy2/file1
+ca390545f0aedb54b808d6128c56a7dd  TEST_DIR/test-309/copy2/subdir/file2
+Overwrite original/file1 and original/subdir/file2 with new data
+md5sums now:
+260f6785c0537fd12582dcae28a3c1a9  TEST_DIR/test-309/original/file1
+b8d91fb600f6f2191f2ba66665374860  TEST_DIR/test-309/original/subdir/file2
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-309/copy1/file1
+ca390545f0aedb54b808d6128c56a7dd  TEST_DIR/test-309/copy1/subdir/file2
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-309/copy2/file1
+ca390545f0aedb54b808d6128c56a7dd  TEST_DIR/test-309/copy2/subdir/file2
+Overwrite copy1/file1 and copy1/subdir/file2 with new data
+md5sums now:
+260f6785c0537fd12582dcae28a3c1a9  TEST_DIR/test-309/original/file1
+b8d91fb600f6f2191f2ba66665374860  TEST_DIR/test-309/original/subdir/file2
+b20229a003e3985c4b0e6806abcd6642  TEST_DIR/test-309/copy1/file1
+b815b24069db14e0a3a07169fd563e93  TEST_DIR/test-309/copy1/subdir/file2
+42d69d1a6d333a7ebdf64792a555e392  TEST_DIR/test-309/copy2/file1
+ca390545f0aedb54b808d6128c56a7dd  TEST_DIR/test-309/copy2/subdir/file2
diff --git a/tests/btrfs/group b/tests/btrfs/group
index f24628c..a5bd6aa 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
< at >< at > -10,3 +10,4 < at >< at >
 284 auto
 307 auto quick
 308 auto quick reflink
+309 auto quick reflink

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

raid6: rmw writes all the time?

Bernd Schubert — 2013-05-23T12:55:47

Hi all,

we got a new test system here and I just also tested btrfs raid6 on 
that. Write performance is slightly lower than hw-raid (LSI megasas) and 
md-raid6, but it probably would be much better than any of these two, if 
it wouldn't read all the during the writes. Is this a known issue? This 
is with linux-3.9.2.

Thanks,
Bernd

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Problem with btrfs send/receive

Felix Blanke — 2013-05-23T11:51:59

Hi everyone,

I was trying the new send/receive feature today but can't make it work.

These are the commands I was using:

btrfs subvol snap -r /mnt/data1/< at >downloads/ /mnt/data1/snapshots/testsnap
btrfs send /mnt/data1/snapshots/testsnap | btrfs receive
/mnt/data1/snapshots/testreceive/

This command never finishes. A 'ls /mnt/data1/snapshots/testreceive/ '
never finishes too. After killing the send/receive process one can see
that the target subvolume was created but is empty.


Sending the snapshot to a file and use this for receiving does work.
What am I missing?


Kind regards,
Felix
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Top shows brtfs-cache-1 and brtfs-endio-met while the hard drives seem busy

Laurent Humblet — 2013-05-22T13:22:42

Hi,

I have setup about a year ago a BTRFS RAID 1 filesystem on two 2TB
Western Digital WD20EARS hard drives and I have created subvolumes
that I mount regularly as I need them.  I put mostly music, videos and
various files on them as well as some Git bare repositories for my
work files but for the last couple of weeks, there seems to be some
activity happening on the drives for a few minutes and the following
processes are showing up while this happens: brtfs-cache-1 and
brtfs-endio-met.  It seems that brtfs-cache-1 is more 'busy' than
brtfs-endio-met but after a while everything seems fine again.

Is there anything I should worry about?  I will soon do a full backup
on ext4 just to make sure I don't loose anything but is there any
checks I can run on the drives to make sure everything is fine both on
a hardware standpoint and on a data integrity standoint?

Thank you for your help,
All the best,
Laurent
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/2] Btrfs: check if leaf's parent exists before pushing items around

Liu Bo — 2013-05-22T12:07:06

During splitting a leaf, pushing items around to hopefully get some space only
works when we have a parent, ie. we have at least one sibling leaf.

Signed-off-by: Liu Bo <bo.li.liu< at >oracle.com>
---
 fs/btrfs/ctree.c |    2 +-
 1 files changed, 1 insertions(+), 1 deletions(-)

diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index c07b6ea..f4654f8 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
< at >< at > -3986,7 +3986,7 < at >< at > static noinline int split_leaf(struct btrfs_trans_handle *trans,
 return -EOVERFLOW;
 
 /* first try to make some room by pushing left and right */
-if (data_size) {
+if (data_size && path->nodes[1]) {
 wret = push_leaf_right(trans, root, path, data_size,
        data_size, 0, 0);
 if (wret < 0)

[PATCH 1/2] Btrfs: dont do log_removal in insert_new_root

Liu Bo — 2013-05-22T12:06:51

As for splitting a leaf, root is just the leaf, and tree mod log does not apply
on leaf, so in this case, we don't do log_removal.

As for splitting a node, the old root is kept as a normal node and we have nicely
put records in tree mod log for moving keys and items, so in this case we don't do
that either.

As above, insert_new_root can get rid of log_removal.

Signed-off-by: Liu Bo <bo.li.liu< at >oracle.com>
---
 fs/btrfs/ctree.c |   10 +++++-----
 1 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/fs/btrfs/ctree.c b/fs/btrfs/ctree.c
index 02fae7f..c07b6ea 100644
--- a/fs/btrfs/ctree.c
+++ b/fs/btrfs/ctree.c
< at >< at > -3143,7 +3143,7 < at >< at > static int balance_node_right(struct btrfs_trans_handle *trans,
  */
 static noinline int insert_new_root(struct btrfs_trans_handle *trans,
    struct btrfs_root *root,
-   struct btrfs_path *path, int level, int log_removal)
+   struct btrfs_path *path, int level)
 {
 u64 lower_gen;
 struct extent_buffer *lower;
< at >< at > -3194,7 +3194,7 < at >< at > static noinline int insert_new_root(struct btrfs_trans_handle *trans,
 btrfs_mark_buffer_dirty(c);
 
 old = root->node;
-tree_mod_log_set_root_pointer(root, c, log_removal);
+tree_mod_log_set_root_pointer(root, c, 0);
 rcu_assign_pointer(root->node, c);
 
 /* the super has an extra ref to root->node */
< at >< at > -3278,14 +3278,14 < at >< at > static noinline int split_node(struct btrfs_trans_handle *trans,
 /*
  * trying to split the root, lets make a new one
  *
- * tree mod log: We pass 0 as log_removal parameter to
+ * tree mod log: We don't log_removal old root in
  * insert_new_root, because that root buffer will be kept as a
  * normal node. We are going to log removal of half of the
  * elements below with tree_mod_log_eb_copy. We're holding a
  * tree lock on the buffer, which is why we cannot race with
  * other tree_mod_log users.
  */
-ret = insert_new_root(trans, root, path, level + 1, 0);
+ret = insert_new_root(trans, root, path, level + 1);
 if (ret)
 return ret;
 } else {
< at >< at > -4005,7 +4005,7 < at >< at > static noinline int split_leaf(struct btrfs_trans_handle *trans,
 }
 
 if (!path->nodes[1]) {
-ret = insert_new_root(trans, root, path, 1, 1);
+ret = insert_new_root(trans, root, path, 1);
 if (ret)
 return ret;
 }

btrfs-tools: debian/patches/09-unaligned-memaccess.patch

Holger Fischer — 2013-05-22T05:08:42

Dear BTRFS-Community,
attached is a patch that probably could be applied upstream:
It is ... Fixing unaligned memory accesses ...
Details to this patch could be read under
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=656955

I rechecked against latest git.
As far as I can see, it's not applied yet.


Best Regards
Holger Fischer

[PATCH 1/2] btrfs-progs: Add block group check funtion.

Qu Wenruo — 2013-05-22T00:18:44

From: Cheng Yang <chenyang.fnst< at >cn.fujitsu.com>

This patch adds the function to check correspondence
between block group, chunk and device extent.

Signed-off-by: Cheng Yang <chenyang.fnst< at >cn.fujitsu.com>
Signed-off-by: Wang Shilong <wangsl-fnst< at >cn.fujitsu.com>
Signed-off-by: Qu Wenruo <quwenruo< at >cn.fujitsu.com>
---
 Makefile           |   4 +-
 cmds-check.c       | 517 ++++++++++++++++++++++++++++++++++++++++++++++++++++-
 dev-extent-cache.c | 187 +++++++++++++++++++
 dev-extent-cache.h |  60 +++++++
 extent-cache.h     |   4 +-
 5 files changed, 762 insertions(+), 10 deletions(-)
 create mode 100644 dev-extent-cache.c
 create mode 100644 dev-extent-cache.h

diff --git a/Makefile b/Makefile
index da7438e..92c5850 100644
--- a/Makefile
+++ b/Makefile
< at >< at > -6,7 +6,7 < at >< at > CFLAGS = -g -O1
 objects = ctree.o disk-io.o radix-tree.o extent-tree.o print-tree.o \
   root-tree.o dir-item.o file-item.o inode-item.o inode-map.o \
   extent-cache.o extent_io.o volumes.o utils.o repair.o \
-  qgroup.o raid6.o free-space-cache.o
+  qgroup.o raid6.o free-space-cache.o dev-extent-cache.o
 cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \
        cmds-inspect.o cmds-balance.o cmds-send.o cmds-receive.o \
        cmds-quota.o cmds-qgroup.o cmds-replace.o cmds-check.o \
< at >< at > -14,7 +14,7 < at >< at > cmds_objects = cmds-subvolume.o cmds-filesystem.o cmds-device.o cmds-scrub.o \
 libbtrfs_objects = send-stream.o send-utils.o rbtree.o btrfs-list.o crc32c.o
 libbtrfs_headers = send-stream.h send-utils.h send.h rbtree.h btrfs-list.h \
        crc32c.h list.h kerncompat.h radix-tree.h extent-cache.h \
-       extent_io.h ioctl.h ctree.h
+       extent_io.h ioctl.h ctree.h dev-extent-cache.h
 
 CHECKFLAGS= -D__linux__ -Dlinux -D__STDC__ -Dunix -D__unix__ -Wbitwise \
     -Wuninitialized -Wshadow -Wundef
diff --git a/cmds-check.c b/cmds-check.c
index 1e5e005..fda2cf2 100644
--- a/cmds-check.c
+++ b/cmds-check.c
< at >< at > -39,6 +39,66 < at >< at >
 #include "utils.h"
 #include "commands.h"
 #include "free-space-cache.h"
+#include "dev-extent-cache.h"
+
+#define REC_UNCHECKED0
+#define REC_CHECKED1
+
+struct block_group_record {
+struct cache_extent cache;
+int state;
+
+u64 objectid;
+u8  type;
+u64 offset;
+
+u64 flags;
+};
+
+struct dev_record {
+struct cache_extent cache;
+int state;
+
+u64 objectid;
+u8  type;
+u64 offset;
+
+u64 devid;
+u64 total_byte;
+u64 byte_used;
+};
+
+struct stripe {
+u64 devid;
+u64 offset;
+};
+
+struct chunk_record {
+struct cache_extent cache;
+int state;
+
+u64 objectid;
+u8  type;
+u64 offset;
+
+u64 length;
+u64 type_flags;
+u16 num_stripes;
+struct stripe stripes[0];
+};
+
+struct dev_extent_record {
+struct cache_dev_extent cache;
+int state;
+
+u64 objectid;
+u8  type;
+u64 offset;
+
+u64 chunk_objecteid;
+u64 chunk_offset;
+u64 length;
+};
 
 static u64 bytes_used = 0;
 static u64 total_csum_bytes = 0;
< at >< at > -1916,7 +1976,7 < at >< at > static int all_backpointers_checked(struct extent_record *rec, int print_errs)
 (unsigned long long)rec->start,
 back->full_backref ?
 "parent" : "root",
-back->full_backref ? 
+back->full_backref ?
 (unsigned long long)dback->parent:
 (unsigned long long)dback->root,
 (unsigned long long)dback->owner,
< at >< at > -2551,6 +2611,153 < at >< at > static int process_extent_ref_v0(struct cache_tree *extent_cache,
 }
 #endif
 
+static int process_chunk_item(struct cache_tree *chunk_cache,
+struct btrfs_key *key, struct extent_buffer *eb, int slot)
+{
+struct btrfs_chunk *ptr;
+struct chunk_record *rec;
+int num_stripes, i;
+int ret = 0;
+
+ptr = btrfs_item_ptr(eb,
+slot, struct btrfs_chunk);
+
+num_stripes = btrfs_chunk_num_stripes(eb, ptr);
+
+rec = malloc(sizeof(*rec) +
+num_stripes * sizeof(*rec->stripes));
+if (!rec) {
+fprintf(stderr, "memory allocation failed\n");
+return -ENOMEM;
+}
+
+rec->cache.start = key->offset;
+rec->cache.size = 1;
+rec->state = REC_UNCHECKED;
+
+rec->objectid = key->objectid;
+rec->type = key->type;
+rec->offset = key->offset;
+
+rec->length = btrfs_chunk_length(eb, ptr);
+rec->type = btrfs_chunk_type(eb, ptr);
+rec->num_stripes = num_stripes;
+
+for (i = 0; i < rec->num_stripes; ++i) {
+rec->stripes[i].devid =
+btrfs_stripe_devid_nr(eb, ptr, i);
+rec->stripes[i].offset =
+btrfs_stripe_offset_nr(eb, ptr, i);
+}
+
+ret = insert_existing_cache_extent(
+chunk_cache, &rec->cache);
+
+return ret;
+}
+
+static int process_dev_item(struct cache_tree *dev_cache,
+struct btrfs_key *key, struct extent_buffer *eb, int slot)
+{
+struct btrfs_dev_item *ptr;
+struct dev_record *rec;
+int ret = 0;
+
+ptr = btrfs_item_ptr(eb,
+slot, struct btrfs_dev_item);
+
+rec = malloc(sizeof(*rec));
+if (!rec) {
+fprintf(stderr, "memory allocation failed\n");
+return -ENOMEM;
+}
+
+rec->cache.start = key->offset;
+rec->cache.size = 1;
+rec->state = REC_UNCHECKED;
+
+rec->objectid = key->objectid;
+rec->type = key->type;
+rec->offset = key->offset;
+
+rec->devid = btrfs_device_id(eb, ptr);
+rec->total_byte = btrfs_device_total_bytes(eb, ptr);
+rec->byte_used = btrfs_device_bytes_used(eb, ptr);
+
+ret = insert_existing_cache_extent(
+dev_cache, &rec->cache);
+
+return ret;
+}
+
+static int process_block_group_item(struct cache_tree *block_group_cache,
+struct btrfs_key *key, struct extent_buffer *eb, int slot)
+{
+struct btrfs_block_group_item *ptr;
+struct block_group_record *rec;
+int ret = 0;
+
+ptr = btrfs_item_ptr(eb, slot,
+struct btrfs_block_group_item);
+
+rec = malloc(sizeof(*rec));
+if (!rec) {
+fprintf(stderr, "memory allocation failed\n");
+return -ENOMEM;
+}
+
+rec->cache.start = key->objectid;
+rec->cache.size = 1;
+rec->state = REC_UNCHECKED;
+
+rec->objectid = key->objectid;
+rec->type = key->type;
+rec->offset = key->offset;
+rec->flags = btrfs_disk_block_group_flags(eb, ptr);
+
+ret = insert_existing_cache_extent(
+block_group_cache, &rec->cache);
+
+return ret;
+}
+
+static int process_dev_extent_item(struct dev_extent_tree *dev_extent_cache,
+struct btrfs_key *key, struct extent_buffer *eb, int slot)
+{
+int ret = 0;
+
+struct btrfs_dev_extent *ptr;
+struct dev_extent_record *rec;
+
+ptr = btrfs_item_ptr(eb,
+slot, struct btrfs_dev_extent);
+
+rec = malloc(sizeof(*rec));
+if (!rec) {
+fprintf(stderr, "memory allocation failed\n");
+return -ENOMEM;
+}
+
+rec->cache.devno = key->objectid;
+rec->cache.offset = key->offset;
+rec->state = REC_UNCHECKED;
+
+rec->objectid = key->objectid;
+rec->type = key->type;
+rec->offset = key->offset;
+
+rec->chunk_objecteid =
+btrfs_dev_extent_chunk_objectid(eb, ptr);
+rec->chunk_offset =
+btrfs_dev_extent_chunk_offset(eb, ptr);
+rec->length = btrfs_dev_extent_length(eb, ptr);
+
+ret = insert_existing_cache_dev_extent(
+dev_extent_cache, &rec->cache);
+
+return ret;
+}
+
 static int process_extent_item(struct btrfs_root *root,
        struct cache_tree *extent_cache,
        struct extent_buffer *eb, int slot)
< at >< at > -3079,7 +3286,11 < at >< at > static int run_next_block(struct btrfs_root *root,
   struct cache_tree *seen,
   struct cache_tree *reada,
   struct cache_tree *nodes,
-  struct cache_tree *extent_cache)
+  struct cache_tree *extent_cache,
+  struct cache_tree *chunk_cache,
+  struct cache_tree *dev_cache,
+  struct cache_tree *block_group_cache,
+  struct dev_extent_tree *dev_extent_cache)
 {
 struct extent_buffer *buf;
 u64 bytenr;
< at >< at > -3177,9 +3388,25 < at >< at > static int run_next_block(struct btrfs_root *root,
 btrfs_item_size_nr(buf, i);
 continue;
 }
+if (key.type == BTRFS_CHUNK_ITEM_KEY) {
+process_chunk_item(chunk_cache, &key, buf, i);
+continue;
+}
+if (key.type == BTRFS_DEV_ITEM_KEY) {
+process_dev_item(dev_cache, &key, buf, i);
+continue;
+}
 if (key.type == BTRFS_BLOCK_GROUP_ITEM_KEY) {
+process_block_group_item(block_group_cache,
+&key, buf, i);
 continue;
 }
+if (key.type == BTRFS_DEV_EXTENT_KEY) {
+process_dev_extent_item(dev_extent_cache,
+&key, buf, i);
+continue;
+
+}
 if (key.type == BTRFS_EXTENT_REF_V0_KEY) {
 #ifdef BTRFS_COMPAT_EXTENT_TREE_V0
 process_extent_ref_v0(extent_cache, buf, i);
< at >< at > -3218,7 +3445,7 < at >< at > static int run_next_block(struct btrfs_root *root,
 ref = btrfs_item_ptr(buf, i,
 struct btrfs_shared_data_ref);
 add_data_backref(extent_cache,
-key.objectid, key.offset, 0, 0, 0, 
+key.objectid, key.offset, 0, 0, 0,
 btrfs_shared_data_ref_count(buf, ref),
 0, root->sectorsize);
 continue;
< at >< at > -3927,9 +4154,260 < at >< at > repair_abort:
 return err;
 }
 
+static void free_chunk_cache(struct cache_tree *chunk_cache)
+{
+struct cache_extent *cache;
+while (1) {
+struct chunk_record *rec;
+
+cache = find_first_cache_extent(chunk_cache, 0);
+if (!cache)
+break;
+rec = container_of(cache, struct chunk_record, cache);
+if (rec->state == REC_UNCHECKED) {
+fprintf(stderr,
+"Chunk[%llu, %u, %llu] "
+"is not referred by any others\n",
+rec->objectid,
+rec->type,
+rec->offset);
+}
+
+remove_cache_extent(chunk_cache, &rec->cache);
+free(rec);
+}
+}
+
+static void free_dev_cache(struct cache_tree *dev_cache)
+{
+struct cache_extent *cache;
+while (1) {
+struct dev_record *rec;
+
+cache = find_first_cache_extent(dev_cache, 0);
+if (!cache)
+break;
+rec = container_of(cache, struct dev_record, cache);
+if (rec->state == REC_UNCHECKED) {
+fprintf(stderr,
+"Dev[%llu, %u, %llu] "
+"is not referred by any others\n",
+rec->objectid,
+rec->type,
+rec->offset);
+}
+
+remove_cache_extent(dev_cache, &rec->cache);
+free(rec);
+}
+}
+
+static void free_block_group_cache(struct cache_tree *block_group_cache)
+{
+struct cache_extent *cache;
+while (1) {
+struct block_group_record *rec;
+
+cache = find_first_cache_extent(block_group_cache, 0);
+if (!cache)
+break;
+rec = container_of(cache, struct block_group_record, cache);
+if (rec->state == REC_UNCHECKED) {
+fprintf(stderr,
+"Block group[%llu, %u, %llu] "
+"is not referred by any others\n",
+rec->objectid,
+rec->type,
+rec->offset);
+}
+
+remove_cache_extent(block_group_cache, &rec->cache);
+free(rec);
+}
+}
+
+static void free_dev_extent_cache(struct dev_extent_tree *dev_extent_cache)
+{
+struct cache_dev_extent *cache;
+while (1) {
+struct dev_extent_record *rec;
+
+cache = find_first_cache_dev_extent(dev_extent_cache, 0);
+if (!cache)
+break;
+rec = container_of(cache, struct dev_extent_record, cache);
+if (rec->state == REC_UNCHECKED) {
+fprintf(stderr,
+"Dev extent[%llu, %u, %llu] "
+"is not referred by any others\n",
+rec->objectid,
+rec->type,
+rec->offset);
+}
+
+remove_cache_dev_extent(dev_extent_cache, &rec->cache);
+free(rec);
+}
+}
+
+/* check btrfs_chunk -> btrfs_dev_extent / btrfs_block_group_item */
+static int check_chunk_refs(struct cache_tree *chunk_cache,
+struct cache_tree *block_group_cache,
+struct dev_extent_tree *dev_extent_cache)
+{
+struct cache_extent *chunk_item;
+struct chunk_record *chunk_rec;
+int err = 0;
+
+chunk_item = find_first_cache_extent(chunk_cache, 0);
+while (chunk_item) {
+struct cache_extent *block_group_item;
+struct block_group_record *block_group_rec;
+
+struct cache_dev_extent *dev_extent_item;
+struct dev_extent_record *dev_extent_rec;
+int i;
+
+chunk_rec = container_of(
+chunk_item, struct chunk_record, cache);
+
+block_group_item = find_cache_extent(block_group_cache,
+chunk_rec->offset, 1);
+if (block_group_item) {
+block_group_rec = container_of(block_group_item,
+struct block_group_record, cache);
+
+if (chunk_rec->length != block_group_rec->offset)
+err = -2;
+if (chunk_rec->offset != block_group_rec->objectid)
+err = -2;
+if (chunk_rec->type != block_group_rec->flags)
+err = -2;
+
+if (err != 0) {
+BUG_ON(1);
+fprintf(stderr,
+"Chunk[%llu, %u, %llu]: "
+"length(%llu), offset(%llu), type(%llu) "
+"mismatch with block group[%llu, %u, %llu]: "
+"offset(%llu), objectid(%llu), flags(%llu)\n",
+chunk_rec->objectid,
+chunk_rec->type,
+chunk_rec->offset,
+chunk_rec->length,
+chunk_rec->offset,
+chunk_rec->type_flags,
+block_group_rec->objectid,
+block_group_rec->type,
+block_group_rec->offset,
+block_group_rec->offset,
+block_group_rec->objectid,
+block_group_rec->flags);
+}
+
+block_group_rec->state = REC_CHECKED;
+chunk_rec->state = REC_CHECKED;
+} else {
+fprintf(stderr,
+"Chunk[%llu, %u, %llu]: "
+"length(%llu), offset(%llu), type(%llu) "
+"is not found in block group\n",
+chunk_rec->objectid,
+chunk_rec->type,
+chunk_rec->offset,
+chunk_rec->length,
+chunk_rec->offset,
+chunk_rec->type_flags);
+err = -1;
+}
+
+for (i = 0; i < chunk_rec->num_stripes; ++i) {
+dev_extent_item = find_cache_dev_extent(
+dev_extent_cache,
+chunk_rec->stripes[i].devid,
+chunk_rec->stripes[i].offset);
+if (dev_extent_item) {
+dev_extent_rec = container_of(dev_extent_item,
+struct dev_extent_record, cache);
+dev_extent_rec->state = REC_CHECKED;
+chunk_rec->state = REC_CHECKED;
+} else {
+fprintf(stderr,
+"Chunk[%llu, %u, %llu] stripe[%llu, %llu]"
+"is not found in dev extent\n",
+chunk_rec->objectid,
+chunk_rec->type,
+chunk_rec->offset,
+chunk_rec->stripes[i].devid,
+chunk_rec->stripes[i].offset);
+err = -1;
+}
+}
+
+chunk_item = next_cache_extent(chunk_item);
+}
+return err;
+}
+
+/* check btrfs_dev_item -> btrfs_dev_extent */
+static int check_dev_refs(struct cache_tree *dev_cache,
+struct dev_extent_tree *dev_extent_cache)
+{
+struct cache_extent *dev_item;
+struct dev_record *dev_rec;
+int err = 0;
+
+dev_item = find_first_cache_extent(dev_cache, 0);
+while (dev_item) {
+struct cache_dev_extent *dev_extent_item;
+struct dev_extent_record *dev_extent_rec;
+u64 total_byte = 0;
+
+dev_rec = container_of(dev_item, struct dev_record, cache);
+
+dev_extent_item = find_first_cache_dev_extent(
+dev_extent_cache, 0);
+while (dev_extent_item) {
+dev_extent_rec = container_of(dev_extent_item,
+struct dev_extent_record, cache);
+
+if (dev_extent_rec->objectid == dev_rec->devid)
+total_byte += dev_extent_rec->length;
+
+dev_extent_item = next_cache_dev_extent(
+dev_extent_item);
+}
+
+if (total_byte != dev_rec->byte_used) {
+err = -2;
+
+BUG_ON(1);
+fprintf(stderr,
+"Dev extent's total-byte(%llu)"
+"is not equal to byte-used(%llu) in"
+"dev[%llu, %u, %llu]\n",
+total_byte,
+dev_rec->byte_used,
+dev_rec->objectid,
+dev_rec->type,
+dev_rec->offset);
+}
+
+dev_rec->state = REC_CHECKED;
+
+dev_item = next_cache_extent(dev_item);
+}
+return err;
+}
+
 static int check_extents(struct btrfs_trans_handle *trans,
  struct btrfs_root *root, int repair)
 {
+struct cache_tree dev_cache;
+struct cache_tree chunk_cache;
+struct cache_tree block_group_cache;
+struct dev_extent_tree dev_extent_cache;
+
 struct cache_tree extent_cache;
 struct cache_tree seen;
 struct cache_tree pending;
< at >< at > -3939,7 +4417,7 < at >< at > static int check_extents(struct btrfs_trans_handle *trans,
 struct btrfs_path path;
 struct btrfs_key key;
 struct btrfs_key found_key;
-int ret;
+int ret, err = 0;
 u64 last = 0;
 struct block_info *bits;
 int bits_nr;
< at >< at > -3947,6 +4425,11 < at >< at > static int check_extents(struct btrfs_trans_handle *trans,
 int slot;
 struct btrfs_root_item ri;
 
+cache_tree_init(&dev_cache);
+cache_tree_init(&chunk_cache);
+cache_tree_init(&block_group_cache);
+dev_extent_tree_init(&dev_extent_cache);
+
 cache_tree_init(&extent_cache);
 cache_tree_init(&seen);
 cache_tree_init(&pending);
< at >< at > -4012,11 +4495,28 < at >< at > static int check_extents(struct btrfs_trans_handle *trans,
 btrfs_release_path(root, &path);
 while(1) {
 ret = run_next_block(root, bits, bits_nr, &last, &pending,
-     &seen, &reada, &nodes, &extent_cache);
+     &seen, &reada, &nodes, &extent_cache,
+     &chunk_cache, &dev_cache,
+     &block_group_cache, &dev_extent_cache);
 if (ret != 0)
 break;
 }
 ret = check_extent_refs(trans, root, &extent_cache, repair);
+if (ret) {
+err = 1;
+fprintf(stderr, "Errors found in extent checking\n");
+}
+ret = check_chunk_refs(&chunk_cache,
+&block_group_cache, &dev_extent_cache);
+if (ret) {
+err = 1;
+fprintf(stderr, "Errors found in chunk refs checking\n");
+}
+ret = check_dev_refs(&dev_cache, &dev_extent_cache);
+if (ret) {
+err = 1;
+fprintf(stderr, "Errors found in dev refs checking\n");
+}
 
 if (repair) {
 free_corrupt_blocks(root->fs_info);
< at >< at > -4026,7 +4526,12 < at >< at > static int check_extents(struct btrfs_trans_handle *trans,
 }
 
 free(bits);
-return ret;
+free_chunk_cache(&chunk_cache);
+free_dev_cache(&dev_cache);
+free_block_group_cache(&block_group_cache);
+free_dev_extent_cache(&dev_extent_cache);
+
+return err;
 }
 
 static struct option long_options[] = {
diff --git a/dev-extent-cache.c b/dev-extent-cache.c
new file mode 100644
index 0000000..0a53639
--- /dev/null
+++ b/dev-extent-cache.c
< at >< at > -0,0 +1,187 < at >< at >
+/*
+ * Copyright (C) 2012 Fujitsu.  All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+#include <stdio.h>
+#include <stdlib.h>
+#include "kerncompat.h"
+#include "dev-extent-cache.h"
+
+void dev_extent_tree_init(struct dev_extent_tree *tree)
+{
+tree->root.rb_node = NULL;
+}
+
+static struct rb_node *tree_insert(struct rb_root *root, u64 devno,
+   u64 offset, struct rb_node *node)
+{
+struct rb_node **p = &root->rb_node;
+struct rb_node *parent = NULL;
+struct cache_dev_extent *entry;
+
+while (*p) {
+parent = *p;
+entry = rb_entry(parent, struct cache_dev_extent, rb_node);
+
+if (devno == entry->devno) {
+if (offset < entry->offset)
+p = &(*p)->rb_left;
+else if (offset > entry->offset)
+p = &(*p)->rb_right;
+else
+return parent;
+} else {
+if (devno < entry->devno)
+p = &(*p)->rb_left;
+else if (devno > entry->devno)
+p = &(*p)->rb_right;
+else
+return parent;
+}
+}
+
+entry = rb_entry(parent, struct cache_dev_extent, rb_node);
+rb_link_node(node, parent, p);
+rb_insert_color(node, root);
+return NULL;
+}
+
+static struct rb_node *__tree_search(struct rb_root *root, u64 devno,
+     u64 offset, struct rb_node **prev_ret)
+{
+struct rb_node *n = root->rb_node;
+struct rb_node *prev = NULL;
+struct cache_dev_extent *entry;
+struct cache_dev_extent *prev_entry = NULL;
+
+while (n) {
+entry = rb_entry(n, struct cache_dev_extent, rb_node);
+prev = n;
+prev_entry = entry;
+
+if (devno == entry->devno) {
+if (offset < entry->offset)
+n = n->rb_left;
+else if (offset > entry->offset)
+n = n->rb_right;
+else
+return n;
+} else {
+if (devno < entry->devno)
+n = n->rb_left;
+else if (devno > entry->devno)
+n = n->rb_right;
+else
+return n;
+}
+}
+if (!prev_ret)
+return NULL;
+
+while (prev && devno >= prev_entry->devno + prev_entry->offset) {
+prev = rb_next(prev);
+prev_entry = rb_entry(prev, struct cache_dev_extent, rb_node);
+}
+*prev_ret = prev;
+return NULL;
+}
+
+struct cache_dev_extent *alloc_cache_dev_extent(u64 devno, u64 offset)
+{
+struct cache_dev_extent *pe = malloc(sizeof(*pe));
+
+if (!pe)
+return pe;
+pe->devno = devno;
+pe->offset = offset;
+return pe;
+}
+
+int insert_existing_cache_dev_extent(struct dev_extent_tree *tree,
+ struct cache_dev_extent *pe)
+{
+struct rb_node *found;
+
+found = tree_insert(&tree->root, pe->devno, pe->offset, &pe->rb_node);
+if (found)
+return -EEXIST;
+
+return 0;
+}
+
+int insert_cache_dev_extent(struct dev_extent_tree *tree, u64 devno, u64 offset)
+{
+struct cache_dev_extent *pe = alloc_cache_dev_extent(devno, offset);
+int ret;
+ret = insert_existing_cache_dev_extent(tree, pe);
+if (ret)
+free(pe);
+return ret;
+}
+
+struct cache_dev_extent *find_cache_dev_extent(struct dev_extent_tree *tree,
+   u64 devno, u64 offset)
+{
+struct rb_node *prev;
+struct rb_node *ret;
+struct cache_dev_extent *entry;
+ret = __tree_search(&tree->root, devno, offset, &prev);
+if (!ret)
+return NULL;
+
+entry = rb_entry(ret, struct cache_dev_extent, rb_node);
+return entry;
+}
+
+struct cache_dev_extent *find_first_cache_dev_extent(
+struct dev_extent_tree *tree, u64 devno)
+{
+struct rb_node *prev;
+struct rb_node *ret;
+struct cache_dev_extent *entry;
+
+ret = __tree_search(&tree->root, devno, 1, &prev);
+if (!ret)
+ret = prev;
+if (!ret)
+return NULL;
+entry = rb_entry(ret, struct cache_dev_extent, rb_node);
+return entry;
+}
+
+struct cache_dev_extent *prev_cache_dev_extent(struct cache_dev_extent *pe)
+{
+struct rb_node *node = rb_prev(&pe->rb_node);
+
+if (!node)
+return NULL;
+return rb_entry(node, struct cache_dev_extent, rb_node);
+}
+
+struct cache_dev_extent *next_cache_dev_extent(struct cache_dev_extent *pe)
+{
+struct rb_node *node = rb_next(&pe->rb_node);
+
+if (!node)
+return NULL;
+return rb_entry(node, struct cache_dev_extent, rb_node);
+}
+
+void remove_cache_dev_extent(struct dev_extent_tree *tree,
+ struct cache_dev_extent *pe)
+{
+rb_erase(&pe->rb_node, &tree->root);
+}
diff --git a/dev-extent-cache.h b/dev-extent-cache.h
new file mode 100644
index 0000000..9be2e2f
--- /dev/null
+++ b/dev-extent-cache.h
< at >< at > -0,0 +1,60 < at >< at >
+/*
+ * Copyright (C) 2012 Fujitsu.  All rights reserved.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public
+ * License v2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public
+ * License along with this program; if not, write to the
+ * Free Software Foundation, Inc., 59 Temple Place - Suite 330,
+ * Boston, MA 021110-1307, USA.
+ */
+
+#ifndef __PENDING_DEV_EXTENT__
+#define __PENDING_DEV_EXTENT__
+#include "kerncompat.h"
+#include "rbtree.h"
+
+struct dev_extent_tree {
+struct rb_root root;
+};
+
+struct cache_dev_extent {
+struct rb_node rb_node;
+u64 devno;
+u64 offset;
+};
+
+void dev_extent_tree_init(struct dev_extent_tree *tree);
+void remove_cache_dev_extent(struct dev_extent_tree *tree,
+  struct cache_dev_extent *pe);
+struct cache_dev_extent *find_first_cache_dev_extent(
+struct dev_extent_tree *tree, u64 devno);
+struct cache_dev_extent *prev_cache_dev_extent(struct cache_dev_extent *pe);
+struct cache_dev_extent *next_cache_dev_extent(struct cache_dev_extent *pe);
+struct cache_dev_extent *find_cache_dev_extent(struct dev_extent_tree *tree,
+   u64 devno, u64 offset);
+int insert_cache_dev_extent(struct dev_extent_tree *tree,
+u64 devno, u64 offset);
+int insert_existing_cache_dev_extent(struct dev_extent_tree *tree,
+struct cache_dev_extent *pe);
+
+static inline int dev_extent_tree_empty(struct dev_extent_tree *tree)
+{
+return RB_EMPTY_ROOT(&tree->root);
+}
+
+static inline void free_cache_dev_extent(struct cache_dev_extent *pe)
+{
+free(pe);
+}
+
+struct cache_dev_extent *alloc_pending_dev_extent(u64 devno, u64 offset);
+
+#endif
diff --git a/extent-cache.h b/extent-cache.h
index 4cd0f79..486766c 100644
--- a/extent-cache.h
+++ b/extent-cache.h
< at >< at > -16,8 +16,8 < at >< at >
  * Boston, MA 021110-1307, USA.
  */
 
-#ifndef __PENDING_EXTENT__
-#define __PENDING_EXTENT__
+#ifndef __EXTENT_CACHE_H__
+#define __EXTENT_CACHE_H__
 
 #if BTRFS_FLAT_INCLUDES
 #include "kerncompat.h"

[PATCH 0/4] btrfs: offline dedupe v1

Mark Fasheh — 2013-05-21T18:28:24

Hi,

The following series of patches implements in btrfs an ioctl to do
offline deduplication of file extents.

To be clear, "offline" in this sense means that the file system is
mounted and running, but the dedupe is not done during file writes,
but after the fact when some userspace software initiates a dedupe.

The primary patch is loosely based off of one sent by Josef Bacik back
in January, 2011.

http://permalink.gmane.org/gmane.comp.file-systems.btrfs/8508

I've made significant updates and changes from the original. In
particular the structure passed is more fleshed out, this series has a
high degree of code sharing between itself and the clone code, and the
locking has been updated.


The ioctl accepts a struct:

struct btrfs_ioctl_same_args {
__u64 logical_offset;/* in - start of extent in source */
__u64 length;/* in - length of extent */
__u16 dest_count;/* in - total elements in info array */
__u16 reserved1;
__u32 reserved2;
struct btrfs_ioctl_same_extent_info info[0];
};

Userspace puts each duplicate extent (other than the source) in an
item in the info array. As there can be multiple dedupes in one
operation, each info item has it's own status and 'bytes_deduped'
member. This provides a number of benefits:

- We don't have to fail the entire ioctl because one of the dedupes failed.

- Userspace will always know how much progress was made on a file as we always
  return the number of bytes deduped.


#define BTRFS_SAME_DATA_DIFFERS1
/* For extent-same ioctl */
struct btrfs_ioctl_same_extent_info {
__s64 fd;/* in - destination file */
__u64 logical_offset;/* in - start of extent in destination */
__u64 bytes_deduped;/* out - total # of bytes we were able
 * to dedupe from this file */
/* status of this dedupe operation:
 * 0 if dedup succeeds
 * < 0 for error
 * == BTRFS_SAME_DATA_DIFFERS if data differs
 */
__s32 status;/* out - see above description */
__u32 reserved;
};


The kernel patches are based off Linux v3.9. The ioctl has been tested
against a small variety of files.

A git tree for the kernel changes can be found at:

https://github.com/markfasheh/btrfs-extent-same


I have a userspace project, duperemove available at:

https://github.com/markfasheh/duperemove

Hopefully this can serve as an example of one possible usage of the ioctl.

duperemove takes a list of files as argument and will search them for
duplicated extents. My most recent changes have been to integrate it
with btrfs_extent_same so that the '-D' switch will have it fire off
dedupe requests once processing of data is complete. Integration with
extent_same has *not* been tested yet so don't expect that to work
flawlessly.

Within the duperemove repo is a file, btrfs-extent-same.c that acts as
a test wrapper around the ioctl. It can be compiled completely
seperately from the rest of the project via "make
btrfs-extent-same". This makes direct testing of the ioctl more
convenient.

Code review is very much appreciated. Thanks,
     --Mark


ChangeLog

- don't error on large length value in btrfs exent-same, instead we just
  dedupe the maximum allowed.  That way userspace doesn't have to worry
  about an arbitrary length limit.

- btrfs_extent_same will now loop over the dedupe range at 1MB increments (for
  a total of 16MB per request)

- cleaned up poorly coded while loop in __extent_read_full_page() (thanks to
  David Sterba <dsterba< at >suse.cz> for reporting this)

- included two fixes from Gabriel de Perthuis <g2p.code< at >gmail.com>:
   - allow dedupe across subvolumes
   - don't lock compressed pages twice when deduplicating

- removed some unused / poorly designed fields in btrfs_ioctl_same_args.
  This should also give us a bit more reserved bytes.

- return -E2BIG instead of -ENOMEM when arg list is too large (thanks to
  David Sterba <dsterba< at >suse.cz> for reporting this)

- Some more reserved bytes are now included as a result of some of my
  cleanups. Quite possibly we could add a couple more.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Regression in kernel v3.9.3

Matteo Fortini — 2013-05-21T11:53:10

Hi,
I just built and installed kernel v3.9.3 (was using v3.9.0 before) and
if I do a git fetch on a quite small (<1MB) directory, the system
starts thrashing and I can't even kill -9 git.
It's a btrfs partition with no fancy mount options (just relatime) on
a core i7 with 8GB RAM, Debian testing.

Please tell me if there's any test I can do to help.

Thank you
Matteo
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

panic BTRFS error (device sdp) in __btrfs_inc_extent_ref:1935: Object already exists [Linux 3.8.4]

Joeri Vanthienen — 2013-05-21T08:37:41

Hi all,

I tried to replace a failed device in a btrfs filesystem on Linux kernel 3.8.4.
After adding a new harddisk (/dev/sdf) to the btrfs pool and removing
the failed device with "btrfs device delete missing /btrfs" the server
panicked.
We can still mount the filesystem in degraded mode, what are the
options to resolve this :) ?

SANOS1:~ # btrfs fi show
Label: 'firstpool'  uuid: 517e8cfa-4275-4589-8da4-6a46ad613daa
        Total devices 15 FS bytes used 2.68TB
        devid   14 size 931.51GB used 394.87GB path /dev/sdp
        devid   13 size 931.51GB used 394.87GB path /dev/sdo
        devid   12 size 931.51GB used 394.87GB path /dev/sdn
        devid   11 size 931.51GB used 394.87GB path /dev/sdm
        devid   10 size 931.51GB used 394.87GB path /dev/sdl
        devid    9 size 931.51GB used 394.87GB path /dev/sdk
        devid    8 size 931.51GB used 394.87GB path /dev/sdj
        devid    7 size 931.51GB used 394.87GB path /dev/sdi
        devid    6 size 931.51GB used 394.87GB path /dev/sdh
        devid   16 size 931.51GB used 373.29GB path /dev/sdf
        devid    5 size 931.51GB used 394.87GB path /dev/sdg
        devid   15 size 931.51GB used 394.87GB path /dev/sde
        devid    2 size 931.51GB used 394.87GB path /dev/sdd
        devid    1 size 931.51GB used 394.87GB path /dev/sdc
        *** Some devices missing

May 20 22:54:00 SANOS1 kernel: btrfs: found 3289 extents
May 20 22:54:07 SANOS1 kernel: ------------[ cut here ]------------
May 20 22:54:07 SANOS1 kernel: WARNING: at fs/btrfs/delayed-ref.c:454
update_existing_ref+0x109/0x140 [btrfs]()
May 20 22:54:07 SANOS1 kernel: Hardware name: X8DTN+-F
May 20 22:54:07 SANOS1 kernel: Modules linked in: joydev st sr_mod
cdrom dm_mod iscsi_scst(O) scst_vdisk(O) scst(O) loop btrfs
zlib_deflate libcrc32c mpt2sas raid_class cpufreq_conservative
cpufreq_userspace cpufreq_powersave iTCO_wdt acpi_cpufreq mperf sg ses
ixgbe igb ptp pps_core iTCO_vendor_support coretemp mdio kvm_intel kvm
crc32c_intel pcspkr serio_raw i2c_i801 lpc_ich mfd_core enclosure
ioatdma i7core_edac edac_core dca mptctl button edd binfmt_misc
microcode autofs4 processor thermal_sys scsi_dh_emc scsi_dh_rdac
scsi_dh_alua scsi_dh_hp_sw scsi_dh mptsas mptscsih mptbase
scsi_transport_sas ata_generic ata_piix [last unloaded: scst]
May 20 22:54:07 SANOS1 kernel: Pid: 9181, comm: btrfs Tainted: G
    O 3.8.4-SCST-1 #2
May 20 22:54:07 SANOS1 kernel: Call Trace:
May 20 22:54:07 SANOS1 kernel:  [<ffffffff8104457a>] ?
warn_slowpath_common+0x7a/0xb0
May 20 22:54:07 SANOS1 kernel:  [<ffffffff810445c5>] ?
warn_slowpath_null+0x15/0x20
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa04541f9>] ?
update_existing_ref+0x109/0x140 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa045448a>] ?
add_delayed_tree_ref+0xca/0x1a0 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa045405a>] ?
add_delayed_ref_head.isra.7+0xca/0x160 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa0454c82>] ?
btrfs_add_delayed_tree_ref+0xf2/0x170 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa03fff69>] ?
btrfs_free_extent+0x99/0xf0 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa0458a62>] ?
replace_path+0x782/0x7f0 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa045b528>] ?
merge_reloc_root+0x1f8/0x510 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa03eeff5>] ?
btrfs_free_path+0x25/0x30 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa045b94a>] ?
merge_reloc_roots+0x10a/0x140 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa045be36>] ?
relocate_block_group+0x266/0x690 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa045c405>] ?
btrfs_relocate_block_group+0x1a5/0x2e0 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa0434990>] ?
btrfs_relocate_chunk.isra.56+0x70/0x730 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa04310d2>] ?
free_extent_buffer+0x32/0x90 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa0438a71>] ?
btrfs_shrink_device+0x1f1/0x420 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa043bbbd>] ?
btrfs_rm_device+0x24d/0x770 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa0443791>] ?
btrfs_ioctl+0xb11/0x1b00 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa04437b3>] ?
btrfs_ioctl+0xb33/0x1b00 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffff815c23b4>] ?
__do_page_fault+0x234/0x4d0
May 20 22:54:07 SANOS1 kernel:  [<ffffffff8116fc4f>] ? do_vfs_ioctl+0x8f/0x530
May 20 22:54:07 SANOS1 kernel:  [<ffffffff8116d241>] ? final_putname+0x21/0x50
May 20 22:54:07 SANOS1 kernel:  [<ffffffff81170181>] ? sys_ioctl+0x91/0xb0
May 20 22:54:07 SANOS1 kernel:  [<ffffffff815c676d>] ?
system_call_fastpath+0x1a/0x1f
May 20 22:54:07 SANOS1 kernel: ---[ end trace 53a7c1fa7b405ba9 ]---
May 20 22:54:07 SANOS1 kernel: ------------[ cut here ]------------
May 20 22:54:07 SANOS1 kernel: WARNING: at fs/btrfs/super.c:256
__btrfs_abort_transaction+0xb9/0xd0 [btrfs]()
May 20 22:54:07 SANOS1 kernel: Hardware name: X8DTN+-F
May 20 22:54:07 SANOS1 kernel: btrfs: Transaction aborted
May 20 22:54:07 SANOS1 kernel: Modules linked in: joydev st sr_mod
cdrom dm_mod iscsi_scst(O) scst_vdisk(O) scst(O) loop btrfs
zlib_deflate libcrc32c mpt2sas raid_class cpufreq_conservative
cpufreq_userspace cpufreq_powersave iTCO_wdt acpi_cpufreq mperf sg ses
ixgbe igb ptp pps_core iTCO_vendor_support coretemp mdio kvm_intel kvm
crc32c_intel pcspkr serio_raw i2c_i801 lpc_ich mfd_core enclosure
ioatdma i7core_edac edac_core dca mptctl button edd binfmt_misc
microcode autofs4 processor thermal_sys scsi_dh_emc scsi_dh_rdac
scsi_dh_alua scsi_dh_hp_sw scsi_dh mptsas mptscsih mptbase
scsi_transport_sas ata_generic ata_piix [last unloaded: scst]
May 20 22:54:07 SANOS1 kernel: Pid: 9181, comm: btrfs Tainted: G
 W  O 3.8.4-SCST-1 #2
May 20 22:54:07 SANOS1 kernel: Call Trace:
May 20 22:54:07 SANOS1 kernel:  [<ffffffff8104457a>] ?
warn_slowpath_common+0x7a/0xb0
May 20 22:54:07 SANOS1 kernel:  [<ffffffff81044651>] ?
warn_slowpath_fmt+0x41/0x50
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa03ebe39>] ?
__btrfs_abort_transaction+0xb9/0xd0 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa03fb140>] ?
__btrfs_inc_extent_ref+0x1c0/0x210 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa0400c8d>] ?
run_clustered_refs+0x6cd/0xb60 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa03eec01>] ?
btrfs_set_path_blocking+0x31/0x70 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa04048b0>] ?
btrfs_run_delayed_refs+0xc0/0x2e0 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa03ef555>] ?
btrfs_put_tree_mod_seq+0x105/0x150 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa0414826>] ?
__btrfs_end_transaction+0xe6/0x3f0 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa0414b4e>] ?
btrfs_end_transaction_throttle+0xe/0x10 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa045b7ad>] ?
merge_reloc_root+0x47d/0x510 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa03eeff5>] ?
btrfs_free_path+0x25/0x30 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa045b94a>] ?
merge_reloc_roots+0x10a/0x140 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa045be36>] ?
relocate_block_group+0x266/0x690 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa045c405>] ?
btrfs_relocate_block_group+0x1a5/0x2e0 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa0434990>] ?
btrfs_relocate_chunk.isra.56+0x70/0x730 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa04310d2>] ?
free_extent_buffer+0x32/0x90 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa0438a71>] ?
btrfs_shrink_device+0x1f1/0x420 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa043bbbd>] ?
btrfs_rm_device+0x24d/0x770 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa0443791>] ?
btrfs_ioctl+0xb11/0x1b00 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa04437b3>] ?
btrfs_ioctl+0xb33/0x1b00 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffff815c23b4>] ?
__do_page_fault+0x234/0x4d0
May 20 22:54:07 SANOS1 kernel:  [<ffffffff8116fc4f>] ? do_vfs_ioctl+0x8f/0x530
May 20 22:54:07 SANOS1 kernel:  [<ffffffff8116d241>] ? final_putname+0x21/0x50
May 20 22:54:07 SANOS1 kernel:  [<ffffffff81170181>] ? sys_ioctl+0x91/0xb0
May 20 22:54:07 SANOS1 kernel:  [<ffffffff815c676d>] ?
system_call_fastpath+0x1a/0x1f
May 20 22:54:07 SANOS1 kernel: ---[ end trace 53a7c1fa7b405baa ]---
May 20 22:54:07 SANOS1 kernel: BTRFS error (device sdp) in
__btrfs_inc_extent_ref:1935: Object already exists
May 20 22:54:07 SANOS1 kernel: btrfs is forced readonly
May 20 22:54:07 SANOS1 kernel: ------------[ cut here ]------------
May 20 22:54:07 SANOS1 kernel: kernel BUG at fs/btrfs/relocation.c:2278!
May 20 22:54:07 SANOS1 kernel: invalid opcode: 0000 [#1] SMP
May 20 22:54:07 SANOS1 kernel: Modules linked in: joydev st sr_mod
cdrom dm_mod iscsi_scst(O) scst_vdisk(O) scst(O) loop btrfs
zlib_deflate libcrc32c mpt2sas raid_class cpufreq_conservative
cpufreq_userspace cpufreq_powersave iTCO_wdt acpi_cpufreq mperf sg ses
ixgbe igb ptp pps_core iTCO_vendor_support coretemp mdio kvm_intel kvm
crc32c_intel pcspkr serio_raw i2c_i801 lpc_ich mfd_core enclosure
ioatdma i7core_edac edac_core dca mptctl button edd binfmt_misc
microcode autofs4 processor thermal_sys scsi_dh_emc scsi_dh_rdac
scsi_dh_alua scsi_dh_hp_sw scsi_dh mptsas mptscsih mptbase
scsi_transport_sas ata_generic ata_piix [last unloaded: scst]
May 20 22:54:07 SANOS1 kernel: CPU 0
May 20 22:54:07 SANOS1 kernel: Pid: 9181, comm: btrfs Tainted: G
 W  O 3.8.4-SCST-1 #2 Supermicro X8DTN+-F/X8DTN+-F
May 20 22:54:07 SANOS1 kernel: RIP: 0010:[<ffffffffa045b972>]
[<ffffffffa045b972>] merge_reloc_roots+0x132/0x140 [btrfs]
May 20 22:54:07 SANOS1 kernel: RSP: 0018:ffff88038b99ba98  EFLAGS: 00010286
May 20 22:54:07 SANOS1 kernel: RAX: 00000000ffffffe2 RBX:
ffff88002cf81000 RCX: 00000000ffffffe2
May 20 22:54:07 SANOS1 kernel: RDX: 0000000000000001 RSI:
ffffffffa047c360 RDI: ffff88082d27e000
May 20 22:54:07 SANOS1 kernel: RBP: ffff88038b99bac8 R08:
0000000000000000 R09: ffff8804aad365c0
May 20 22:54:07 SANOS1 kernel: R10: ffff8800602834c0 R11:
ffff88010469a940 R12: ffff88007a20b000
May 20 22:54:07 SANOS1 kernel: R13: ffff88038b99ba98 R14:
ffff88007a20b590 R15: ffff88007a20b000
May 20 22:54:07 SANOS1 kernel: FS:  00007f82fbf8b740(0000)
GS:ffff8804adc00000(0000) knlGS:0000000000000000
May 20 22:54:07 SANOS1 kernel: CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
May 20 22:54:07 SANOS1 kernel: CR2: 00007f57ea573210 CR3:
00000005e55d1000 CR4: 00000000000007f0
May 20 22:54:07 SANOS1 kernel: DR0: 0000000000000000 DR1:
0000000000000000 DR2: 0000000000000000
May 20 22:54:07 SANOS1 kernel: DR3: 0000000000000000 DR6:
00000000ffff0ff0 DR7: 0000000000000400
May 20 22:54:07 SANOS1 kernel: Process btrfs (pid: 9181, threadinfo
ffff88038b99a000, task ffff8802ce00c200)
May 20 22:54:07 SANOS1 kernel: Stack:
May 20 22:54:07 SANOS1 kernel:  ffff8804a0fe1c40 ffff8803e8da6440
0000000000000000 ffff88007a20b020
May 20 22:54:07 SANOS1 kernel:  ffff88007a20b0e8 ffff88013ef8e640
ffff88038b99bb58 ffffffffa045be36
May 20 22:54:07 SANOS1 kernel:  ffffea0019132140 ffff88038b99bb20
0000000000000001 0000000014e62280
May 20 22:54:07 SANOS1 kernel: Call Trace:
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa045be36>]
relocate_block_group+0x266/0x690 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa045c405>]
btrfs_relocate_block_group+0x1a5/0x2e0 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa0434990>]
btrfs_relocate_chunk.isra.56+0x70/0x730 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa04310d2>] ?
free_extent_buffer+0x32/0x90 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa0438a71>]
btrfs_shrink_device+0x1f1/0x420 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa043bbbd>]
btrfs_rm_device+0x24d/0x770 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa0443791>] ?
btrfs_ioctl+0xb11/0x1b00 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffffa04437b3>]
btrfs_ioctl+0xb33/0x1b00 [btrfs]
May 20 22:54:07 SANOS1 kernel:  [<ffffffff815c23b4>] ?
__do_page_fault+0x234/0x4d0
May 20 22:54:07 SANOS1 kernel:  [<ffffffff8116fc4f>] do_vfs_ioctl+0x8f/0x530
May 20 22:54:07 SANOS1 kernel:  [<ffffffff8116d241>] ? final_putname+0x21/0x50
May 20 22:54:07 SANOS1 kernel:  [<ffffffff81170181>] sys_ioctl+0x91/0xb0
May 20 22:54:07 SANOS1 kernel:  [<ffffffff815c676d>]
system_call_fastpath+0x1a/0x1f
May 20 22:54:07 SANOS1 kernel: Code: f9 ff ff 85 c0 74 97 0f 0b 85 d2
0f 85 10 ff ff ff 49 83 bc 24 80 05 00 00 00 75 15 48 83 c4 10 31 c0
5b 41 5c 41 5d 41 5e 5d c3 <0f> 0b 0f 0b 0f 0b 0f 0b 66 0f 1f 44 00 00
55 48 89 e5 41 57 48
May 20 22:54:07 SANOS1 kernel: RIP  [<ffffffffa045b972>]
merge_reloc_roots+0x132/0x140 [btrfs]
May 20 22:54:07 SANOS1 kernel:  RSP <ffff88038b99ba98>
May 20 22:54:07 SANOS1 kernel: ---[ end trace 53a7c1fa7b405bab ]---
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH -next] Btrfs: return error code in btrfs_check_trunc_cache_free_space()

Wei Yongjun — 2013-05-21T02:39:21

From: Wei Yongjun <yongjun_wei< at >trendmicro.com.cn>

Fix to return error code instead always return 0 from function
btrfs_check_trunc_cache_free_space().
Introduced by commit 7b61cd92242542944fc27024900c495a6a7b3396
(Btrfs: don't use global block reservation for inode cache truncation)

Signed-off-by: Wei Yongjun <yongjun_wei< at >trendmicro.com.cn>
---
 fs/btrfs/free-space-cache.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/fs/btrfs/free-space-cache.c b/fs/btrfs/free-space-cache.c
index e530096..73577f4 100644
--- a/fs/btrfs/free-space-cache.c
+++ b/fs/btrfs/free-space-cache.c
< at >< at > -213,7 +213,7 < at >< at > int btrfs_check_trunc_cache_free_space(struct btrfs_root *root,
 else
 ret = 0;
 spin_unlock(&rsv->lock);
-return 0;
+return ret;
 }
 
 int btrfs_truncate_free_space_cache(struct btrfs_root *root,

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] xfstests: btrfs 308: regression test for btrfs send

Josef Bacik — 2013-05-20T17:20:34

I'm not sure how the numbering is supposed to work now that we've split
everything out so I'm just going with the next number in the directory.  This is
a regression test for btrfs send, we had a problem where we'd try to send a file
that had been deleted in the source snapshot.  This is just to make sure we
don't have the same problem in the future.  Thanks,

Signed-off-by: Josef Bacik <jbacik< at >fusionio.com>
---
 src/Makefile        |    2 +-
 src/hold-open.c     |   29 ++++++++++++++++++++
 tests/btrfs/308     |   74 +++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/btrfs/308.out |    2 +
 tests/btrfs/group   |    1 +
 5 files changed, 107 insertions(+), 1 deletions(-)
 create mode 100644 src/hold-open.c
 create mode 100644 tests/btrfs/308
 create mode 100644 tests/btrfs/308.out

diff --git a/src/Makefile b/src/Makefile
index c18ffc9..3ab9a21 100644
--- a/src/Makefile
+++ b/src/Makefile
< at >< at > -18,7 +18,7 < at >< at > LINUX_TARGETS = xfsctl bstat t_mtab getdevicesize preallo_rw_pattern_reader \
 locktest unwritten_mmap bulkstat_unlink_test t_stripealign \
 bulkstat_unlink_test_modified t_dir_offset t_futimens t_immutable \
 stale_handle pwrite_mmap_blocked t_dir_offset2 seek_sanity_test \
-seek_copy_test t_readdir_1 t_readdir_2 fsync-tester
+seek_copy_test t_readdir_1 t_readdir_2 fsync-tester hold-open
 
 SUBDIRS =
 
diff --git a/src/hold-open.c b/src/hold-open.c
new file mode 100644
index 0000000..f44c1dd
--- /dev/null
+++ b/src/hold-open.c
< at >< at > -0,0 +1,29 < at >< at >
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <errno.h>
+#include <unistd.h>
+
+int main(int argc, char **argv)
+{
+int fd;
+
+if (argc < 2) {
+fprintf(stderr, "Usage: hold-open <filename>\n");
+return -1;
+}
+
+fd = open(argv[1], O_RDONLY);
+if (fd < 0) {
+fprintf(stderr, "Error opening file: %d\n", errno);
+return -1;
+}
+
+while (1) {
+sleep(60);
+}
+
+close(fd);
+return 0;
+}
diff --git a/tests/btrfs/308 b/tests/btrfs/308
new file mode 100644
index 0000000..144be99
--- /dev/null
+++ b/tests/btrfs/308
< at >< at > -0,0 +1,74 < at >< at >
+#! /bin/bash
+# FS QA Test No. btrfs/308
+#
+# btrfs send ESTALE regression test, kernel.org bugzilla 57491
+#
+#-----------------------------------------------------------------------
+# Copyright (c) 2013 Fusion IO.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#
+#-----------------------------------------------------------------------
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1# failure is the default!
+
+_cleanup()
+{
+    rm -f $tmp.*
+}
+
+trap "_cleanup ; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch
+
+_scratch_mkfs > /dev/null 2>&1
+_scratch_mount
+
+dd if=/dev/zero of=$SCRATCH_MNT/foo bs=1M count=10 > /dev/null 2>&1
+sync
+$BTRFS_UTIL_PROG subvol snap -r $SCRATCH_MNT $SCRATCH_MNT/snap > /dev/null 2>&1
+dd if=/dev/zero of=$SCRATCH_MNT/foo bs=1M count=10 oflag=append > /dev/null 2>&1
+sync
+
+# We need to hold the file open, tail -f makes the output file look weird so
+# here is this awful hack to get around that
+$here/src/hold-open $SCRATCH_MNT/foo &
+
+# need this so that tail doesn't error out racing with the rm
+sleep 1
+rm -f $SCRATCH_MNT/foo
+$BTRFS_UTIL_PROG subvol snap -r $SCRATCH_MNT $SCRATCH_MNT/snap1 > /dev/null 2>&1
+$BTRFS_UTIL_PROG send -f /dev/null -p $SCRATCH_MNT/snap $SCRATCH_MNT/snap1 >/dev/null 2>&1
+if [ $? -ne 0 ]
+then
+echo "send failed"
+fi
+kill $!
+
+echo "Silence is golden"
+status=0 ; exit
diff --git a/tests/btrfs/308.out b/tests/btrfs/308.out
new file mode 100644
index 0000000..a731eaa
--- /dev/null
+++ b/tests/btrfs/308.out
< at >< at > -0,0 +1,2 < at >< at >
+QA output created by 308
+Silence is golden
diff --git a/tests/btrfs/group b/tests/btrfs/group
index bc6c256..666054b 100644
--- a/tests/btrfs/group
+++ b/tests/btrfs/group
< at >< at > -9,3 +9,4 < at >< at >
 276 auto rw metadata
 284 auto
 307 auto quick
+308 auto quick

[PATCH] Btrfs: fix estale with btrfs send

Josef Bacik — 2013-05-20T15:30:45

This fixes bugzilla 57491.  If we take a snapshot of a fs with a unlink ongoing
and then try to send that root we will run into problems.  When comparing with a
parent root we will search the parents and the send roots commit_root, which if
we've just created the snapshot will include the file that needs to be evicted
by the orphan cleanup.  So when we find a changed extent we will try and copy
that info into the send stream, but when we lookup the inode we use the normal
root, which no longer has the inode because the orphan cleanup deleted it.  The
best solution I have for this is to check our otransid with the generation of
the commit root and if they match just commit the transaction again, that way we
get the changes from the orphan cleanup.  With this patch the reproducer I made
for this bugzilla no longer returns ESTALE when trying to do the send.  Thanks,

Cc: stable< at >vger.kernel.org
Signed-off-by: Josef Bacik <jbacik< at >fusionio.com>
---
 fs/btrfs/send.c |   35 +++++++++++++++++++++++++++++++++++
 1 files changed, 35 insertions(+), 0 deletions(-)

diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index ff40f1c..ba9690b 100644
--- a/fs/btrfs/send.c
+++ b/fs/btrfs/send.c
< at >< at > -4579,6 +4579,41 < at >< at > long btrfs_ioctl_send(struct file *mnt_file, void __user *arg_)
 send_root = BTRFS_I(file_inode(mnt_file))->root;
 fs_info = send_root->fs_info;
 
+/*
+ * This is done when we lookup the root, it should already be complete
+ * by the time we get here.
+ */
+WARN_ON(send_root->orphan_cleanup_state != ORPHAN_CLEANUP_DONE);
+
+/*
+ * If we just created this root we need to make sure that the orphan
+ * cleanup has been done and committed since we search the commit root,
+ * so check its commit root transid with our otransid and if they match
+ * commit the transaction to make sure everything is updated.
+ */
+down_read(&send_root->fs_info->extent_commit_sem);
+if (btrfs_header_generation(send_root->commit_root) ==
+    btrfs_root_otransid(&send_root->root_item)) {
+struct btrfs_trans_handle *trans;
+
+up_read(&send_root->fs_info->extent_commit_sem);
+
+trans = btrfs_attach_transaction_barrier(send_root);
+if (IS_ERR(trans)) {
+if (PTR_ERR(trans) != -ENOENT) {
+ret = PTR_ERR(trans);
+goto out;
+}
+/* ENOENT means theres no transaction */
+} else {
+ret = btrfs_commit_transaction(trans, send_root);
+if (ret)
+goto out;
+}
+} else {
+up_read(&send_root->fs_info->extent_commit_sem);
+}
+
 arg = memdup_user(arg_, sizeof(*arg));
 if (IS_ERR(arg)) {
 ret = PTR_ERR(arg);

btrfs-tools: debian/patches/08-fsck.patch

Holger Fischer — 2013-05-20T15:00:28

Dear BTRFS-Community,

this patch is a reworked one of the debian-original to address the latest changes in the btrfs-tools source code.

It fixes problems that can occur when you boot a machine with btrfs root filesystem.
Boot can stop, because fsck of the btrfs-root-filesystem fails.

Here the patch:
 >cat 08-fsck.patch
--- a/cmds-check.c
+++ b/cmds-check.c
< at >< at > -3549,6 +3549,12 < at >< at > const char * const cmd_check_usage[] = {
         NULL
  };

+static void cmd_check_info(void)
+{
+       fprintf(stderr, "%s\n", BTRFS_BUILD_VERSION);
+       exit(0);
+}
+
  int cmd_check(int argc, char **argv)
  {
         struct cache_tree root_cache;
< at >< at > -3566,12 +3572,11 < at >< at > int cmd_check(int argc, char **argv)

         while(1) {
                 int c;
-               c = getopt_long(argc, argv, "as:", long_options,
+               c = getopt_long(argc, argv, "as:nry", long_options,
                                 &option_index);
                 if (c < 0)
                         break;
                 switch(c) {
-                       case 'a': /* ignored */ break;
                         case 's':
                                 num = atol(optarg);
                                 bytenr = btrfs_sb_offset(num);
< at >< at > -3581,6 +3586,13 < at >< at > int cmd_check(int argc, char **argv)
                         case '?':
                         case 'h':
                                 usage(cmd_check_usage);
+                               break;
+                       case 'a':
+                       case 'n':
+                       case 'r':
+                       case 'y':
+                               cmd_check_info();
+
                 }
                 if (option_index == 1) {
                         printf("enabling repair mode\n");
--- a/btrfs.c
+++ b/btrfs.c
< at >< at > -270,7 +270,8 < at >< at > int main(int argc, char **argv)
         else
                 bname = argv[0];

-       if (!strcmp(bname, "btrfsck")) {
+       if (!strcmp(bname, "btrfsck") ||
+           !strcmp(bname, "fsck.btrfs")) {
                 argv[0] = "check";
         } else {
                 argc--;


It addresses 2 issues:


- fsck.btrfs is not working anymore:
   to make it work again:
   - fsck.btrfs must be a (sym)link to btrfs (should be fixed in the Makefile which I did not do)
   - when calling $BINPATH/fsck.btrfs the main-function in btrfs.c must add the option "check" as 1st argument
     or described in a 2nd way: do "btrfs check" when basename of executable is fsck.btrfs

   this is equal to the behaviour of btrfsck

   the changes to btrfs.c resolve this issue


- "fsck -a <btrfsroot> (done during boot of at least ubuntu 12.04 with standard settings)
   does not return 0
   -> boot fails:
   to make it work the debian maintainers decided to create a patch which
   - prints an info (btrfs version) and return 0 when
     fsck.btrfs is called with one of the options
     "-a", "-n", "-r" or "-y"

   I rewrote the original patch to make this work again with newer versions of btrfs-tools

   Somebody upstream already cared about including "-a", so it seems somebody already tried to address this issue.


I would be happy to see these patches (or the thoughts behind, if you don't like how it's made) included upstream.

Best Regards
Holger Fischer

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

btrfs-tools: debian/patches/02-ftbfs.patch

Holger Fischer — 2013-05-20T12:59:27

Dear BTRFS-Community,

as far as I understand I believe it would make sense to apply that one upstream:

like described, it ...  Fixes FTBFS on alpha and ia64 ...

 >cat 02-ftbfs.patch
Authors:
  Luca Bruno <lucab< at >debian.org>
  Alexander Kurtz <kurtz.alex< at >googlemail.com>
  Daniel Baumann <daniel.baumann< at >progress-technologies.net>
Description:
  Patch to properly cast and avoiding compiler warnings. Fixes FTBFS on alpha
  and ia64 (Closes: #539433, #583768).

Index: b/convert.c
===================================================================
--- a/convert.c
+++ b/convert.c
< at >< at > -2512,7 +2512,7 < at >< at > int do_rollback(const char *devname, int
         ext2_root = btrfs_read_fs_root(root->fs_info, &key);
         if (!ext2_root || IS_ERR(ext2_root)) {
                 fprintf(stderr, "unable to open subvol %llu\n",
-                       key.objectid);
+                       (unsigned long long) key.objectid);
                 goto fail;
         }


The file convert.c was renamed recently to btrfs-convert.c.
The fprintf in the do_rollback-function doesn't contain the mentioned fix.

Best Regards
Holger Fischer
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

btrfs pseudo-drbd

Martin — 2013-05-19T17:32:09

Dear Devs,

Would there be any problem to use nbd (/dev/ndX) devices to gain
btrfs-raid across multiple physical hosts across a network? (For a sort
of btrfs-drbd! :-) )


Regards,
Martin


http://en.wikipedia.org/wiki/Network_block_device

http://www.drbd.org/

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

btrfs (general) raid for other filesystems?

Martin — 2013-05-19T17:31:14

Just a random Sunday afternoon thought:

We've got some rather nice variations on the block-level RAID schemes
but instead being implemented at the filesystem level in btrfs...

Could the btrfs RAID be coded to be general so that a filesystem stack
could be set up whereby the filesystem level raids could be used for ANY
filesystem?


So for example, we could have the stack:


filesystem level RAID

 |
 V

filesystem

 |
 V

Block level


So, an interesting variation could be to have filesystem level raid
operating on ext4 or nilfs or whatever... Would that be a sensible idea?



Regards,
Martin

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majordomo< at >vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V2] Btrfs-progs: fix compile warning in btrfs_free_block_groups()

Wang Shilong — 2013-05-18T16:54:37

From: Wang Shilong <wangsl-fnst< at >cn.fujitsu.com>

extent-tree.c: In function 'btrfs_free_block_groups':
extent-tree.c:3190:12: warning: cast to pointer from integer of
 different size [-Wint-to-pointer-cast]

Signed-off-by: Wang Shilong <wangsl-fnst< at >cn.fujitsu.com>
---
v1->v2: fix encoding error in changelog
---
 extent-tree.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/extent-tree.c b/extent-tree.c
index 381572d..153d14b 100644
--- a/extent-tree.c
+++ b/extent-tree.c
< at >< at > -3187,7 +3187,8 < at >< at > int btrfs_free_block_groups(struct btrfs_fs_info *info)
 break;
 ret = get_state_private(&info->block_group_cache, start, &ptr);
 if (!ret) {
-cache = (struct btrfs_block_group_cache *)ptr;
+cache = (struct btrfs_block_group_cache *)
+(unsigned long)ptr;
 if (cache->free_space_ctl) {
 btrfs_remove_free_space_cache(cache);
 kfree(cache->free_space_ctl);