gmane.linux.kernel.bcache.devel

Possible bug? bcache: error opening /dev/md126: Not enough buckets

John Clark — 2013-06-14T05:34:02

Hi there,

I'm new to kernel mailing lists so beg your pardon if I'm asking this
question in the wrong place or in the wrong way.

Anyway, I'm tantalisingly close to getting my bcache-enabled system
up-and-running, but I've hit a brick wall which seems might be a bug
or undocumented performance limit of some kind. I'm hoping Kent or
others on this fine list can help.

I have successfully created the following two RAID1 block devices, as
shown per /proc/mdstat:

Personalities : [raid1]
md126 : active raid1 sdb[0] sdc[1]
      1953383360 blocks super 1.2 [2/2] [UU]

md127 : active raid1 sdd[0] sde[1]
      117155200 blocks super 1.2 [2/2] [UU]

/dev/md126 comprises two 2.0TB spinning HDD's which I'm intending to
use as a bcache backing device
/dev/md127 comprises two 60GB SSD's which I'm intending to use as a
bcache cache device

So far so good.

Creating the bcache superblock on the md127 appears to go smoothly:

root< at >bigdata:~# make-bcache -C /dev/md127
UUID:                   1736b20f-6b85-4fee-a801-4cf7c1bba009
Set UUID:               b2c9e8e2-0606-4d51-bf9a-e9b8a6f150b3
version:                0
nbuckets:               228818
block_size:             8
bucket_size:            1024
nr_in_set:              1
nr_this_dev:            0
first_bucket:           1


Creating the bcache superblock on md126 *appears* to succeed also:

root< at >bigdata:~# make-bcache -B /dev/md/spinning

UUID:                   9e2bd59a-a413-4ad2-a07b-6998dfa3e049
Set UUID:               8c70baad-6941-4550-9fc8-b009e016b00d
version:                1
block_size:             8
data_offset:            16

However, the following is output on syslog when executing the above command:
Jun 14 14:21:37 bigdata kernel: [ 1602.102646] bcache: error opening
/dev/md126: Not enough buckets

Indeed, although I can register the cache device (and the UUID shows
up in /sys/fs/bcache), all attempts to register the backing device
fails as follows:

root< at >bigdata:~# echo /dev/md126 >/sys/fs/bcache/register
-bash: echo: write error: Invalid argument

And, sure enough, the backing device UUID doesn't appear in
/sys/fs/bcache nor at /dev/bcacheN

I've tried using the make-bcache -b parameter to specify a different
bucket size but I still get the same failure (unless I choose
ridiculously high or low bucket sizes, which results in bucket size
errors to be emitted on syslog).

As I was just finishing up writing this and was about to hit SEND, I
noticed something that I wish I had noticed earlier - specifically
that the "bcache-3.2" section of the bcache git repo was last updated
6 months ago.

The kernel I'm running is built from that tree. I had assumed that it
was the version 3.2 kernel patched with a relatively current version
of bcache, but now I think I may be seriously mistaken and my problems
could be due to the possibility that I'm running an old and buggy
version of bcache. I don't want to get ahead of myself, but it seems a
decent guess that my problems might be stemming from the fact that the
'bcache-3.2' tree uses old version of bcache.

At the risk of getting seriously ahead of myself, assuming that my
problem is due to 'bcache-3.2' being out of date and buggy, I'll ask
this: what is the recommended way to get a current and stable version
of bcache running on a stable linux kernel? I'm wanting to use bcache
on a production system and so I'm a little wary of building the
'bcache-for-3.10' tree. Ideally, I'd like to use bcache on a 3.2
kernel, because that's the kernel version readily available presently
in debian stable/wheezy so I will be less likely to encounter kernel
version incompatibilities with my debian wheezy system.

Many thanks in advance if anyone can help me along here.

John

(b)cache trashing

Piergiorgio Sartor — 2013-06-13T20:33:49

Hi all,

one (or two), naive, question.

I assume that bcache will cache on reads too.

What will happen if something like this is run:

find /home -type -f -exec md5sum {} \;

That is, the md5 of *all* files is computed.

I assume that the cache will be completely
trashed and no old data will be present.

Is this the case?

Now, assuming we have a RAID something, with
LVM on top, with different volumes.
For example one for /home, one for /usr, etc.
Would it make sense, in respect of the above
trashing, to have different SSD (or paritions)
to cache the different volumes?
Instead of caching the complete RAID, of course.

Thanks a lot in advance,

bye,

[PATCH] bcache: Set the logical block size to a sensible value

Gabriel de Perthuis — 2013-06-13T20:25:02

Preserve the backing device's logical size, use 512 byte
sectors when there is no backing device.

The logical block size has no impact on performance, but
it alters the meaning of on-disk structures like partition
tables.  Preserve the backing device's sector size to keep
bcache transparent, and use 512 byte sectors like everyone
else when no backing device is present.

Signed-off-by: Gabriel de Perthuis <g2p.code-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>
---
 drivers/md/bcache/super.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
index 1e3bc4c..94fad70 100644
--- a/drivers/md/bcache/super.c
+++ b/drivers/md/bcache/super.c
< at >< at > -745,11 +745,12 < at >< at > static void bcache_device_free(struct bcache_device *d)
 bioset_free(d->bio_split);
 
 closure_debug_destroy(&d->cl);
 }
 
-static int bcache_device_init(struct bcache_device *d, unsigned block_size)
+static int bcache_device_init(struct bcache_device *d, unsigned block_size,
+      unsigned logical_size)
 {
 struct request_queue *q;
 
 if (!(d->bio_split = bioset_create(4, offsetof(struct bbio, bio))) ||
     !(d->unaligned_bvec = mempool_create_kmalloc_pool(1,
< at >< at > -774,11 +775,11 < at >< at > static int bcache_device_init(struct bcache_device *d, unsigned block_size)
 q->limits.max_sectors= UINT_MAX;
 q->limits.max_segment_size= UINT_MAX;
 q->limits.max_segments= BIO_MAX_PAGES;
 q->limits.max_discard_sectors= UINT_MAX;
 q->limits.io_min= block_size;
-q->limits.logical_block_size= block_size;
+q->limits.logical_block_size= logical_size;
 q->limits.physical_block_size= block_size;
 set_bit(QUEUE_FLAG_NONROT,&d->disk->queue->queue_flags);
 set_bit(QUEUE_FLAG_DISCARD,&d->disk->queue->queue_flags);
 
 return 0;
< at >< at > -1064,11 +1065,11 < at >< at > static int cached_dev_init(struct cached_dev *dc, unsigned block_size)
 for (io = dc->io; io < dc->io + RECENT_IO; io++) {
 list_add(&io->lru, &dc->io_lru);
 hlist_add_head(&io->hash, dc->io_hash + RECENT_IO);
 }
 
-ret = bcache_device_init(&dc->disk, block_size);
+ret = bcache_device_init(&dc->disk, block_size, q->limits.logical_block_size);
 if (ret)
 return ret;
 
 set_capacity(dc->disk.disk,
      dc->bdev->bd_part->nr_sects - dc->sb.data_offset);
< at >< at > -1163,11 +1164,11 < at >< at > static int flash_dev_run(struct cache_set *c, struct uuid_entry *u)
 closure_init(&d->cl, NULL);
 set_closure_fn(&d->cl, flash_dev_flush, system_wq);
 
 kobject_init(&d->kobj, &bch_flash_dev_ktype);
 
-if (bcache_device_init(d, block_bytes(c)))
+if (bcache_device_init(d, block_bytes(c), 512))
 goto err;
 
 bcache_device_attach(d, c, u - c->uuids);
 set_capacity(d->disk, u->sectors);
 bch_flash_dev_request_init(d);

(unknown)

OS Engineering — 2013-06-11T14:54:05

Hi Jens,

In continuation with our previous communication, we have carried out performance comparison among EnhanceIO, bcache and dm-cache.

We found that EnhanceIO provides better throughput on zipf workload (with theta=1.2) in comparison to bcache and dm-cache for write through caches.
However, for write back caches, we found that dm-cache had best throughput followed by EnhanceIO and then bcache. Dm-cache commits on-disk metadata every time a REQ_SYNC or REQ_FUA bio is written. If no such requests are made then it commits metadata once every second. If power is lost, it may lose some recent writes. However, EnhanceIO and bcache do not acknowledge IO completion until both IO and metadata hits the SSD. Hence, EnhanceIO and bcache provide higher data integrity at a cost of performance.

The fio config and setup information follows:
HDD              : 100GB
SSD              :  20GB
Mode             : write through / write back
Cache block_size : 4KB for bcache, EnhanceIO and 256KB for dm-cache

The other options have been left to their default values.

Note:
1) In case of dm-cache, we used 2 partitions of same SSD with 1GB partition as metadata and 20GB partition as caching device. This has been done so as to ensure a fair comparison as EnhanceIO and bcache do not have a separate metadata device.

2) In order to ensure proper cache warm up, We have turned off sequential bypass in bcache. This does not impact our performance results as they are taken for random workload.

For each test, we first performed a warm up run using the following fio options:
fio --direct=1 --size=90% --filesize=20G --blocksize=4k --ioengine=libaio --rw=rw --rwmixread=100 --rwmixwrite=0 --iodepth=8 ...

Then, we performed our actual run with the following fio options:
fio --direct=1 --size=100% --filesize=20G --blocksize=4k --ioengine=libaio --rw=randrw --rwmixread=90 --rwmixwrite=10 --iodepth=8 --numjobs=4 --random_distribution=zipf:1.2 ...

============================= Write Through ===============================
Type      Read Latency(ms)   Write Latency(ms)    Read(MB/s)    Write(MB/s)
===========================================================================
EnhanceIO      1.58              16.53               32.91       3.65
bcache         0.58              31.05               27.17       3.02
dm-cache       0.24              27.45               31.05       3.44

============================= Write Back ==================================
Type      Read Latency(ms)    Write Latency(ms)    Read(MB/s)   Write(MB/s)
===========================================================================
EnhanceIO      0.34               4.98               138.72      15.40
bcache         0.95               1.76               106.82      11.85
dm-cache       0.58               0.55               193.76      21.52

============================ Base Line ====================================
Type      Read Latency(ms)    Write Latency(ms)    Read(MB/s)   Write(MB/s)
===========================================================================
HDD            6.22              27.23                13.51       1.49
SSD            0.47               0.42               235.87      26.21

We have written scripts that aid in cache creation, deletion and performance run for all these caching solutions. These scripts can be found at:
https://github.com/stec-inc/EnhanceIO/tree/master/performance_test

Thanks and Regards,
sTec Team

PROPRIETARY-CONFIDENTIAL INFORMATION INCLUDED



This electronic transmission, and any documents attached hereto, may contain confidential, proprietary and/or legally privileged information. The information is intended only for use by the recipient named above. If you received this electronic message in error, please notify the sender and delete the electronic message. Any disclosure, copying, distribution, or use of the contents of information received in error is strictly prohibited, and violators will be pursued legally.

Performance Comparison among EnhanceIO, bcache and dm-cache.

OS Engineering — 2013-06-11T15:05:07

Hi Jens,

In continuation with our previous communication, we have carried out performance comparison among EnhanceIO, bcache and dm-cache.

We found that EnhanceIO provides better throughput on zipf workload (with theta=1.2) in comparison to bcache and dm-cache for write through caches.
However, for write back caches, we found that dm-cache had best throughput followed by EnhanceIO and then bcache. Dm-cache commits on-disk metadata every time a REQ_SYNC or REQ_FUA bio is written. If no such requests are made then it commits metadata once every second. If power is lost, it may lose some recent writes. However, EnhanceIO and bcache do not acknowledge IO completion until both IO and metadata hits the SSD. Hence, EnhanceIO and bcache provide higher data integrity at a cost of performance.

The fio config and setup information follows:
HDD              : 100GB
SSD              :  20GB
Mode             : write through / write back
Cache block_size : 4KB for bcache, EnhanceIO and 256KB for dm-cache

The other options have been left to their default values.

Note:
1) In case of dm-cache, we used 2 partitions of same SSD with 1GB partition as metadata and 20GB partition as caching device. This has been done so as to ensure a fair comparison as EnhanceIO and bcache do not have a separate metadata device.

2) In order to ensure proper cache warm up, We have turned off sequential bypass in bcache. This does not impact our performance results as they are taken for random workload.

For each test, we first performed a warm up run using the following fio options:
fio --direct=1 --size=90% --filesize=20G --blocksize=4k --ioengine=libaio --rw=rw --rwmixread=100 --rwmixwrite=0 --iodepth=8 ...

Then, we performed our actual run with the following fio options:
fio --direct=1 --size=100% --filesize=20G --blocksize=4k --ioengine=libaio --rw=randrw --rwmixread=90 --rwmixwrite=10 --iodepth=8 --numjobs=4 --random_distribution=zipf:1.2 ...

============================= Write Through ===============================
Type      Read Latency(ms)   Write Latency(ms)    Read(MB/s)    Write(MB/s)
===========================================================================
EnhanceIO      1.58              16.53               32.91       3.65
bcache         0.58              31.05               27.17       3.02
dm-cache       0.24              27.45               31.05       3.44

============================= Write Back ==================================
Type      Read Latency(ms)    Write Latency(ms)    Read(MB/s)   Write(MB/s)
===========================================================================
EnhanceIO      0.34               4.98               138.72      15.40
bcache         0.95               1.76               106.82      11.85
dm-cache       0.58               0.55               193.76      21.52

============================ Base Line ====================================
Type      Read Latency(ms)    Write Latency(ms)    Read(MB/s)   Write(MB/s)
===========================================================================
HDD            6.22              27.23                13.51       1.49
SSD            0.47               0.42               235.87      26.21

We have written scripts that aid in cache creation, deletion and performance run for all these caching solutions. These scripts can be found at:
https://github.com/stec-inc/EnhanceIO/tree/master/performance_test

Thanks and Regards,
sTec Team

PROPRIETARY-CONFIDENTIAL INFORMATION INCLUDED



This electronic transmission, and any documents attached hereto, may contain confidential, proprietary and/or legally privileged information. The information is intended only for use by the recipient named above. If you received this electronic message in error, please notify the sender and delete the electronic message. Any disclosure, copying, distribution, or use of the contents of information received in error is strictly prohibited, and violators will be pursued legally.

[PATCH] md: bcache: Fixed a typo with the word 'arithmetic'

Phil Viana — 2013-06-03T12:51:42

The word 'arithmetic' was typed as 'arithmatic'

Signed-off-by: Phil Viana <phillip.l.viana-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>
---
 drivers/md/bcache/bset.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/md/bcache/bset.c b/drivers/md/bcache/bset.c
index d33f5ad..8822ed2 100644
--- a/drivers/md/bcache/bset.c
+++ b/drivers/md/bcache/bset.c
< at >< at > -403,7 +403,7 < at >< at > void inorder_test(void)
 #endif
 
 /*
- * Cacheline/offset <-> bkey pointer arithmatic:
+ * Cacheline/offset <-> bkey pointer arithmetic:
  *
  * t->tree is a binary search tree in an array; each node corresponds to a key
  * in one cacheline in t->set (BSET_CACHELINE bytes).
< at >< at > -412,7 +412,7 < at >< at > void inorder_test(void)
  * the binary tree points to; to_inorder() gives us the cacheline, and then
  * bkey_float->m gives us the offset within that cacheline, in units of 8 bytes.
  *
- * cacheline_to_bkey() and friends abstract out all the pointer arithmatic to
+ * cacheline_to_bkey() and friends abstract out all the pointer arithmetic to
  * make this work.
  *
  * To construct the bfloat for an arbitrary key we need to know what the key

bcache on top of drbd or the contrary?

Giovanni Lenzi — 2013-06-01T15:45:09

Hi,
I'm planning to user bcache for speed, and drbd for HA between two nodes.

I'm facing with this question:
Is it better to have drbd on top of a bcached volume(caching + backing device)
or to use bcache to put a caching device on top of a drbd backing device?

I think that with the second option and writeback enabled, write
requests should be faster than with the first option... Do you agree?

But.. what about the data integrity and consistency of the data on the
drbd slave volume? What if the power goes down on the master node?
Could the drbd volume on the slave node, get corrupted If the caching
device is not detached correctly on the master node?

On the contrary with the first option(drbd on top of bcache), I think
drbd should always preserve data integrity on the two nodes, isn't it?
Can bcache with writeback mode, cause corrupted data on both nodes in
case of a ssd, or power unit failure?

And where is it better to put drbd metadata? External or internal?
External to ssd is the obvious answer, but if I want to store them
internally, what will be the advantages and disadvantages of the two
solutions?

With the first option, storing metadata internally on the bcached
device should not be so slow, given that bcache should always cache
most used data. Isn't it?

What do you think about this subject?

--
Giovanni Lenzi

[PATCH] md: bcache: Fixed a spelling mistake with the word 'journaling' in a comment

Phil Viana — 2013-05-31T02:42:38

The word 'journaling' is spelt with a single 'l' throughout the Linux
source code, and this spelling is also found in the Oxford dictionary
and used in other projects in general.

Signed-off-by: Phil Viana <phillip.l.viana-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>
---
 drivers/md/bcache/bset.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/md/bcache/bset.c b/drivers/md/bcache/bset.c
index a0f190a..d33f5ad 100644
--- a/drivers/md/bcache/bset.c
+++ b/drivers/md/bcache/bset.c
< at >< at > -31,7 +31,7 < at >< at > int bch_keylist_realloc(struct keylist *l, int nptrs, struct cache_set *c)
 unsigned newsize = oldsize + 2 + nptrs;
 uint64_t *new;
 
-/* The journalling code doesn't handle the case where the keys to insert
+/* The journaling code doesn't handle the case where the keys to insert
  * is bigger than an empty write: If we just return -ENOMEM here,
  * bio_insert() and bio_invalidate() will insert the keys created so far
  * and finish the rest when the keylist is empty.

[PATCH] md: bcache: io.c: fix a potential NULL pointer dereference

Kumar Amit Mehta — 2013-05-28T07:31:15

bio_alloc_bioset returns NULL on failure. This fix adds a missing check
for potential NULL pointer dereferencing.

Signed-off-by: Kumar Amit Mehta <gmate.amit-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>
---
 drivers/md/bcache/io.c |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/md/bcache/io.c b/drivers/md/bcache/io.c
index 29f344b..4be2a07 100644
--- a/drivers/md/bcache/io.c
+++ b/drivers/md/bcache/io.c
< at >< at > -98,6 +98,8 < at >< at > struct bio *bch_bio_split(struct bio *bio, int sectors,
 
 if (bio->bi_rw & REQ_DISCARD) {
 ret = bio_alloc_bioset(gfp, 1, bs);
+if (!ret)
+return NULL;
 idx = 0;
 goto out;
 }

speed

folkert — 2013-05-21T12:19:30

Hi,

I've got a couple of questions regarding bcache:

- will it be faster to implement bcache instead of letting the
  write-intent-map of a raid1 volume be written on an ssd together with
  the journals of the filesystems on that raid1 volume?

- if i have multiple logical volumes in a volume group, do I need to
  explicitly configure bcache for each of them? or can I "connect" bcache
  to the whole volume group?

- does bcache do trim? and/or pass it through?

- maybe bcache can check if a written block is all 0x00 and then do a
  trim instead? that is not only usefull for SSDs but can help
  data-deduplication software/appliances as well

- what will happen if an SSD suddenly stops working in the sense that it
  won't allow anymore writes? will bcache then still commit all the
  buffered data which was already written to the SSD but not to the HDDs?

- any suggestions what SSD to buy? e.g. an OCZ Vertex 4?

- are there any suggestions on what size SSD to use for a certain size
  HDD? or does it depend on the amount of data written per second? or
  the number of iops?

- is it possible to do an emergency flush where everything in the cache
  is written to disk, *as much as possible*? e.g. if a read fails,
  continue with other blocks?
  I'm asking this as I'm not entirely convinced of the quality of SSDs

- is it possible to use more than 1 SSD as a bcache for the same
  storage? not as raid0 as you then can't remove one from the system
  without affecting the other

- does bcache work with e.g. iscsi/nbd remote storage?
  - then maybe it would be nice if it is possible to temporarily bring
    down the remote-storage + bcache combination when the network
    connection goes down

- is it possible to swap via bcache?

- maybe zcache can be combined so that you can work with more data with
  a smaller SSD?

- I think I read on the website that blocks are stored in sorted order
  on disk. that's probably on the HDD below it and not on the SSD,
  right? (because afaik an ssd has always the same access latency,
  independent of write order. altough I can image if that is not true)

- does it keep track of checksums/crcs of the data it writes to the SSD?
  e.g. it writes a block to the SSD and then later on it reads that
  block back from the SSD (to write to e.g. HDD) and verifies that what
  it got back is what it expected


Folkert van Heusden

(unknown)

sheng qiu — 2013-05-17T14:47:16

subscribe linux-bcache

--
Sheng Qiu
Texas A & M University
Room 332B Wisenbaker
email: herbert1984106-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org
College Station, TX 77843-3259

Poor performance with bcache write-back mode

Patrick Ng — 2013-05-16T21:51:46

Hi all,

I have installed Linux kernel 3.9.2 in my Fedora 18.  When I ran FIO in bcache write-through mode, the performance was good, but running a similar test in bcache write-back mode, I got very poor performance.
Did I miss any options when setting the write-back mode?

Environment
===========
500G backing store at /dev/sdb1
150G SSD at /dev/sdc1

Write-through mode
==================
make-bcache -B /dev/sdb1 -C /dev/sdc1
echo /dev/sdb1 > /sys/fs/bcache/register
echo /dev/sdc1 > /sys/fs/bcache/register
mkfs.ext4 /dev/bcache0
mount /dev/bcache0 /mnt/bcache


Running FIO with random, 100% read, iosize=4096, queue depth=16
the result is approx. 65k IOPS


Write-backe mode (setup is almost the same as write-through except the extra step at the end "echo writeback ...")
================
make-bcache -B /dev/sdb1 -C /dev/sdc1
echo /dev/sdb1 > /sys/fs/bcache/register
echo /dev/sdc1 > /sys/fs/bcache/register
mkfs.ext4 /dev/bcache0
mount /dev/bcache0 /mnt/bcache
echo writeback > /sys/block/bcache0/bcache/cache_mode

Running FIO with random, 100% read, iosize=4096, queue depth=16
the result is approx. 12k IOPS

Running FIO with random, 80% read 20% write, iosize=4096, queue depth=16
the result is approx. 2k IOPS

Thanks,
Patrick

Bcache problem perhaps bug?

Campbell Steven — 2013-05-15T22:03:38

Hi,

We have been using bcache in a test environment here for a while to
vastly improve performance of a CEPH cluster. We have really only run
into one issue, and I'm sure it's something to do with the way we have
things configured (otherwise others would have noticed I'm sure) but
I'm not exactly sure where. We are currently running this
http://atlas.evilpiepirate.org/git/linux-bcache.git/commit/?id=dbc52367ad8e63b5bcb8c6b1442e8fb5ed251421
commit  from the bcache tree after this when we reboot the server it
will hang for ever with timeouts shown in the screenshots of the KVM
below (sorry limited options for collecting debug data via a windows
only KVM):

https://www.btg.co.nz/Public/2.jpg
https://www.btg.co.nz/Public/3.jpg
https://www.btg.co.nz/Public/4.jpg
https://www.btg.co.nz/Public/6.jpg

I have tested a plain mainline kernel without bcache on the same
machine and that works fine as does everything up to and around the
commit from the bcache tree mentioned above. Anything later than that
(up to and including the latest mainline checkout as of today)
exhibits the behaviour above.

Our setup is:

2 x Intel 520 120GB SSD's of which 10 GB is in a RAID 1 for the OS
with LVM over top and the remaining space on both SSD's is a RAID 0
with LVM on top and used as a bcache caching device for the 4 spinning
disks in the servers.

As a reference this post to the list appears to show the same errors
we are getting.

http://comments.gmane.org/gmane.linux.kernel.bcache.devel/1562

I made contact with Aaron, he wasn't able to resolve the issue.

Any pointers (or glaring problems with our setup) gladly accepted.

Thanks

Campbell

RAID5/6 stripe awareness

Kent Overstreet — 2013-05-15T09:48:46

There's now raid5/6 stripe awareness in the bcache branch. I'm running
it on this machine right now.

If your backing device is a raid5 or 6 and you're using normal md raid,
in writeback mode it'll try really hard to buffer up and write out full
stripes:
 * writes to dirty stripes are forced to writeback - even if normally
   the write would've bypassed the cache because of sequential IO
   bypass/congested throttling
 * background writeback preferentially flushes full stripes, if there
   are any

There isn't currently a way to flip on the optimizations for other types
of raid where the settings can't be autodetected (could add an interface
for that if anyone wants).

I haven't benchmarked it yet, should theoretically be a big boost to
write performance - if anyone else benchmarks it I'd love to see.

[GIT PULL] Bcache fixes for 3.10

Kent Overstreet — 2013-05-15T07:53:02

Jens - couple more bcache patches. Bug fixes and a doc update.

The following changes since commit f50efd2fdbd9b35b11f5778ed85beb764184bda9:

  Merge branch 'bcache-for-upstream' of http://evilpiepirate.org/git/linux-bcache into for-3.10/drivers (2013-05-01 09:23:05 +0200)

are available in the git repository at:


  http://evilpiepirate.org/git/linux-bcache.git bcache-for-upstream

for you to fetch changes up to f59fce847fc8483508b5028c24e2b1e00523dd88:

  bcache: Fix error handling in init code (2013-05-15 00:48:14 -0700)

----------------------------------------------------------------
Emil Goode (1):
      bcache: Fix incompatible pointer type warning

Gabriel (1):
      bcache: clarify free/available/unused space

Kent Overstreet (1):
      bcache: Fix error handling in init code

Paul Bolle (1):
      bcache: drop "select CLOSURES"

 Documentation/bcache.txt      |   12 ++-
 drivers/md/bcache/Kconfig     |    1 -
 drivers/md/bcache/bcache.h    |    2 +-
 drivers/md/bcache/stats.c     |   34 ++++----
 drivers/md/bcache/super.c     |  185 ++++++++++++++++++-----------------------
 drivers/md/bcache/writeback.c |    2 +-
 6 files changed, 109 insertions(+), 127 deletions(-)

..

Mr. Dale Alexander — 2013-05-12T23:24:09

I am Mr. Dale Alexander from the bank of Canada, I have a monetary deal
for you. Contact me if interested for more information via: dalealexander-Yld1nPKkoZhBDgjK7y7TUQ< at >public.gmane.org

[PATCH] bcache: fix compilation warning

Vincent Stehlé — 2013-05-12T16:30:51

The release function of a block device does not return a value; make
release_dev void. This fixes the following warning:

  drivers/md/bcache/super.c:653:2: warning: initialization from incompatible pointer type [enabled by default]
  drivers/md/bcache/super.c:653:2: warning: (near initialization for ‘bcache_ops.release’) [enabled by default]

Signed-off-by: Vincent Stehlé <vincent.stehle-QFKgK+z4sOrR7s880joybQ< at >public.gmane.org>
---
 drivers/md/bcache/super.c |    3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
index c8046bc..b09beb2 100644
--- a/drivers/md/bcache/super.c
+++ b/drivers/md/bcache/super.c
< at >< at > -634,11 +634,10 < at >< at > static int open_dev(struct block_device *b, fmode_t mode)
 return 0;
 }
 
-static int release_dev(struct gendisk *b, fmode_t mode)
+static void release_dev(struct gendisk *b, fmode_t mode)
 {
 struct bcache_device *d = b->private_data;
 closure_put(&d->cl);
-return 0;
 }
 
 static int ioctl_dev(struct block_device *b, fmode_t mode,

Patching kernel 3.2

Marcel Ammerlaan — 2013-05-12T13:03:35

Hi,

Although I see the patches being pushed to kernel 3.10, I'd like to
patch a 3.2 kernel as well. However, it seems I cannot clone the git
repository (many timeouts) and git cannot restart a partial clone. Is
it possible to either download the repo as a zipfile or otherwise
receive the repo or a patch containing the changes for a 3.2 kernel.

Regards,

Marcel Ammerlaan.

bcache vs dm-cache

matthew patton — 2013-05-11T04:09:08

so this was a surprise to me in the latest 3.9 kernel. I like the flexibility of the design but the million dollar question is which performs better. Congrats for getting bcache into the queue for 3.10. Are there efforts to combine the best aspects of both projects?

bcache on ubuntu

Giovanni Lenzi — 2013-05-10T10:28:24

Hi,
I want to use bcache on ubuntu.

I saw that I have to download it from git with git clone command. Is it right?

I never recompiled a kernel. Is it so hard?

What kind of steps am I supposed to do, if I have a fresh ubuntu installation?

Thanks for any help.

Giovanni

[PATCH v2] bcache: Reload device size

Gabriel de Perthuis — 2013-05-05T19:33:56

Adds /sys/block/bcache*/bcache/resize; writing "max"
to such a file will reload the device size to match
the size of the underlying device, minus the bcache
superblock.

This is useful to grow a filesystem stacked on top
of bcache and a logical volume.

Other values than max are reserved for now; support
for shrinking might be added if the need comes up.

Signed-off-by: Gabriel de Perthuis <g2p.code+bcache-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>
---
v2: use revalidate_disk to commit changes

 Documentation/bcache.txt   |  4 ++++
 drivers/md/bcache/bcache.h |  1 +
 drivers/md/bcache/super.c  | 10 +++++++++-
 drivers/md/bcache/sysfs.c  | 11 +++++++++++
 4 files changed, 25 insertions(+), 1 deletion(-)

diff --git a/Documentation/bcache.txt b/Documentation/bcache.txt
index acfa679..a53ee2d 100644
--- a/Documentation/bcache.txt
+++ b/Documentation/bcache.txt
< at >< at > -215,10 +215,14 < at >< at > label
 readahead
   Size of readahead that should be performed.  Defaults to 0.  If set to e.g.
   1M, it will round cache miss reads up to that size, but without overlapping
   existing cache entries.
 
+resize
+  Write "max" to this file after resizing the underlying device.
+  bcache will update its device size to match.
+
 running
   1 if bcache is running (i.e. whether the /dev/bcache device exists, whether
   it's in passthrough mode or caching).
 
 sequential_cutoff
diff --git a/drivers/md/bcache/bcache.h b/drivers/md/bcache/bcache.h
index cd185ad..0ccb8f0 100644
--- a/drivers/md/bcache/bcache.h
+++ b/drivers/md/bcache/bcache.h
< at >< at > -1214,10 +1214,11 < at >< at > void bcache_write_super(struct cache_set *);
 int bch_flash_dev_create(struct cache_set *c, uint64_t size);
 
 int bch_cached_dev_attach(struct cached_dev *, struct cache_set *);
 void bch_cached_dev_detach(struct cached_dev *);
 void bch_cached_dev_run(struct cached_dev *);
+void bch_cached_dev_resize(struct cached_dev *);
 void bcache_device_stop(struct bcache_device *);
 
 void bch_cache_set_unregister(struct cache_set *);
 void bch_cache_set_stop(struct cache_set *);
 
diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
index 4c16411..b862910 100644
--- a/drivers/md/bcache/super.c
+++ b/drivers/md/bcache/super.c
< at >< at > -805,10 +805,18 < at >< at > static void calc_cached_dev_sectors(struct cache_set *c)
 sectors += bdev_sectors(dc->bdev);
 
 c->cached_dev_sectors = sectors;
 }
 
+void bch_cached_dev_resize(struct cached_dev *dc)
+{
+set_capacity(
+dc->disk.disk,
+dc->bdev->bd_part->nr_sects - dc->sb.data_offset);
+revalidate_disk(dc->disk.disk);
+}
+
 void bch_cached_dev_run(struct cached_dev *dc)
 {
 struct bcache_device *d = &dc->disk;
 
 if (atomic_xchg(&dc->running, 1))
< at >< at > -1087,11 +1095,11 < at >< at > static const char *register_bdev(struct cache_sb *sb, struct page *sb_page,
 dc->bdev = bdev;
 dc->bdev->bd_holder = dc;
 
 g = dc->disk.disk;
 
-set_capacity(g, dc->bdev->bd_part->nr_sects - dc->sb.data_offset);
+bch_cached_dev_resize(dc);
 
 g->queue->backing_dev_info.ra_pages =
 max(g->queue->backing_dev_info.ra_pages,
     bdev->bd_queue->backing_dev_info.ra_pages);
 
diff --git a/drivers/md/bcache/sysfs.c b/drivers/md/bcache/sysfs.c
index 6b9f0ee..69a5d13 100644
--- a/drivers/md/bcache/sysfs.c
+++ b/drivers/md/bcache/sysfs.c
< at >< at > -26,10 +26,11 < at >< at > write_attribute(unregister);
 write_attribute(stop);
 write_attribute(clear_stats);
 write_attribute(trigger_gc);
 write_attribute(prune_cache);
 write_attribute(flash_vol_create);
+write_attribute(resize);
 
 read_attribute(bucket_size);
 read_attribute(block_size);
 read_attribute(nbuckets);
 read_attribute(tree_depth);
< at >< at > -200,10 +201,19 < at >< at > STORE(__cached_dev)
 
 if (attr == &sysfs_running &&
     strtoul_or_return(buf))
 bch_cached_dev_run(dc);
 
+if (attr == &sysfs_resize) {
+if (!strcmp(buf, "max") || !strcmp(buf, "max\n"))
+bch_cached_dev_resize(dc);
+else
+/* Reserved for future use, if someone needs sizes below the max.
+ * memparse units would be consistent with fs resizing tools. */
+return -EINVAL;
+}
+
 if (attr == &sysfs_cache_mode) {
 ssize_t v = bch_read_string_list(buf, bch_cache_modes + 1);
 
 if (v < 0)
 return v;
< at >< at > -287,10 +297,11 < at >< at > static struct attribute *bch_cached_dev_files[] = {
 &sysfs_dirty_data,
 &sysfs_sequential_cutoff,
 &sysfs_sequential_merge,
 &sysfs_clear_stats,
 &sysfs_running,
+&sysfs_resize,
 &sysfs_state,
 &sysfs_label,
 &sysfs_readahead,
 #ifdef CONFIG_BCACHE_DEBUG
 &sysfs_verify,