gmane.linux.kernel.bcache.devel

Re: [PATCH] md: bcache: Fixed a typo with the word 'arithmetic'

Jiri Kosina — 2013-06-18T11:42:18


I don't see this in linux-next as of today, so I am taking it. Thanks,

Re: Performance Comparison among EnhanceIO, bcache and dm-cache.

Joe Thornber — 2013-06-17T10:29:44

...


Not true (though it is true for thinp, which may be where you got this
idea?).  For caching we have to commit whenever data is moved about,
otherwise a crash could result in us reading data that is not just out
of date (acceptable for some), but used to belong to a totally
different part of the device (always unacceptable).

- Joe

Re: Possible bug? bcache: error opening /dev/md126: Not enough buckets

John Clark — 2013-06-16T03:21:30


Jason,

Thanks for the advice and vote of confidence in 3.10-rc5.

I just want to report that I successfully built and installed kernel
3.10-rc6 (committed by Linus a few hours ago). I chose rc6 because I
noticed that some fixes for filesystem-related issues had be added to
rc6.

I'm happy to report that this has resolved my problem. make-bcache now works.

However (in case anyone else runs into this problem) even after
deploying kernel 3.10-rc6 make-bcache failed for me initially. This
time the error was "device or resource busy" - as though some process
had my /dev/md126 and/or /dev/md/127 devices open. Weirder still was
that sometimes (after a reboot) make-bcache succeeded only for my SSD
raid device and other times only for my spinning HDD raid device.

In the course of trying to figure out what the heck was going on I
used 'dd' to write zero's to the beginning of both raid devices. From
that point on make-bcache worked for both devices. I presume that
there might have been some data at the start of th

Re: Possible bug? bcache: error opening /dev/md126: Not enough buckets

Jason Warr — 2013-06-14T14:45:13

Build from the 3.10 RC series.  It may not be "stable" yet but should be
within a few weeks I think and it has the bcache bits in it.

I have been running from the bcache-for3.10 tree on CentOS 6 for a few
months without any issues.  I will be using the mainline 3.10-RC5 tree
sometime over the weekend.

Possible bug? bcache: error opening /dev/md126: Not enough buckets

John Clark — 2013-06-14T05:34:02

Hi there,

I'm new to kernel mailing lists so beg your pardon if I'm asking this
question in the wrong place or in the wrong way.

Anyway, I'm tantalisingly close to getting my bcache-enabled system
up-and-running, but I've hit a brick wall which seems might be a bug
or undocumented performance limit of some kind. I'm hoping Kent or
others on this fine list can help.

I have successfully created the following two RAID1 block devices, as
shown per /proc/mdstat:

Personalities : [raid1]
md126 : active raid1 sdb[0] sdc[1]
      1953383360 blocks super 1.2 [2/2] [UU]

md127 : active raid1 sdd[0] sde[1]
      117155200 blocks super 1.2 [2/2] [UU]

/dev/md126 comprises two 2.0TB spinning HDD's which I'm intending to
use as a bcache backing device
/dev/md127 comprises two 60GB SSD's which I'm intending to use as a
bcache cache device

So far so good.

Creating the bcache superblock on the md127 appears to go smoothly:

root< at >bigdata:~# make-bcache -C /dev/md127
UUID:                   1736b20f-6b85-4fee-a801-4cf7c1b

(b)cache trashing

Piergiorgio Sartor — 2013-06-13T20:33:49

Hi all,

one (or two), naive, question.

I assume that bcache will cache on reads too.

What will happen if something like this is run:

find /home -type -f -exec md5sum {} \;

That is, the md5 of *all* files is computed.

I assume that the cache will be completely
trashed and no old data will be present.

Is this the case?

Now, assuming we have a RAID something, with
LVM on top, with different volumes.
For example one for /home, one for /usr, etc.
Would it make sense, in respect of the above
trashing, to have different SSD (or paritions)
to cache the different volumes?
Instead of caching the complete RAID, of course.

Thanks a lot in advance,

bye,

[PATCH] bcache: Set the logical block size to a sensible value

Gabriel de Perthuis — 2013-06-13T20:25:02

Preserve the backing device's logical size, use 512 byte
sectors when there is no backing device.

The logical block size has no impact on performance, but
it alters the meaning of on-disk structures like partition
tables.  Preserve the backing device's sector size to keep
bcache transparent, and use 512 byte sectors like everyone
else when no backing device is present.

Signed-off-by: Gabriel de Perthuis <g2p.code-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>
---
 drivers/md/bcache/super.c | 9 +++++----
 1 file changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/md/bcache/super.c b/drivers/md/bcache/super.c
index 1e3bc4c..94fad70 100644
--- a/drivers/md/bcache/super.c
+++ b/drivers/md/bcache/super.c
< at >< at > -745,11 +745,12 < at >< at > static void bcache_device_free(struct bcache_device *d)
 bioset_free(d->bio_split);
 
 closure_debug_destroy(&d->cl);
 }
 
-static int bcache_device_init(struct bcache_device *d, unsigned block_size)
+static int bcache_device_init(struct bcache_device *d, unsigned block_size,
+

RE: Performance Comparison among EnhanceIO, bcache and dm-cache.

OS Engineering — 2013-06-12T11:39:30

We have chosen a representative workload for our tests, we could do more experiments  
that tests SSD to its performance limits.  



Yes, READFILL's (SSD writes at the time of read of an uncached block) is the reason for higher read latency in EnhanceIO. 


 
EnhanceIO has higher write latencies as it acknowledges IO completion only when both data and metadata has
hit the SSD.

Re: Performance Comparison among EnhanceIO, bcache and dm-cache.

Amit Kale — 2013-06-12T04:58:03

So it won't be fair to compare them with dm-cache in write-back mode since 
guarantees are different. I am sure if similar (SYNC/FUA enforcement for 
persistence) guarantees are implemented in bcache/EnhanceIO, they'll offer a 
much better performance. 


Did you experiment a little with varying iodepth and numjobs? Is this the best 
combination you found out? The numbers below appear low considering a 90% hit 
ratio for EnhanceIO. SSD baseline performance shown below appears low. However 
it should not affect this comparison. All caching solutions are being subjected 
to the same ratio of HDD/SSD performance.


EnhanceIO shows much higher read latency here. Is that because of READFILLs ? 
Write latency, read-write throughputs are good.


Here EnhanceIO's read latency is better compared the other two. Write latency 
is larger than the other two. 

-Amit

(unknown)

OS Engineering — 2013-06-11T14:54:05

Hi Jens,

In continuation with our previous communication, we have carried out performance comparison among EnhanceIO, bcache and dm-cache.

We found that EnhanceIO provides better throughput on zipf workload (with theta=1.2) in comparison to bcache and dm-cache for write through caches.
However, for write back caches, we found that dm-cache had best throughput followed by EnhanceIO and then bcache. Dm-cache commits on-disk metadata every time a REQ_SYNC or REQ_FUA bio is written. If no such requests are made then it commits metadata once every second. If power is lost, it may lose some recent writes. However, EnhanceIO and bcache do not acknowledge IO completion until both IO and metadata hits the SSD. Hence, EnhanceIO and bcache provide higher data integrity at a cost of performance.

The fio config and setup information follows:
HDD              : 100GB
SSD              :  20GB
Mode             : write through / write back
Cache block_size : 4KB for bcache, EnhanceIO and 256KB for dm-cache

The other opti

Performance Comparison among EnhanceIO, bcache and dm-cache.

OS Engineering — 2013-06-11T15:05:07

Hi Jens,

In continuation with our previous communication, we have carried out performance comparison among EnhanceIO, bcache and dm-cache.

We found that EnhanceIO provides better throughput on zipf workload (with theta=1.2) in comparison to bcache and dm-cache for write through caches.
However, for write back caches, we found that dm-cache had best throughput followed by EnhanceIO and then bcache. Dm-cache commits on-disk metadata every time a REQ_SYNC or REQ_FUA bio is written. If no such requests are made then it commits metadata once every second. If power is lost, it may lose some recent writes. However, EnhanceIO and bcache do not acknowledge IO completion until both IO and metadata hits the SSD. Hence, EnhanceIO and bcache provide higher data integrity at a cost of performance.

The fio config and setup information follows:
HDD              : 100GB
SSD              :  20GB
Mode             : write through / write back
Cache block_size : 4KB for bcache, EnhanceIO and 256KB for dm-cache

The other opti

Re: Poor performance with bcache write-back mode

Patrick Ng — 2013-06-03T21:44:59

bcache write-through mode, the
mode, I got very poor performance.
extra step at the end "echo

Hi Kent, thank you for responding to my question.

I have tried setting both congested_read_threshold_us and 
congested_write_threshold_us to zero.

The result of FIO random 100% read test in WriteBack mode has improved and 
the IOPS is as good as in WriteThur mode.

The result of FIO random 80% read 20% write remain the same with no 
improvement.

It appears that the option does not help on the WRITE?

Thanks,
Patrick

Re: bcache on top of drbd or the contrary?

Kent Overstreet — 2013-06-03T19:14:57

If you're using bcache in writethrough mode, running bcache on top of
drbd is fine and may be faster (but benchmark it if you care).

Reason being in writethrough mode the SSD failing won't hurt anything.

If you're using bcache in writeback mode, _definitely_ run drbd on top
of bcache.

In writeback mode, if the SSD fails the backing device is now useless.
If you were using bcache under drbd, no big deal - you have a replica of
the backing device. If you're using bcache on top of drbd... well,
that's /dev/drbd0 that's now toast, and you're screwed.

[PATCH] md: bcache: Fixed a typo with the word 'arithmetic'

Phil Viana — 2013-06-03T12:51:42

The word 'arithmetic' was typed as 'arithmatic'

Signed-off-by: Phil Viana <phillip.l.viana-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>
---
 drivers/md/bcache/bset.c |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/md/bcache/bset.c b/drivers/md/bcache/bset.c
index d33f5ad..8822ed2 100644
--- a/drivers/md/bcache/bset.c
+++ b/drivers/md/bcache/bset.c
< at >< at > -403,7 +403,7 < at >< at > void inorder_test(void)
 #endif
 
 /*
- * Cacheline/offset <-> bkey pointer arithmatic:
+ * Cacheline/offset <-> bkey pointer arithmetic:
  *
  * t->tree is a binary search tree in an array; each node corresponds to a key
  * in one cacheline in t->set (BSET_CACHELINE bytes).
< at >< at > -412,7 +412,7 < at >< at > void inorder_test(void)
  * the binary tree points to; to_inorder() gives us the cacheline, and then
  * bkey_float->m gives us the offset within that cacheline, in units of 8 bytes.
  *
- * cacheline_to_bkey() and friends abstract out all the pointer arithmatic to
+ * cacheline_to_bkey() and friends abstract out all the p

bcache on top of drbd or the contrary?

Giovanni Lenzi — 2013-06-01T15:45:09

Hi,
I'm planning to user bcache for speed, and drbd for HA between two nodes.

I'm facing with this question:
Is it better to have drbd on top of a bcached volume(caching + backing device)
or to use bcache to put a caching device on top of a drbd backing device?

I think that with the second option and writeback enabled, write
requests should be faster than with the first option... Do you agree?

But.. what about the data integrity and consistency of the data on the
drbd slave volume? What if the power goes down on the master node?
Could the drbd volume on the slave node, get corrupted If the caching
device is not detached correctly on the master node?

On the contrary with the first option(drbd on top of bcache), I think
drbd should always preserve data integrity on the two nodes, isn't it?
Can bcache with writeback mode, cause corrupted data on both nodes in
case of a ssd, or power unit failure?

And where is it better to put drbd metadata? External or internal?
External to ssd is the obvious answer, but i

[PATCH] md: bcache: Fixed a spelling mistake with the word 'journaling' in a comment

Phil Viana — 2013-05-31T02:42:38

The word 'journaling' is spelt with a single 'l' throughout the Linux
source code, and this spelling is also found in the Oxford dictionary
and used in other projects in general.

Signed-off-by: Phil Viana <phillip.l.viana-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>
---
 drivers/md/bcache/bset.c |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/md/bcache/bset.c b/drivers/md/bcache/bset.c
index a0f190a..d33f5ad 100644
--- a/drivers/md/bcache/bset.c
+++ b/drivers/md/bcache/bset.c
< at >< at > -31,7 +31,7 < at >< at > int bch_keylist_realloc(struct keylist *l, int nptrs, struct cache_set *c)
 unsigned newsize = oldsize + 2 + nptrs;
 uint64_t *new;
 
-/* The journalling code doesn't handle the case where the keys to insert
+/* The journaling code doesn't handle the case where the keys to insert
  * is bigger than an empty write: If we just return -ENOMEM here,
  * bio_insert() and bio_invalidate() will insert the keys created so far
  * and finish the rest when the keylist is empty.

Re: [PATCH] md: bcache: io.c: fix a potential NULL pointer dereference

Kumar amit mehta — 2013-05-29T00:01:13

Using smatch[1]

[1] http://smatch.sourceforge.net/

Re: Poor performance with bcache write-back mode

Kent Overstreet — 2013-05-29T00:40:58

You're probably getting bit by the congested bypass - try setting
congested_read_threshold_us and congested_write_threshold_us both to 0.

I wish there was a better solution to the congestion bypass stuff, it's
a hard problem...

Re: [PATCH] md: bcache: io.c: fix a potential NULL pointer dereference

Kent Overstreet — 2013-05-29T00:20:41

Whoops, that's definitely a bug. Thanks, applied.

How'd you find it?

[PATCH] md: bcache: io.c: fix a potential NULL pointer dereference

Kumar Amit Mehta — 2013-05-28T07:31:15

bio_alloc_bioset returns NULL on failure. This fix adds a missing check
for potential NULL pointer dereferencing.

Signed-off-by: Kumar Amit Mehta <gmate.amit-Re5JQEeQqe8AvxtiuMwx3w< at >public.gmane.org>
---
 drivers/md/bcache/io.c |    2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/md/bcache/io.c b/drivers/md/bcache/io.c
index 29f344b..4be2a07 100644
--- a/drivers/md/bcache/io.c
+++ b/drivers/md/bcache/io.c
< at >< at > -98,6 +98,8 < at >< at > struct bio *bch_bio_split(struct bio *bio, int sectors,
 
 if (bio->bi_rw & REQ_DISCARD) {
 ret = bio_alloc_bioset(gfp, 1, bs);
+if (!ret)
+return NULL;
 idx = 0;
 goto out;
 }

Re: EnhanceIO(TM) caching driver features [1/3]

Amit Kale — 2013-05-25T16:00:27

Got to use a different email client for that. Note that I am writing this from 
my personal email address. This email and any future emails I write from this 
address are my personal views and sTec can't be held responsible for them.


That's correct. I haven't seen any application level benchmark based 
comparisons between different caching solutions on any platform.


Yes. However as I have said above, application level testing is primarly a 
test of cache replacement algorithm. So the effect of a platform is less, 
although not zero.


Agreed that's a fair thing to ask.


While the running a test is trivial deciding what IO patterns to run is a 
difficult problem. The bottom line for sequential, random, zipf and pareto is 
the same - they all test a fixed IO pattern, which at best is very unlike an 
application pattern. Cache behavior affects IO addresses that an application 
issues. The block list of IOs requested by an application is different when 
running on HDD, SSD and cache. Memory usage by cache