gmane.comp.file-systems.tux3

Re: Tux3 Report: Faster than tmpfs, what?

Andreas Dilger — 2013-05-15T17:10:52

The new inode allocation is only needed for the truncate-to-zero case. If the inode is being deleted it is used directly. 

Sorry for confusion, it has been a long time since I looked at that code. 

Cheers, Andreas

Re: Tux3 Report: Faster than tmpfs, what?

OGAWA Hirofumi — 2013-05-14T07:59:29


Thanks for adivce.

Re: Tux3 Report: Faster than tmpfs, what?

Dave Chinner — 2013-05-14T06:34:38

Because nobody could reproduce your results without working that
out. You didn't disclose that you'd made these changes, and that
makes it extremely misleading as to what the results mean. Given the
headline-grab nature of it, it's deceptive at best.

I don't care how fast tux3 is - I care about being able to reproduce
other people's results. Hence if you are going to report benchmark
results comparing filesystems then you need to tell everyone exactly
what you've tweaked and why, from the hardware all the way up to the
benchmark config.

Work on how *you* report *your* results - don't let Daniel turn them
into some silly marketing fluff that tries to grab headlines.

-Dave.

Re: Tux3 Report: Faster than tmpfs, what?

Daniel Phillips — 2013-05-14T06:25:28

Interesting, Andreas. We don't do anything as heavyweight as
allocating an inode in this path, just mark the inode dirty (which
puts it on a list) and set a bit in the inode flags.

Regards,

Daniel

Re: Tux3 Report: Faster than tmpfs, what?

Andreas Dilger — 2013-05-14T00:08:16


Ages ago, before we implemented extents for ext3, we had an asynchronous unlink/truncate-to-zero thread that was handing the busywork of traversing the indirect tree and updating all of the bitmaps.

This was transactionally safe, since the blocks were moved over to a temporary inode in the main process' transaction, and the unlinked inode was on the orphan list.

With the extent-mapped inodes the latency of the unlink/truncate-to-zero was greatly reduced, and we dropped that code. If anyone is interested to revive this for some reason, the newest version I could find was for 2.4.24:

http://git.whamcloud.com/?p=fs/lustre-release.git;a=blob_plain;f=lustre/kernel_patches/patches/ext3-delete_thread-2.4.24.patch;hb=21420e6d66eaaf8de0342beab266460c207c054d

IIRC, it only pushed unlink/truncate to the thread if it had indirect blocks, since the effort of allocating a separate inode and transferring over the allocated blocks wasn't worthwhile otherwise.

Cheers, Andreas___________________________________________

Re: Updated pre-merge checklist

Raymond Jennings — 2013-05-14T00:50:10

I hope so soon.

I'm well aware of this experimental thing...or rather I was, but I think
that mainline recently or is about to remove CONFIG_EXPERIMENTAL.


On Mon, May 13, 2013 at 2:34 AM, OGAWA Hirofumi <hirofumi< at >mail.parknet.co.jp

_______________________________________________
Tux3 mailing list
Tux3< at >phunq.net
http://phunq.net/mailman/listinfo/tux3

Re: Tux3 Report: Faster than tmpfs, what?

Daniel Phillips — 2013-05-13T23:22:56

Hi Ted,

You said:


After after pondering it for a while, I realized that is not
completely accurate. The reduced delete latency will
allow the dbench process to proceed to the fsync point
faster, then if our fsync is reasonably efficient (not the
case today, but planned) we may still see an overall
speedup.


Nothing stops our frontend from calling its backend
synchronously, which is just what we intend to do for
fsync. The real design issue for Tux3 fsync is writing
out the minimal set of blocks to update a single file.
As it is now, Tux3 commits all dirty file data at each
delta, which is fine for many common loads, but not
all. Two examples of loads where this may be less
than optimal:

  1) fsync (as you say)

  2) multiple tasks accessing different files

To excel under those loads, Tux3 needs to be able to
break its "always commit everything rule" in an
organized way. We have considered several design
options for this but not yet prototyped any because we
feel that that work can reasonably be attacke

Re: Tux3 - mmap and snapshotting

OGAWA Hirofumi — 2013-05-13T09:43:29


For now, if mmap write happens, kernel/fs would be the cause of kernel
panic. mmap read on read-only file should works though.

Because we are not using generic routine in kernel. So, we have to
implement mmap write by our way.


For now, we are not caring fsfreeze. So I think it would not be safe, or
it doesn't work.

I'm not checking how many jobs need for fsfreeze support, so I'm not
sure how easy/hard to implement it.


Benchmarks was done on kvm on host using HDD. Fragmentation avoidance,
and optimize for rotating drive is we are currently working one.

Honestly, we would be still far from production use. tux3 was good on
some benchmark loads before optimizing though.

Thanks.

Re: Updated pre-merge checklist

OGAWA Hirofumi — 2013-05-13T09:34:18


I am personally targetting around Auguest. To merge, we would need to
provide nice base for developers can work.

Probably, allocation policy, and some random small things like "."/"..".


BTW, even if merged to mainline, I think it would still be experimental
for a long while.

Thanks.

Tux3 - mmap and snapshotting

Maciej Piechotka — 2013-05-12T19:21:07

Hi,

I've become interested in testing tux3 but I have a few questions before I try 
it out:

1. In todo list there was mentioned mmap support. What's happening currently 
on mmap call (does it fail or is it just using for example slower emulation by 
kernel using other entry points)?

2. Is tux3 lvm snapshot safe (i.e. I can create a snapshot while fs is 
mounted)? I understend that tux3 native snapshotting will be added in future 
but some form of snapshotting is required by my current backup strategy

3. Have the benchmarks been done using sdd or traditional hdd? How does tux3 
handles fragmentation if it was sdd?

Best regards

Updated pre-merge checklist

Raymond Jennings — 2013-05-12T17:50:38

When will tux3 be ready to merge to the kernel?

Also, what else needs to b edone?

Those latest performance numbers have me drooling and I can't wait to
migrate.
_______________________________________________
Tux3 mailing list
Tux3< at >phunq.net
http://phunq.net/mailman/listinfo/tux3

Re: Tux3 Report: Faster than tmpfs, what?

Daniel Phillips — 2013-05-12T04:39:10

On Sat, May 11, 2013 at 11:35 AM, james northrup
<northrup.james< at >gmail.com> wrote:

Could you please be specific about the meaning you intend? Because
innuendo is less than useful in this forum. If you mean to say that
our posted results might not be independently verifiable then I invite
you to run the tests as described (including removing fsync) yourself.
If you require any assistance from us in doing that, we will be
pleased to provide it.

Regards,

Daniel

Re: Tux3 Report: Faster than tmpfs, what?

Daniel Phillips — 2013-05-12T04:28:46

(resent as plain text)

On Sat, May 11, 2013 at 2:26 PM, Theodore Ts'o <tytso< at >mit.edu> wrote:

Exactly, Ted. We avoided measuring the fsync load on this particular
benchmark because we have not yet optimized fsync. When we do get to
it (not an immediate priority) I expect Tux3 to perform competitively,
because our delta commit scheme does manage the job with a minimal
number of block writes. To have a really efficient fsync we need to
isolate just the changes for the fsynced file into a special "half
delta" that gets its own commit, ahead of any other pending changes
to the filesystem. There is a plan for this, however we would rather
not get sidetracked on that project now while we are getting ready
for merge.

The point that seems to be getting a little lost in this thread is,
the benchmark just as we ran it models an important and common type
of workload, arguably the most common workload for real users, and
the resulting performance measurement is easily reproducible for
anyone who cares to try. In fact,

Re: Tux3 Report: Faster than tmpfs, what?

Daniel Phillips — 2013-05-12T04:16:29

(resent as plain text)

On Sat, May 11, 2013 at 2:26 PM, Theodore Ts'o <tytso< at >mit.edu> wrote:

Exactly, Ted. We avoided measuring the fsync load on this particular
benchmark because we have not yet optimized fsync. When we do get to it
(not an immediate priority) I expect Tux3 to perform competitively, because
our delta commit scheme does manage the job with a minimal number of block
writes. To have a really efficient fsync we need to isolate just the
changes for the fsynced file into a special "half delta" that gets its own
commit, ahead of any other pending changes to the filesystem. There is a
plan for this, however we would rather not get sidetracked on that now,
while we are getting ready for merge.

The point that seems to be getting a little lost in this thread is, the
benchmark just as we ran it models an important and common type of
workload, arguably the most common workload for real users, and the
resulting performance measurement is easily reproducible for anyone who
cares to try. In fact, I thin

Re: Tux3 Report: Faster than tmpfs, what?

Theodore Ts'o — 2013-05-11T21:26:08

Dropping fsync() does a lot more than "amplify Tux3's advantage in
delete performace".  Since fsync(2) is defined as not returning until
the data written to the file descriptor is flushed out to stable
storage --- so it is guaranteed to be seen after a system crash --- it
means that the foreground application must not continue until the data
is written by Tux3's back-end.

So it also means that any advantage of decoupling the front/back end
is nullified, since fsync(2) requires a temporal coupling.  In fact,
if there is any delays introdued between when the front-end sends the
fsync request, and when the back-end finishes writing the data and
then communicates this back to the front-end --- i.e., caused by
schedular latencies, this may end up being a disadvantage compared to
more traditional file system designs.

Like many things in file system design, there are tradeoffs.  It's
perhaps more quseful when having these discussions to be clear what
you are trading off for what; in this case, the front/back des

Re: Tux3 Report: Faster than tmpfs, what?

james northrup — 2013-05-11T18:35:45

also interesting information... Study of 2,047 papers on PubMed finds
that two-thirds of retracted papers were down to scientific
misconduct, not error

On Fri, May 10, 2013 at 11:12 PM, Daniel Phillips
<daniel.raymond.phillips< at >gmail.com> wrote:

Re: Tux3 Report: Faster than tmpfs, what?

Daniel Phillips — 2013-05-12T01:10:24

Exactly, Ted. We avoided measuring the fsync load on this particular
benchmark because we have not yet optimized fsync. When we do get to it
(not an immediate priority) I expect we will perform competitively, because
Tux3 does manage to get deltas onto disk with a minimal number of block
writes.

Regards,

Daniel
_______________________________________________
Tux3 mailing list
Tux3< at >phunq.net
http://phunq.net/mailman/listinfo/tux3

Re: Tux3 Report: Faster than tmpfs, what?

Daniel Phillips — 2013-05-11T06:12:27

Hi Dave,

Thanks for the catch - I should indeed have noted that "modified
dbench" was used for this benchmark, thus amplifying Tux3's advantage
in delete performance. This literary oversight does not make the
results any less interesting: we beat Tmpfs on that particular load.
Beating tmpfs at anything is worthy of note. Obviously, all three
filesystems ran the same load.

We agree that "classic unadulterated dbench" is an important Linux
benchmark for comparison with other filesystems. I think we should
implement a proper fsync for that one and not just use fsync = sync.
That isn't very far in the future, however our main focus right now is
optimizing spinning disk allocation. It probably makes logistical
sense to leave fsync as it is for now and concentrate on the more
important issues.

I do not agree with your assertion that the benchmark as run is
invalid, only that the modified load should have been described in
detail. I presume you would like to see a new bakeoff using "classic"
dbench. Patience ple

Re: Tux3 Report: Faster than tmpfs, what?

OGAWA Hirofumi — 2013-05-10T05:47:35


Right. Because tux3 is not implementing fsync() yet. So, I did

grep -v Flush /usr/share/dbench/client.txt > client2.txt

Why is it important for comparing?


Our backend is still using debugging mode (flush each 10 transactions
for stress/debugging). Since no interface to use normal writeback
timing yet, and I'm not tackling about it yet.

And if normal writeback can't beat crappy fixed timing (4 secs), Rather,
it means we have to improve writeback timing. I.e. sync should be rather
slower than best timing, right?


Simply wrong. I did this to start optimization of tux3 (We know we have
many places to optimize in tux3), but the result was that post. If you
can't see at all what we did by frontend/backend design from that, I'm a
bit sad for it.

I think I can improve tmpfs/ext4 like tux3 (Unlink/Deltree) if I want to
do, from this result.

Thanks.

Re: Tux3 Report: Faster than tmpfs, what?

Dave Chinner — 2013-05-10T04:50:49

I'm deeply suspicious of what is in that client2.txt file. dbench on
ext4 on a 4 SSD RAID0 array with a single process gets 130MB/s
(kernel is 3.9.0). Your workload gives you over 1GB/s on ext4.....

....

Hmmm... No "Flush" operations. Gotcha - you've removed the data
integrity operations from the benchmark.

Ah, I get it now - you've done that so the front end of tux3 won't
encounter any blocking operations and so can offload 100% of
operations. It also explains the sync call every 4 seconds to keep
tux3 back end writing out to disk so that a) all the offloaded work
is done by the sync process and not measured by the benchmark, and
b) so the front end doesn't overrun queues and throttle or run out
of memory.

Oh, so nicely contrived. But terribly obvious now that I've found
it.  You've carefully crafted the benchmark to demonstrate a best
case workload for the tux3 architecture, then carefully not
measured the overhead of the work tux3 has offloaded, and then not
disclosed any of this in the hope that all

Re: [PATCH] kernel: fix a compile error

OGAWA Hirofumi — 2013-05-08T12:38:45


OK. Looks like the bug of gcc. (BTW, gcc 4.7.2 doesn't warn about it.)
I will fix this by uninitialized_var().

Thanks.



Fix the following warning.

/home/yliu/tux3/user/kernel/dir.c: In function `tux_create_entry':
/home/yliu/tux3/user/kernel/dir.c:46:9: error: `name_len' may be used uninitialized in this function [-Werror=uninitialized]
/home/yliu/tux3/user/kernel/dir.c:130:47: note: `name_len' was declared here

Signed-off-by: Liu Yuan <tailai.ly< at >taobao.com>
[Use uninitialized_var()]
Signed-off-by: OGAWA Hirofumi <hirofumi< at >mail.parknet.co.jp>
---

 user/kernel/dir.c |    3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff -puN user/kernel/dir.c~gcc-4.6-name_len-warn-fix user/kernel/dir.c
--- tux3/user/kernel/dir.c~gcc-4.6-name_len-warn-fix2013-05-08 21:33:06.000000000 +0900
+++ tux3-hirofumi/user/kernel/dir.c2013-05-08 21:33:45.000000000 +0900
< at >< at > -127,7 +127,8 < at >< at > loff_t tux_create_entry(struct inode *di
 struct sb *sb = tux_sb(dir->i_sb);
 tux_dirent *entry;
 struct buffer_head *buffer, *