http://permalink.gmane.org/gmane.comp.file-systems.ext4 hourly 1 1901-01-01T00:00+00:00 http://gmane.org/img/gmane-25t.png http://gmane.org http://permalink.gmane.org/gmane.comp.file-systems.ext4/32401 <pre>https://bugzilla.kernel.org/show_bug.cgi?id=42895 --- Comment #15 from Theodore Tso &lt;tytso&lt; at &gt;mit.edu&gt; 2012-05-16 20:24:41 --- Eugene, one thing which may not be obvious is that it's not the number of disk blocks that are written; it's how often the disk drive seeks, and more importantly, wakes up. Once you wake up the hard drive, and the hard drive has positioned the hard drive heads to the right place on disk, whether you write 8k or 32k or 128k doesn't make that much difference in terms of time and power requirements. So for example, if gedit writes a single file, it might cause multiple jbd2 writes: for the block allocation bitmap, for the inode allocation bitmap, the inode table, the block group summary block, and then the data block itself. But the jbd2 writes are contiguous, and happen all at once, during a journal commit (which is caused either by an explicit fsync or by the 5 second commit timer which starts a journal commit 5 seconds after metadata changes have been applied to the file system</pre> bugzilla-daemon< at >bugzilla.kernel.org 2012-05-16T20:24:41 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32400 <pre>https://bugzilla.kernel.org/show_bug.cgi?id=42895 --- Comment #14 from Jan Kara &lt;jack&lt; at &gt;suse.cz&gt; 2012-05-16 19:29:54 --- I've looked at the results and they look normal. Different processes dirty some inodes - after a while jbd2 journal thread writes these changes to the journal (it checks once per 5s). The journal thread writes somewhat more because there are some descriptor blocks etc. Also once per 5s, flusher wakes up and writes all dirty data older than 30s. So really sources of all the activity are those few processes (gedit, NetworkManager, xfconfd) which do dirtying and the rest is just triggered by that. </pre> bugzilla-daemon< at >bugzilla.kernel.org 2012-05-16T19:29:54 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32399 <pre> I agree with you, this issue is not big enough to be worth reordering ext4 priorities and making an interim fix. I don't think it has actually inconvenienced anyone at all, but merely came to my notice when I was trying to work out the correct behaviour for tmpfs. However, the issues that Jan is grappling with in "Hole punching and mmap races" seem more serious, and may end up affecting or solving this one too. Hugh -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Hugh Dickins 2012-05-16T18:09:53 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32398 <pre>https://bugzilla.kernel.org/show_bug.cgi?id=42895 --- Comment #13 from Eugene &lt;ivanovstm2&lt; at &gt;zmail.ru&gt; 2012-05-16 16:34:39 --- Thanks, now it works. Traced system for around 10mins and I don't see previously unseen processes. Just normal system activity. But there was a number of hdd writes (around 6 in 10mins) which doesn't have corresponding entries in trace results and separated from entries by roughly 30sec (flush?) - that's not including 1-2 writes closely following (~3-5sec) almost each entry (jbd2?). http://paste.ubuntu.com/990927/ Can I trace some function which gives output in trace results when actual write takes place but shows initial cause of the write (not just flush or jbd2)? Like block_dump but more informative? If it's not possible then for now I'm going to presume that it's just the way how system works. But still it's lots of flushes/jbd2 following writes of even small files: https://launchpadlibrarian.net/101378473/15mins_almost_idle.txt For example in above block_dump Network Manag</pre> bugzilla-daemon< at >bugzilla.kernel.org 2012-05-16T16:34:39 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32397 <pre> Still not sure how you got a filesystem w/o that feature though, unless I am forgetting something obvious. Isn't it on by default? -Eric -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Eric Sandeen 2012-05-16T15:49:32 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32395 <pre>From: Saugata Das &lt;saugata.das&lt; at &gt;linaro.org&gt; On eMMC and UFS devices there is a new feature of setting context with each read or write. The idea is to classify the data from different files and apply the realibility on the complete file instead of individual writes, which helps in performance. A new address space operation has been a added to get the context from file system and set up the bi_context field in bio. Then we need to ensure that bio from different contexts are not merged. The context is then passed to the underlying driver as part of the read or write request. Since the number of MMC contexts is limited, multiple file system contexts are mapped to single MMC context. Signed-off-by: Saugata Das &lt;saugata.das&lt; at &gt;linaro.org&gt; --- block/blk-core.c | 1 + block/blk-merge.c | 3 +++ fs/mpage.c | 12 ++++++++++++ include/linux/blk_types.h | 1 + include/linux/blkdev.h | 1 + include/linux/buffer_head.h | 2 ++ include/linux/fs.h | 1 </pre> Saugata Das 2012-05-16T15:30:28 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32393 <pre> there are regression tests in the e2fsprogs git tree itself. I think a prepared filesytem image may be the way to go for this one. Oh, sorry - this "tried to pick" is what I meant to say :) Ok. Thanks. -Eric -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Eric Sandeen 2012-05-16T15:31:01 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32392 <pre>thank you Andreas, and I will change it in the next version. Thanks Tao -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Tao Ma 2012-05-16T15:10:16 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32391 <pre>sure, but could you please tell me where I can find the repo? No, it doesn't. With fallocate(FALLOC_FL_KEEP_SIZE), we have no idea of what is the last block until we iterate the last leaf ext4_extent. As I have mentioned above, now there is no way for us to tell the end block of a file at the very beginning of ext2fs_extent_open2, so actually any value less than u32 could be OK if we have a sparse file while the last block is fallocated near the end of u32 logical block offset. Actually path[0]-&gt;end_blk is only used when we have no idea of the length of the last ext4_extent_idx. See ext2fs_extent_get. if (path-&gt;left &gt; 0) { ix++; newpath-&gt;end_blk = ext2fs_le32_to_cpu(ix-&gt;ei_block); } else newpath-&gt;end_blk = path-&gt;end_blk; Having said that, I have to admit that I didn't think of the case of ext3 and I am not sure whether this change will affect it or not. Thanks Tao -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More </pre> Tao Ma 2012-05-16T15:07:42 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32390 <pre> Minor coding style fix - when one half of if/else is using braces then the other half should also use braces, like: if { : } else { : } Cheers, Andreas -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Andreas Dilger 2012-05-16T14:58:23 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32389 <pre>sure, I will add it in v2. See my comments below. I found it when running xfstests 269. Thanks Tao -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Tao Ma 2012-05-16T14:55:05 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32388 <pre> Thanks for your comments. I will take care of this in the next version. -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Saugata Das 2012-05-16T14:06:36 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32387 <pre> Should this be put into an e2fsprogs regression test? Hm, so this picked the actual last block of the file, whereas this gives it an upper bound... why is that ok? It's been a long time since I looked at this code, but some explanation in the commit and in code comments would be helpful. If end_blk can be any value less than u32, what is its purpose? -Eric -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Eric Sandeen 2012-05-16T14:04:52 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32386 <pre> This is only in the ! EXT4_FEATURE_RO_COMPAT_GDT_CSUM case I guess? That would be worth mentioning in the summary &amp; changelog. I guess you were testing without that for some reason? -eric -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Eric Sandeen 2012-05-16T13:43:28 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32384 <pre> Its not nice to duplicate a call site for the parameter difference it's better to change the parameter and call the function in one place. (an assembler call site can get big) You can do: +submit_bh(write_op | (buffer_meta(bh) &lt;&lt; __REQ_META), bh); And also avoid a conditional inside a loop. Here too Boaz -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Boaz Harrosh 2012-05-16T12:32:39 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32382 <pre>From: Tao Ma &lt;boyu.mt&lt; at &gt;taobao.com&gt; Now we can use fallocate to create a large file while keep the size to be small. It will cause the e2fsck complain about it. The test script is simple and I have pasted it here. DEVICE=/dev/sdb1 mount -t ext4 $DEVICE /mnt/ext4 for((i=0;i&lt;10;i++))do fallocate -n -o $[$i*8192] -l 4096 /mnt/ext4/a;done umount $DEVICE e2fsck -fn $DEVICE The error message will be like this: e2fsck 1.42.3 (14-May-2012) Pass 1: Checking inodes, blocks, and sizes Inode 12 has zero length extent (invalid logical block 0, physical block 32775) Clear? no Inode 12, i_blocks is 88, should be 0. Fix? no Pass 2: Checking directory structure Pass 3: Checking directory connectivity Pass 4: Checking reference counts Pass 5: Checking group summary information Block bitmap differences: -(8231--8232) -(32770--32778) Fix? no Now actually the end_blk can be any value which is less than u32, so make end_blk be the maximum value of u32. Cc: Theodore Ts'o &lt;tytso&lt; at &gt;mit.edu&gt; Signed-off-by: Tao Ma &lt;boyu.mt&lt; at &gt;taoba</pre> Tao Ma 2012-05-16T08:50:25 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32381 <pre>From: Tao Ma &lt;boyu.mt&lt; at &gt;taobao.com&gt; Now when we set the group inode free count, we don't have a proper group lock so that multiple threads may decrease the inode free count at the same time. And e2fsck will complain something like: Free inodes count wrong for group #1 (1, counted=0). Fix? no Free inodes count wrong for group #2 (3, counted=0). Fix? no Directories count wrong for group #2 (780, counted=779). Fix? no Free inodes count wrong for group #3 (2272, counted=2273). Fix? no So this patch try to protect it with the ext4_lock_group. btw, it is found by xfstests test case 269 and I have run it 100 times and the error in e2fsck doesn't show up again. Cc: Theodore Ts'o &lt;tytso&lt; at &gt;mit.edu&gt; Signed-off-by: Tao Ma &lt;boyu.mt&lt; at &gt;taobao.com&gt; --- fs/ext4/ialloc.c | 9 +++++---- 1 files changed, 5 insertions(+), 4 deletions(-) diff --git a/fs/ext4/ialloc.c b/fs/ext4/ialloc.c index 409c2ee..25595e2 100644 --- a/fs/ext4/ialloc.c +++ b/fs/ext4/ialloc.c &lt; at &gt;&lt; at &gt; -772,7 +772,9 &lt; at &gt;&lt; at &gt; got: ext4_itable_unused_set(sb, gdp, </pre> Tao Ma 2012-05-16T08:49:52 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32379 <pre> Yes, we've been talking about this issue on LSF with Ted and the conclusion is that we want to wait for the range locks to be ready. This way we can avoid taking imutex for the punch hole when punching beyond isize which we would have to do otherwise. I am not sure how big of an issue this is, probably not so big. If we can not wait for the range locks, I can make a patch with imutex protection. Thanks! -Lukas -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Lukáš Czerner 2012-05-16T06:14:56 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32377 <pre> Even if there's no runtime change, it's also useful to measure the CPU utilization. If there's an increase in CPU utilization, then it can show up in workloads and benchmarks which are sensitive to CPU utilization as well as disk utilization, e.g., TPC-C/H. But since it takes so long for performance teams to notice, they tend to get very cranky when they observe regressions. So for changes like this it's really important to measure any changes in CPU utilization, especially on larger on SMP systems when there multiple processes writing to the same file at high rates --- you know, like what an Enterprise database might do to a table space file. :-) - Ted -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Ted Ts'o 2012-05-16T01:36:42 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32375 <pre> Thanks, Allison. I just added Jan to the Cc list to make sure he sees, since we mentioned this in the inode_dio_wait thread (which I skilfully directed to an almost disjoint set of addressees - though I expect he already saw via linux-ext4). Hugh -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Hugh Dickins 2012-05-15T22:38:33 http://permalink.gmane.org/gmane.comp.file-systems.ext4/32373 <pre> Hi all, I had a fix for this a while ago and I believe Lukas had rebased it when he was working on some punch hole optimizations, but Im not sure what happened to it after that. I think Lukas might still be working on that set? If not, I can take a peek at it again and see if I can get it updated and resent. Thx! Allison Henderson -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo&lt; at &gt;vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html </pre> Allison Henderson 2012-05-15T21:36:57 Search the mailing list at Gmane query http://search.gmane.org/?group=$group=gmane.comp.file-systems.ext4