gmane.comp.file-systems.ext4

Re: [PATCH]ext4: fix s_dirty_blocks_counter if block allocationfailed with nodelalloc

Akira Fujita — 2008-12-04T01:27:25

Hi Aneesh, Aneesh Kumar K.V wrote: My case was *nodelalloc* and FS was almost full. This problem occurs in multiple defrag running in short time. Usually defrag releases temporary inode's blocks with iput, then FS free blocks are recover but contiguous blocks do not recover until next journal commit. so we can not re-use contiguous blocks immediately. There are enough free blocks in FS so that ext4_claim_free_blocks marks claimed blocks as dirty, but ext4_regular_allocator can not find enough blocks, so mb_new_blocks returns ENOSPC without decreasing dirty blocks. Regards, Akira Fujita -- To unsubscribe from this list: send the line "unsubscribe linux-ext4" in the body of a message to majordomo< at >vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html

Re: Problems with checking corrupted large ext3 file system

Andreas Dilger — 2008-12-04T00:09:36

A big question is what kernel you are running on. Anything less than 2.6.18-rhel5 (not sure what vanilla kernel) has bugs with ext3 > 8TB. The other question is whether there is any expectation that the data moved from the bad RAID arrays was corrupted. Running "e2fsck -y" vs. "e2fsck -p" will sometimes do "bad" things because the "-y" forces it to continue on no matter what. It looks like there was some serious filesystem corruption beyond the 8TB boundary, and the inode table for at one or more groups (depending on how many of the "SEVERE DATA LOSS POSSIBLE" messages were printed) is completely lost. This is likely fallout from the original corruption above. The bad news is that these "multiply-claimed blocks" are really bogus because of the garbage in the missing inode tables... e2fsck has turned random garbage into inodes, and it results in what you are seeing now. The pass1b (clone multiply-claimed blocks) code is very slow, because it involves an O(n^2) operation to find all of the duplicat

[Bug 12151] New: Unexplained fsck errors on a ext4 filesystem

2008-12-03T21:38:32

http://bugzilla.kernel.org/show_bug.cgi?id=12151 Summary: Unexplained fsck errors on a ext4 filesystem Product: File System Version: 2.5 KernelVersion: kernel-2.6.27.5-117.fc10.x86_64 Platform: All OS/Version: Linux Tree: Mainline Status: NEW Severity: normal Priority: P1 Component: ext4 AssignedTo: fs_ext4< at >kernel-bugs.osdl.org ReportedBy: kernel-bugzilla< at >cygnusx-1.org Distribution: Fedora 10 Hardware Environment: Processor: Q6600 2.4ghz Memory: 4gb dmidecode: Manufacturer: ASUSTeK Computer INC. Product Name: P5B-Deluxe lspci: 00:00.0 Host bridge: Intel Corporation 82P965/G965 Memory Controller Hub (rev 02) 00:01.0 PCI bridge: Intel Corporation 82P965/G965 PCI Express Root Port (rev 02) 00:1a.0 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #4 (rev 02) 00:1a.1 USB Controller: Intel Corporation 82801H (ICH8 Family) USB UHCI Controller #5 (rev 02) 00

Re: [PATCH V3 3/3]ext4: quota handling for delayed allocation

Jan Kara — 2008-12-03T20:11:11

Hi, On Thu 06-11-08 15:39:05, Mingming Cao wrote: claim_quota seems to be unused here and I'm not sure we need it for anything... Honza

[PATCH 2/6] ext3: don't inherit inappropriate inode flags from parent

Duane Griffin — 2008-12-03T19:54:59

At present INDEX is the only flag that new ext3 inodes do NOT inherit from their parent. In addition prevent the flags DIRTY, ECOMPR, IMAGIC and TOPDIR from being inherited. List inheritable flags explicitly to prevent future flags from accidentally being inherited. This fixes the TOPDIR flag inheritance bug reported at http://bugzilla.kernel.org/show_bug.cgi?id=9866. Signed-off-by: Duane Griffin dghda.com> --- fs/ext3/ialloc.c | 2 +- include/linux/ext3_fs.h | 7 +++++++ 2 files changed, 8 insertions(+), 1 deletions(-) diff --git a/fs/ext3/ialloc.c b/fs/ext3/ialloc.c index 490bd0e..33a6bdf 100644 --- a/fs/ext3/ialloc.c +++ b/fs/ext3/ialloc.c < at >< at > -559,7 +559,7 < at >< at > got: ei->i_dir_start_lookup = 0; ei->i_disksize = 0; -ei->i_flags = EXT3_I(dir)->i_flags & ~EXT3_INDEX_FL; +ei->i_flags = EXT3_I(dir)->i_flags & EXT3_FL_INHERITED; if (S_ISLNK(mode)) ei->i_flags &= ~(EXT3_IMMUTABLE_FL|EXT3_APPEND_FL); /* dirsync only applies to directories */ diff --git a/include/linux/ext3_fs.

[PATCH 5/6] ext3: tighten restrictions on inode flags

Duane Griffin — 2008-12-03T19:55:02

At the moment there are few restrictions on which flags may be set on which inodes. Specifically DIRSYNC may only be set on directories and IMMUTABLE and APPEND may not be set on links. Tighten that to disallow TOPDIR being set on non-directories and only NODUMP and NOATIME to be set on non-regular file, non-directories. Introduces a flags masking function which masks flags based on mode and use it during inode creation and when flags are set via the ioctl to facilitate future consistency. Signed-off-by: Duane Griffin dghda.com> --- fs/ext3/ialloc.c | 8 ++------ fs/ext3/ioctl.c | 3 +-- include/linux/ext3_fs.h | 17 +++++++++++++++++ 3 files changed, 20 insertions(+), 8 deletions(-) diff --git a/fs/ext3/ialloc.c b/fs/ext3/ialloc.c index 33a6bdf..48699ce 100644 --- a/fs/ext3/ialloc.c +++ b/fs/ext3/ialloc.c < at >< at > -559,12 +559,8 < at >< at > got: ei->i_dir_start_lookup = 0; ei->i_disksize = 0; -ei->i_flags = EXT3_I(dir)->i_flags & EXT3_FL_INHERITED; -if (S_ISLNK(mode)) -ei->i_fl

[PATCH 3/6] ext4: don't inherit inappropriate inode flags from parent

Duane Griffin — 2008-12-03T19:55:00

At present INDEX and EXTENTS are the only flags that new ext4 inodes do NOT inherit from their parent. In addition prevent the flags DIRTY, ECOMPR, IMAGIC, TOPDIR, HUGE_FILE and EXT_MIGRATE from being inherited. List inheritable flags explicitly to prevent future flags from accidentally being inherited. This fixes the TOPDIR flag inheritance bug reported at http://bugzilla.kernel.org/show_bug.cgi?id=9866. Signed-off-by: Duane Griffin dghda.com> --- fs/ext4/ext4.h | 8 ++++++++ fs/ext4/ialloc.c | 2 +- 2 files changed, 9 insertions(+), 1 deletions(-) diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index 8370ffd..63e5163 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h < at >< at > -247,6 +247,14 < at >< at > struct flex_groups { #define EXT4_FL_USER_VISIBLE0x000BDFFF /* User visible flags */ #define EXT4_FL_USER_MODIFIABLE0x000B80FF /* User modifiable flags */ +/* Flags that should be inherited by new inodes from their parent. */ +#define EXT4_FL_INHERITED (EXT4_SECRM_FL | EXT4_UNRM_FL | EXT4_COMPR_FL |\

[PATCH 6/6] ext4: tighten restrictions on inode flags

Duane Griffin — 2008-12-03T19:55:03

At the moment there are few restrictions on which flags may be set on which inodes. Specifically DIRSYNC may only be set on directories and IMMUTABLE and APPEND may not be set on links. Tighten that to disallow TOPDIR being set on non-directories and only NODUMP and NOATIME to be set on non-regular file, non-directories. Introduces a flags masking function which masks flags based on mode and use it during inode creation and when flags are set via the ioctl to facilitate future consistency. Signed-off-by: Duane Griffin dghda.com> --- fs/ext4/ext4.h | 17 +++++++++++++++++ fs/ext4/ialloc.c | 14 +++++--------- fs/ext4/ioctl.c | 3 +-- 3 files changed, 23 insertions(+), 11 deletions(-) diff --git a/fs/ext4/ext4.h b/fs/ext4/ext4.h index 63e5163..7b5c86d 100644 --- a/fs/ext4/ext4.h +++ b/fs/ext4/ext4.h < at >< at > -255,6 +255,23 < at >< at > struct flex_groups { EXT4_JOURNAL_DATA_FL | EXT4_NOTAIL_FL|\ EXT4_DIRSYNC_FL) +/* Flags that are appropriate for regular files (all but dir-specific ones).

[PATCH 4/6] ext2: tighten restrictions on inode flags

Duane Griffin — 2008-12-03T19:55:01

At the moment there are few restrictions on which flags may be set on which inodes. Specifically DIRSYNC may only be set on directories and IMMUTABLE and APPEND may not be set on links. Tighten that to disallow TOPDIR being set on non-directories and only NODUMP and NOATIME to be set on non-regular file, non-directories. Introduces a flags masking function which masks flags based on mode and use it during inode creation and when flags are set via the ioctl to facilitate future consistency. Signed-off-by: Duane Griffin dghda.com> --- fs/ext2/ialloc.c | 8 ++------ fs/ext2/ioctl.c | 3 +-- include/linux/ext2_fs.h | 17 +++++++++++++++++ 3 files changed, 20 insertions(+), 8 deletions(-) diff --git a/fs/ext2/ialloc.c b/fs/ext2/ialloc.c index 8c1897e..bee9709 100644 --- a/fs/ext2/ialloc.c +++ b/fs/ext2/ialloc.c < at >< at > -565,12 +565,8 < at >< at > got: inode->i_blocks = 0; inode->i_mtime = inode->i_atime = inode->i_ctime = CURRENT_TIME_SEC; memset(ei->i_data, 0, sizeof(ei->i_data)); -ei-

[PATCH 0/6][REPOST] ext{2,3,4}: tighten inheritance and setting of inode flags

Duane Griffin — 2008-12-03T19:54:57

This patch series prevents the inheritance and setting of flags that are inappropriate for specific inode types. Flags which should be inherited are listed explicitly so as to prevent future flags being overlooked and inherited by accident. It introduces a function to mask flags based on the inode type and uses it in inode creation and the SETFLAGS ioctl to help prevent future inconsistency. Patches 1-3 fix the TOPDIR flag inheritance bug reported at http://bugzilla.kernel.org/show_bug.cgi?id=9866. Patches 4-6 fix a related problem with non-regular file/dir inodes inheriting inappropriate flags, discovered while testing. For example, on an unpatched system, the following sequence will create an un(re)movable device node: mkdir a chattr +a a touch a/a mknod a/b c 1 3 chattr -a a a/a All attempts to delete, move or modify a/b will fail. Fsck will report there is a problem but will not fix it. Diffstat from linux-next: fs/ext2/ialloc.c | 8 ++------ fs/ext2/ioctl.c | 3 +-- fs/ext3/

[PATCH 1/6] ext2: don't inherit inappropriate inode flags from parent

Duane Griffin — 2008-12-03T19:54:58

At present BTREE/INDEX is the only flag that new ext2 inodes do NOT inherit from their parent. In addition prevent the flags DIRTY, ECOMPR, INDEX, IMAGIC and TOPDIR from being inherited. List inheritable flags explicitly to prevent future flags from accidentally being inherited. This fixes the TOPDIR flag inheritance bug reported at http://bugzilla.kernel.org/show_bug.cgi?id=9866. Signed-off-by: Duane Griffin dghda.com> --- fs/ext2/ialloc.c | 2 +- include/linux/ext2_fs.h | 7 +++++++ 2 files changed, 8 insertions(+), 1 deletions(-) diff --git a/fs/ext2/ialloc.c b/fs/ext2/ialloc.c index 8d0add6..8c1897e 100644 --- a/fs/ext2/ialloc.c +++ b/fs/ext2/ialloc.c < at >< at > -565,7 +565,7 < at >< at > got: inode->i_blocks = 0; inode->i_mtime = inode->i_atime = inode->i_ctime = CURRENT_TIME_SEC; memset(ei->i_data, 0, sizeof(ei->i_data)); -ei->i_flags = EXT2_I(dir)->i_flags & ~EXT2_BTREE_FL; +ei->i_flags = EXT2_I(dir)->i_flags & EXT2_FL_INHERITED; if (S_ISLNK(mode)) ei->i_flags &= ~(EXT2_IMMUTABLE_F

[patch 098/104] ext4: add checksum calculation when clearingUNINIT flag in ext4_new_inode

Greg KH — 2008-12-03T19:56:46

2.6.27-stable review patch. If anyone has any objections, please let us know. ------------------ From: Frederic Bohe bull.net> (cherry picked from commit 23712a9c28b9f80a8cf70c8490358d5f562d2465) When initializing an uninitialized block group in ext4_new_inode(), its block group checksum must be re-calculated. This fixes a race when several threads try to allocate a new inode in an UNINIT'd group. There is some question whether we need to be initializing the block bitmap in ext4_new_inode() at all, but for now, if we are going to init the block group, let's eliminate the race. Signed-off-by: Frederic Bohe bull.net> Signed-off-by: "Theodore Ts'o" mit.edu> Signed-off-by: Greg Kroah-Hartman suse.de> --- fs/ext4/ialloc.c | 2 ++ 1 file changed, 2 insertions(+) --- a/fs/ext4/ialloc.c +++ b/fs/ext4/ialloc.c < at >< at > -717,6 +717,8 < at >< at > got: gdp->bg_flags &= cpu_to_le16(~EXT4_BG_BLOCK_UNINIT); free = ext4_free_blocks_after_init(sb, group, gdp); gdp->bg_fr

[patch 092/104] ext4: Fix duplicate entries returned fromgetdents() system call

Greg KH — 2008-12-03T19:56:34

2.6.27-stable review patch. If anyone has any objections, please let us know. ------------------ From: "Theodore Ts'o" mit.edu> (cherry picked from commit 3c37fc86d20fe35be656f070997d62f75c2e4874) Fix a regression caused by commit d0156417, "ext4: fix ext4_dx_readdir hash collision handling", where deleting files in a large directory (requiring more than one getdents system call), results in some filenames being returned twice. This was caused by a failure to update info->curr_hash and info->curr_minor_hash, so that if the directory had gotten modified since the last getdents() system call (as would be the case if the user is running "rm -r" or "git clean"), a directory entry would get returned twice to the userspace. Signed-off-by: "Theodore Ts'o" mit.edu> This patch fixes the bug reported by Markus Trippelsdorf at: http://bugzilla.kernel.org/show_bug.cgi?id=11844 Signed-off-by: "Theodore Ts'o" mit.edu> Tested-by: Markus Trippelsdorf trippelsdorf.de> Signed-off-by: Greg

[patch 095/104] ext4: wait on all pending commits in ext4_sync_fs()

Greg KH — 2008-12-03T19:56:40

2.6.27-stable review patch. If anyone has any objections, please let us know. ------------------ From: "Theodore Ts'o" mit.edu> (cherry picked from commit 14ce0cb411c88681ab8f3a4c9caa7f42e97a3184) In ext4_sync_fs, we only wait for a commit to finish if we started it, but there may be one already in progress which will not be synced. In the case of a data=ordered umount with pending long symlinks which are delayed due to a long list of other I/O on the backing block device, this causes the buffer associated with the long symlinks to not be moved to the inode dirty list in the second phase of fsync_super. Then, before they can be dirtied again, kjournald exits, seeing the UMOUNT flag and the dirty pages are never written to the backing block device, causing long symlink corruption and exposing new or previously freed block data to userspace. To ensure all commits are synced, we flush all journal commits now when sync_fs'ing ext4. Signed-off-by: Arthur Jones riverbed.com> Signed-off-by: A

[patch 094/104] ext4: Convert to host order before using thevalues.

Greg KH — 2008-12-03T19:56:38

2.6.27-stable review patch. If anyone has any objections, please let us know. ------------------ From: Aneesh Kumar K.V linux.vnet.ibm.com> (cherry picked from commit d94e99a64c3beece22dbfb2b335771a59184eb0a) Use le16_to_cpu to read the s_reserved_gdt_blocks values from super block. Signed-off-by: Aneesh Kumar K.V linux.vnet.ibm.com> Signed-off-by: "Theodore Ts'o" mit.edu> Signed-off-by: Greg Kroah-Hartman suse.de> --- fs/ext4/super.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) --- a/fs/ext4/super.c +++ b/fs/ext4/super.c < at >< at > -1506,9 +1506,8 < at >< at > static int ext4_fill_flex_info(struct su /* We allocate both existing and potentially added groups */ flex_group_count = ((sbi->s_groups_count + groups_per_flex - 1) + - ((sbi->s_es->s_reserved_gdt_blocks +1 ) << - EXT4_DESC_PER_BLOCK_BITS(sb))) / - groups_per_flex; +((le16_to_cpu(sbi->s_es->s_reserved_gdt_blocks) + 1) << + EXT4_DESC_PER_BLOCK_BITS(sb))) / groups_per

[patch 097/104] ext4: Mark the buffer_heads as dirty and uptodateafter prepare_write

Greg KH — 2008-12-03T19:56:44

2.6.27-stable review patch. If anyone has any objections, please let us know. ------------------ From: Aneesh Kumar K.V linux.vnet.ibm.com> (cherry picked from commit ed9b3e3379731e9f9d2f73f3d7fd9e7d2ce3df4a) We need to make sure we mark the buffer_heads as dirty and uptodate so that block_write_full_page write them correctly. This fixes mmap corruptions that can occur in low memory situations. Signed-off-by: Aneesh Kumar K.V linux.vnet.ibm.com> Signed-off-by: "Theodore Ts'o" mit.edu> Signed-off-by: Greg Kroah-Hartman suse.de> --- fs/ext4/inode.c | 2 ++ 1 file changed, 2 insertions(+) --- a/fs/ext4/inode.c +++ b/fs/ext4/inode.c < at >< at > -2242,6 +2242,8 < at >< at > static int ext4_da_writepage(struct page unlock_page(page); return 0; } +/* now mark the buffer_heads as dirty and uptodate */ +block_commit_write(page, 0, PAGE_CACHE_SIZE); } if (test_opt(inode->i_sb, NOBH) && ext4_should_writeback_data(inode)) -- To unsubscribe from this list: send th

[patch 088/104] jbd2: Fix buffer head leak when writing the commitblock

Greg KH — 2008-12-03T19:56:22

2.6.27-stable review patch. If anyone has any objections, please let us know. ------------------ From: "Theodore Ts'o" mit.edu> (cherry picked from commit 45a90bfd90c1215bf824c0f705b409723f52361b) Also make sure the buffer heads are marked clean before submitting bh for writing. The previous code was marking the buffer head dirty, which would have forced an unneeded write (and seek) to the journal for no good reason. Signed-off-by: "Theodore Ts'o" mit.edu> Signed-off-by: Greg Kroah-Hartman suse.de> --- fs/jbd2/commit.c | 5 ++--- 1 file changed, 2 insertions(+), 3 deletions(-) --- a/fs/jbd2/commit.c +++ b/fs/jbd2/commit.c < at >< at > -126,8 +126,7 < at >< at > static int journal_submit_commit_record( JBUFFER_TRACE(descriptor, "submit commit block"); lock_buffer(bh); -get_bh(bh); -set_buffer_dirty(bh); +clear_buffer_dirty(bh); set_buffer_uptodate(bh); bh->b_end_io = journal_end_buffer_io_sync; < at >< at > -160,7 +159,7 < at >< at > static int journal_submit_commit_record( /* And try again, witho

[patch 087/104] jbd2: abort instead of waiting for nonexistenttransaction

Greg KH — 2008-12-03T19:56:20

2.6.27-stable review patch. If anyone has any objections, please let us know. ------------------ From: Duane Griffin dghda.com> (cherry picked from commit 23f8b79eae8a74e42a006ffa7c456e295c7e1c0d) The __jbd2_log_wait_for_space function sits in a loop checkpointing transactions until there is sufficient space free in the journal. However, if there are no transactions to be processed (e.g. because the free space calculation is wrong due to a corrupted filesystem) it will never progress. Check for space being required when no transactions are outstanding and abort the journal instead of endlessly looping. This patch fixes the bug reported by Sami Liedes at: http://bugzilla.kernel.org/show_bug.cgi?id=10976 Signed-off-by: Duane Griffin dghda.com> Cc: Sami Liedes cc.hut.fi> Cc: vger.kernel.org> Signed-off-by: Andrew Morton linux-foundation.org> Signed-off-by: "Theodore Ts'o" mit.edu> Signed-off-by: Greg Kroah-Hartman suse.de> --- fs/jbd2/checkpoi

[patch 086/104] ext4: fix initialization of UNINIT bitmap blocks

Greg KH — 2008-12-03T19:56:18

2.6.27-stable review patch. If anyone has any objections, please let us know. ------------------ From: Frederic Bohe bull.net> (cherry picked from commit c806e68f5647109350ec546fee5b526962970fd2) This fixes a bug which caused on-line resizing of filesystems with a 1k blocksize to fail. The root cause of this bug was the fact that if an uninitalized bitmap block gets read in by userspace (which e2fsprogs does try to avoid, but can happen when the blocksize is less than the pagesize and an adjacent blocks is read into memory) ext4_read_block_bitmap() was erroneously depending on the buffer uptodate flag to decide whether it needed to initialize the bitmap block in memory --- i.e., to set the standard set of blocks in use by a block group (superblock, bitmaps, inode table, etc.). Essentially, ext4_read_block_bitmap() assumed it was the only routine that might try to read a block containing a block bitmap, which is simply not true. To fix this, ext4_read_block_bitmap() and ext4_read_inode_b

[patch 101/104] ext3: dont try to resize if there are no reservedgdt blocks left

Greg KH — 2008-12-03T19:56:53

2.6.27-stable review patch. If anyone has any objections, please let us know. ------------------ From: Josef Bacik redhat.com> commit 972fbf779832e5ad15effa7712789aeff9224c37 upstream. When trying to resize a ext3 fs and you run out of reserved gdt blocks, you get an error that doesn't actually tell you what went wrong, it just says that the gdb it picked is not correct, which is the case since you don't have any reserved gdt blocks left. This patch adds a check to make sure you have reserved gdt blocks to use, and if not prints out a more relevant error. Signed-off-by: Josef Bacik redhat.com> Cc: vger.kernel.org> Cc: Andreas Dilger sun.com> Signed-off-by: Andrew Morton linux-foundation.org> Signed-off-by: Linus Torvalds linux-foundation.org> Cc: Willy Tarreau 1wt.eu> Signed-off-by: Greg Kroah-Hartman suse.de> --- fs/ext3/resize.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) --- a/fs/ext3/resize.c +++ b/fs/ext3/resize.c < at >< at >

[patch 089/104] ext4: fix xattr deadlock

Greg KH — 2008-12-03T19:56:26

2.6.27-stable review patch. If anyone has any objections, please let us know. ------------------ From: Kalpak Shah sun.com> (cherry picked from commit 4d20c685fa365766a8f13584b4c8178a15ab7103) ext4_xattr_set_handle() eventually ends up calling ext4_mark_inode_dirty() which tries to expand the inode by shifting the EAs. This leads to the xattr_sem being downed again and leading to a deadlock. This patch makes sure that if ext4_xattr_set_handle() is in the call-chain, ext4_mark_inode_dirty() will not expand the inode. Signed-off-by: Kalpak Shah sun.com> Signed-off-by: "Theodore Ts'o" mit.edu> Signed-off-by: Greg Kroah-Hartman suse.de> --- fs/ext4/xattr.c | 6 ++++++ 1 file changed, 6 insertions(+) --- a/fs/ext4/xattr.c +++ b/fs/ext4/xattr.c < at >< at > -959,6 +959,7 < at >< at > ext4_xattr_set_handle(handle_t *handle, struct ext4_xattr_block_find bs = { .s = { .not_found = -ENODATA, }, }; +unsigned long no_expand; int error; if (!name) < at >< at > -966,6 +967,9 < at >< at > ext4