| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <AANLkTimJVDB4hqCeD2dPBAD8krJScgiBP8-3sBmRaFB1@mail.gmail.com>
Date: Fri, 25 Jun 2010 18:33:28 +0800
From: Hsuan-Ting <acht93@...ccu.edu.tw>
To: tytso@....edu,
"linux-ext4@...r.kernel.org development" <linux-ext4@...r.kernel.org>
Subject: Re: E2fsprogs master branch now has all 64-bit patch applied
Hi Ted,
I try to :
1. force to enable 64bits feature even the size <16TB
2. fix the "block_bmap" overflow issue when resizing.
It seems OK when resize to >16TB .
(the content of the test file is correct).
But after it grown up, it will get error when do fsck.
My modification is as the following:
lib/ext2fs/openfs.c
@@ -109,6 +109,8 @@ errcode_t ext2fs_open2(const char *name, const
char *io_options,
memset(fs, 0, sizeof(struct struct_ext2_filsys));
fs->magic = EXT2_ET_MAGIC_EXT2FS_FILSYS;
fs->flags = flags;
+ fs->flags |= EXT2_FLAG_64BITS;
/* don't overwrite sb backups unless flag is explicitly cleared */
fs->flags |= EXT2_FLAG_MASTER_SB_ONLY;
fs->umask = 022;
misc/mke2fs.c
@@ -1530,6 +1531,8 @@ static void PRS(int argc, char *argv[])
EXT2_BLOCK_SIZE(&fs_param));
exit(1);
}
+ fs_param.s_feature_incompat |= EXT4_FEATURE_INCOMPAT_64BIT;
ext2fs_blocks_count_set(&fs_param, fs_blocks_count);
resize/resize2fs.c
index 064c4c4..e28f5f2 100644
--- a/resize/resize2fs.c
+++ b/resize/resize2fs.c
@@ -294,7 +294,8 @@ errcode_t adjust_fs_info(ext2_filsys fs, ext2_filsys old_fs,
blk64_t overhead = 0;
blk64_t rem;
blk64_t blk, group_block;
- ext2_ino_t real_end;
+ __u64 real_end;
blk64_t adj, old_numblocks, numblocks, adjblocks;
unsigned long i, j, old_desc_blocks, max_group;
unsigned int meta_bg, meta_bg_size;
@@ -381,9 +382,9 @@ retry:
fs->inode_map);
if (retval) goto errout;
- real_end = ((EXT2_BLOCKS_PER_GROUP(fs->super)
- * fs->group_desc_count)) - 1 +
- fs->super->s_first_data_block;
+ real_end = ((__u64)(EXT2_BLOCKS_PER_GROUP(fs->super)
+ * (__u64)fs->group_desc_count)) - 1ULL +
+ (__u64)fs->super->s_first_data_block;
retval = ext2fs_resize_block_bitmap2(ext2fs_blocks_count(fs->super)-1,
real_end, fs->block_map);
@@ -585,6 +586,8 @@ static errcode_t adjust_superblock(ext2_resize_t
rfs, blk64_t new_size)
if (retval)
return retval;
+ fs->super->s_feature_incompat |= EXT4_FEATURE_INCOMPAT_64BIT;
retval = adjust_fs_info(fs, rfs->old_fs, rfs->reserve_blocks, new_size);
if (retval)
goto errout;
My test case:
1. build a linear raid (1 x 2TB disk)
2. mkfs.ext4, mount it and"echo 123 > test" to
touch a test file.
3. grown the linear raid to >16TB (9 x 2TB + 1 x 1.5TB)
4. do resize ( resize -fpF /dev/md2 )
After resizing, the content of the test file is correct.
But "fsck -nyv" will get the following error:
e2fsck 1.41.12 (17-May-2010)
###open|= EXT2_FLAG_64BITS
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
Block bitmap differences: +(244154882--244187135) +(244187650--244188163)
Fix? no
/dev/md2: ********** WARNING: Filesystem still has errors **********
12 inodes used (0.00%)
0 non-contiguous files (0.0%)
0 non-contiguous directories (0.0%)
# of inodes with ind/dind/tind blocks: 0/0/0
Extent depth histogram: 2
74638645 blocks used (1.57%)
0 bad blocks
0 large files
1 regular file
2 directories
0 character device files
0 block device files
0 fifos
0 links
0 symbolic links (0 fast symbolic links)
0 sockets
--------
3 files
I think maybe I should modify "ext2_ino_t" type from
"__u32" to "__u64".
Maybe this modification will fix many overflow issue.
Do you have any idea of this fsck error or any opinions of this approach?
Thanks
-HsuanTing
2010/6/22 Hsuan-Ting <acht93@...ccu.edu.tw>:
> 2010/6/22 <tytso@....edu>
>>
>> On Mon, Jun 21, 2010 at 09:44:31PM +0800, Hsuan-Ting wrote:
>> > Hi Ted,
>> >
>> > Resize seems not work when the size is bigger than 16TB (offline resize).
>> >
>> > My test machine:
>> > x64 platform 2.6.32 kernel + this newest patch
>> >
>> > 1. <16TB ext4 enlarge to >16TB (offline)
>> > a. I use "8 x 2TB WD disks" and "mdadm" build linear raid
>> > b. then use mkfs.ext4 to make ext4 file system
>> > c. grow the linear raid to "10 X 2TB"
>> > d. finally it grow to "2.X TB" smaller than before
>>
>> This doesn't surprise me. We should add some checks to simply not
>> allow the file system growing greater than 16TB if the 64-bit feature
>> is not set for now. Making this work is going to be tricky, because
>> enabling the 64-bit feature doubles the size of the block group
>> descriptors, which means we have to make room for them. This could
>> involve moving files out of the way, as well as moving the inode
>> table.
>>
>> This means that we may want to enable the 64-bit feature flag if there
>> is an expectation that the filesystem might be grown to a size large
>> enough where this would be an issue.
>
> Sounds like I must enable 64-bit feature when mkfs.
> Then it will work, right?
>
> But base on my test, it will occur core dump when resize:
> (gdb) bt
> #0 0x00000000004160bf in ext2fs_test_bit64 ()
> #1 0x0000000000416318 in ba_test_bmap ()
> #2 0x0000000000410629 in ext2fs_test_generic_bmap ()
> #3 0x0000000000410656 in ext2fs_test_block_bitmap_range2 ()
> #4 0x000000000040873d in ext2fs_get_free_blocks2 ()
> #5 0x000000000040936d in ext2fs_allocate_group_table ()
> #6 0x0000000000404456 in adjust_fs_info ()
> #7 0x0000000000404a81 in resize_fs ()
> #8 0x00000000004069c7 in main ()
>
> I do the following modification
> (to enable "EXT4_FEATURE_INCOMPAT_64BIT" and "EXT2_FLAG_64BITS"):
>
> misc/mke2fs.c :
> @@ -1530,6 +1945,8 @@ static void PRS(int argc, char *argv[])
> EXT2_BLOCK_SIZE(&fs_param));
> exit(1);
> }
> + fs_param.s_feature_incompat |= EXT4_FEATURE_INCOMPAT_64BIT;
>
> ext2fs_blocks_count_set(&fs_param, fs_blocks_count);
>
>
> resize/resize2fs.c :
> @@ -585,6 +598,9 @@ static errcode_t adjust_superblock(ext2_resize_t
> rfs, blk64_t new_size)
> if (retval)
> return retval;
>
> + fs->super->s_feature_incompat |= EXT4_FEATURE_INCOMPAT_64BIT;
> retval = adjust_fs_info(fs, rfs->old_fs, rfs->reserve_blocks, new_size);
>
>
> lib/ext2fs/openfs.c :
> @@ -109,6 +109,8 @@ errcode_t ext2fs_open2(const char *name, const
> char *io_options,
> memset(fs, 0, sizeof(struct struct_ext2_filsys));
> fs->magic = EXT2_ET_MAGIC_EXT2FS_FILSYS;
> fs->flags = flags;
> + fs->flags |= EXT2_FLAG_64BITS;
>
>
> Did I mistake something?
>
>>
>> > 2. >16TB offline resize, the steps is similiar as before.
>> > a. I use "9 x 2TB WD disks" build linear raid
>> > b. mkfs.ext4 and not mount
>> > c. grow the linear raid to "10 X 2TB"
>> > d. do resize
>> > e. finally it grow to "2.X TB" smaller than before
>> >
>> > I try to on "EXT4_FEATURE_INCOMPAT_64BIT" and "EXT2_FLAG_64BITS"
>> > when mkfs and resize.
>> > And modify ext2fs_div_ceil code to ext2fs_div64_ceil.
>> > It seems work something, the fs size isn't grow but also not deduce,
>> > remain the same.
>>
>> I'm not sure I understand that last sentence; it's not parsing as an
>> understandable English sentence, sorry. Are you saying that both
>> attempts to grow and shrink the filesystem is failing? If so, how?
>> Are you getting an error message? Is it appearing to succeed but the
>> file system size isn't changing?
>
> Sorry for my poor English. The last sentence means "succeed but the
> file system size isn't changing".
> I also remove "flex_bg" feature in this case.
>
> Thanks.
>
> -HsuanTing
>
>>
>> Thanks for the bug report,
>>
>> - Ted
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
>> the body of a message to majordomo@...r.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists