lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening PHC | |
Open Source and information security mailing list archives
| ||
|
Date: Thu, 6 May 2021 16:26:24 +0800 From: yebin <yebin10@...wei.com> To: Jan Kara <jack@...e.cz> CC: <tytso@....edu>, <adilger.kernel@...ger.ca>, <linux-ext4@...r.kernel.org>, <linux-kernel@...r.kernel.org> Subject: Re: [PATCH v3] ext4: Fix bug on in ext4_es_cache_extent as ext4_split_extent_at failed On 2021/5/5 18:41, Jan Kara wrote: > On Wed 05-05-21 11:29:57, yebin wrote: >> >> On 2021/4/30 20:58, Jan Kara wrote: >>> On Wed 28-04-21 16:51:58, Ye Bin wrote: >>>> We got follow bug_on when run fsstress with injecting IO fault: >>>> [130747.323114] kernel BUG at fs/ext4/extents_status.c:762! >>>> [130747.323117] Internal error: Oops - BUG: 0 [#1] SMP >>>> ...... >>>> [130747.334329] Call trace: >>>> [130747.334553] ext4_es_cache_extent+0x150/0x168 [ext4] >>>> [130747.334975] ext4_cache_extents+0x64/0xe8 [ext4] >>>> [130747.335368] ext4_find_extent+0x300/0x330 [ext4] >>>> [130747.335759] ext4_ext_map_blocks+0x74/0x1178 [ext4] >>>> [130747.336179] ext4_map_blocks+0x2f4/0x5f0 [ext4] >>>> [130747.336567] ext4_mpage_readpages+0x4a8/0x7a8 [ext4] >>>> [130747.336995] ext4_readpage+0x54/0x100 [ext4] >>>> [130747.337359] generic_file_buffered_read+0x410/0xae8 >>>> [130747.337767] generic_file_read_iter+0x114/0x190 >>>> [130747.338152] ext4_file_read_iter+0x5c/0x140 [ext4] >>>> [130747.338556] __vfs_read+0x11c/0x188 >>>> [130747.338851] vfs_read+0x94/0x150 >>>> [130747.339110] ksys_read+0x74/0xf0 >>>> >>>> If call ext4_ext_insert_extent failed but new extent already inserted, we just >>>> update "ex->ee_len = orig_ex.ee_len", this will lead to extent overlap, then >>>> cause bug on when cache extent. >>> Thanks for the patch but I'm still not quite sure, how overlapping extents >>> in the extent tree can lead to triggering BUG_ON(lblk + len - 1 < lblk) in >>> ext4_es_cache_extent(). Can you ellaborate a bit more how this happens? >> Assume that there is extent [10, 100] (ee_block=10 ee_len=91), call >> ext4_split_extent_at split at 50, >> we get two extent [10, 49] and [50, 100], then call ext4_ext_insert_extent >> to insert new extent [50, 100], >> if insert extent successed, but call ext4_ext_dirty failed(return -EROFS) >> as JBD maybe abort as io error. >> Then fix old extent length with old value, so we get two extent [10, >> 100] (ee_block=10 ee_len=91) and >> [50, 100](ee_block=50 ee_len=51). >> If call ext4_cache_extent to cache above extents as follow: >> prev = 0 lblk = 10 len = 91 --> cache [10, 100] ---> prev = lblk + len = >> 101 >> prev = 101 lblk = 50 len = 51 --> prev != 0 && prev != lblk --> cache [prev >> = 101, lblk - prev = 50 - 101 = -51] >> Obvious if call ext4_es_cache_extent cache extent[101, -51] wil trigger >> "BUG_ON(end < lblk)" . > Thanks for great explanation! Now I understand. > >>>> If call ext4_ext_insert_extent failed don't update ex->ee_len with old value. >>>> Maybe there will lead to block leak, but it can be fixed by fsck later. >>>> >>>> After we fixed above issue with v2 patch, but we got the same issue. >>>> ext4_split_extent_at: >>>> { >>>> ...... >>>> err = ext4_ext_insert_extent(handle, inode, ppath, &newex, flags); >>>> if (err == -ENOSPC && (EXT4_EXT_MAY_ZEROOUT & split_flag)) { >>>> ...... >>>> ext4_ext_try_to_merge(handle, inode, path, ex); ->step(1) >>>> err = ext4_ext_dirty(handle, inode, path + path->p_depth); ->step(2) >>>> if (err) >>>> goto fix_extent_len; >>>> ...... >>>> } >>>> ...... >>>> fix_extent_len: >>>> ex->ee_len = orig_ex.ee_len; ->step(3) >>>> ...... >>>> } >>>> If step(1) have been merged, but step(2) dirty extent failed, then go to >>>> fix_extent_len label to fix ex->ee_len with orig_ex.ee_len. But "ex" may not be >>>> old one, will cause overwritten. Then will trigger the same issue as previous. >>>> If step(2) failed, just return error, don't fix ex->ee_len with old value. >>>> >>>> Signed-off-by: Ye Bin <yebin10@...wei.com> >>>> --- >>>> fs/ext4/extents.c | 13 +++++-------- >>>> 1 file changed, 5 insertions(+), 8 deletions(-) >>>> >>>> diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c >>>> index 77c84d6f1af6..d4aa24a09d8b 100644 >>>> --- a/fs/ext4/extents.c >>>> +++ b/fs/ext4/extents.c >>>> @@ -3238,15 +3238,12 @@ static int ext4_split_extent_at(handle_t *handle, >>>> ex->ee_len = cpu_to_le16(ee_len); >>>> ext4_ext_try_to_merge(handle, inode, path, ex); >>>> err = ext4_ext_dirty(handle, inode, path + path->p_depth); >>>> - if (err) >>>> - goto fix_extent_len; >>>> - >>>> - /* update extent status tree */ >>>> - err = ext4_zeroout_es(inode, &zero_ex); >>>> - >>>> - goto out; >>>> - } else if (err) >>>> + if (!err) >>>> + /* update extent status tree */ >>>> + err = ext4_zeroout_es(inode, &zero_ex); >>>> + } else if (err && err != -EROFS) { >>> I fail to see why EROFS is special here. Can you explain a bit please? >> V1 patch Ted suggest me to fix length only when "err != -EROSFS". As if >> we don't >> fix origin extent with old extent length, it will lead to block leak. >> Ted said as follow: >> >> If you don't want to do that, then a "do no harm" fix would be >> something like this: >> >> ... >> } else if (err == -EROFS) { >> return err; >> } else if (err) >> goto fix_extent_len; >> >> So in the journal abort case, when err is set to EROFS, we don't try >> to reset the length, since in theory the file system is read-only >> already anyway. However, in the ENOSPC case, we won't end up silently >> leaking blocks that will be lost until the user somehow decides to run >> fsck. > I see. Now I understand your patch. Honestly, seeing how fragile is trying > to fix extent tree after split has failed in the middle, I would probably > go even further and make sure we fix the tree properly in case of ENOSPC > and EDQUOT (those are easily user triggerable). Anything else indicates a > HW problem or fs corruption so I'd rather leave the extent tree as is and > don't try to fix it (which also means we will not create overlapping > extents). So something like: > > > err = ext4_ext_insert_extent(handle, inode, ppath, &newex, flags); > - if (err == -ENOSPC && (EXT4_EXT_MAY_ZEROOUT & split_flag)) { > + if (err == -ENOSPC || err == -EDQUOT) { > + if (EXT4_EXT_MAY_ZEROOUT & split_flag) > + err = handle zeroing... > if (err) { > fix extent len > goto out; > } > ... > } > > and in all other cases just 'goto out' in case of error. What do you think? > > Honza Thanks for your suggesttion. If you have no objection to following modification, i will send it as V4. diff --git a/fs/ext4/extents.c b/fs/ext4/extents.c index 77c84d6f1af6..f9cbd11e1eae 100644 --- a/fs/ext4/extents.c +++ b/fs/ext4/extents.c @@ -3206,7 +3206,10 @@ static int ext4_split_extent_at(handle_t *handle, ext4_ext_mark_unwritten(ex2); err = ext4_ext_insert_extent(handle, inode, ppath, &newex, flags); - if (err == -ENOSPC && (EXT4_EXT_MAY_ZEROOUT & split_flag)) { + if (err != -ENOSPC && err != -EDQUOT) + goto out; + + if (EXT4_EXT_MAY_ZEROOUT & split_flag) { if (split_flag & (EXT4_EXT_DATA_VALID1|EXT4_EXT_DATA_VALID2)) { if (split_flag & EXT4_EXT_DATA_VALID1) { err = ext4_ext_zeroout(inode, ex2); @@ -3232,22 +3235,23 @@ static int ext4_split_extent_at(handle_t *handle, ext4_ext_pblock(&orig_ex)); } - if (err) - goto fix_extent_len; - /* update the extent length and mark as initialized */ - ex->ee_len = cpu_to_le16(ee_len); - ext4_ext_try_to_merge(handle, inode, path, ex); - err = ext4_ext_dirty(handle, inode, path + path->p_depth); - if (err) - goto fix_extent_len; - - /* update extent status tree */ - err = ext4_zeroout_es(inode, &zero_ex); - - goto out; - } else if (err) - goto fix_extent_len; - + if (!err) { + /* update the extent length and mark as initialized */ + ex->ee_len = cpu_to_le16(ee_len); + ext4_ext_try_to_merge(handle, inode, path, ex); + err = ext4_ext_dirty(handle, inode, path + path->p_depth); + if (!err) + /* update extent status tree */ + err = ext4_zeroout_es(inode, &zero_ex); + /* At here, ext4_ext_try_to_merge maybe already merge + * extent, if fix origin extent length may lead to + * overwritten. + */ + goto out; + } + } + if (err) + goto fix_extent_len; out: ext4_ext_show_leaf(inode, path); return err;
Powered by blists - more mailing lists