linux-kernel - Re: [PATCH 1/1] quota: flush quota_release

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87plmwcjcd.fsf@gmail.com>
Date: Sat, 16 Nov 2024 02:20:26 +0530
From: Ritesh Harjani (IBM) <ritesh.list@...il.com>
To: Ojaswin Mujoo <ojaswin@...ux.ibm.com>, linux-ext4@...r.kernel.org, Jan Kara <jack@...e.com>
Cc: linux-kernel@...r.kernel.org, linux-fsdevel@...r.kernel.org, Disha Goel <disgoel@...ux.ibm.com>
Subject: Re: [PATCH 1/1] quota: flush quota_release_work upon quota writeback

Ojaswin Mujoo <ojaswin@...ux.ibm.com> writes:

> One of the paths quota writeback is called from is:
>
> freeze_super()
>   sync_filesystem()
>     ext4_sync_fs()
>       dquot_writeback_dquots()
>
> Since we currently don't always flush the quota_release_work queue in
> this path, we can end up with the following race:
>
>  1. dquot are added to releasing_dquots list during regular operations.
>  2. FS freeze starts, however, this does not flush the quota_release_work queue.
>  3. Freeze completes.
>  4. Kernel eventually tries to flush the workqueue while FS is frozen which
>     hits a WARN_ON since transaction gets started during frozen state:
>
>   ext4_journal_check_start+0x28/0x110 [ext4] (unreliable)
>   __ext4_journal_start_sb+0x64/0x1c0 [ext4]
>   ext4_release_dquot+0x90/0x1d0 [ext4]
>   quota_release_workfn+0x43c/0x4d0
>
> Which is the following line:
>
>   WARN_ON(sb->s_writers.frozen == SB_FREEZE_COMPLETE);
>
> Which ultimately results in generic/390 failing due to dmesg
> noise. This was detected on powerpc machine 15 cores.
>
> To avoid this, make sure to flush the workqueue during
> dquot_writeback_dquots() so we dont have any pending workitems after
> freeze.

Not just that, sync_filesystem can also be called from other places and
quota_release_workfn() could write out and and release the dquot
structures if such are found during processing of releasing_dquots list. 
IIUC, this was earlier done in the same dqput() context but had races
with dquot_mark_dquot_dirty(). Hence the final dqput() will now add the
dquot structures to releasing_dquots list and will schedule a delayed
workfn which will process the releasing_dquots list. 

And so after the final dqput and before the release_workfn gets
scheduled, if dquot gets marked as dirty or dquot_transfer gets called -
then I am suspecting that it could lead to a dirty or an active dquot.

Hence, flushing the delayed quota_release_work at the end of
dquot_writeback_dquots() looks like the right thing to do IMO.

But I can give another look as this part of the code is not that well
known to me. 

>
> Reported-by: Disha Goel <disgoel@...ux.ibm.com>
> Signed-off-by: Ojaswin Mujoo <ojaswin@...ux.ibm.com>
> ---

Maybe a fixes tag as well?

>  fs/quota/dquot.c | 2 ++
>  1 file changed, 2 insertions(+)
>
> diff --git a/fs/quota/dquot.c b/fs/quota/dquot.c
> index 3dd8d6f27725..2782cfc8c302 100644
> --- a/fs/quota/dquot.c
> +++ b/fs/quota/dquot.c
> @@ -729,6 +729,8 @@ int dquot_writeback_dquots(struct super_block *sb, int type)
>  			sb->dq_op->write_info(sb, cnt);
>  	dqstats_inc(DQST_SYNCS);
>  
> +	flush_delayed_work(&quota_release_work);
> +
>  	return ret;
>  }
>  EXPORT_SYMBOL(dquot_writeback_dquots);
> -- 
> 2.43.5