linux-kernel - Re: [f2fs-dev] [PATCH 2/3] f2fs: run GCs synchronously given user requests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Tue, 28 Jun 2022 11:17:26 -0700
From:   Jaegeuk Kim <jaegeuk@...nel.org>
To:     Chao Yu <chao@...nel.org>
Cc:     linux-kernel@...r.kernel.org,
        linux-f2fs-devel@...ts.sourceforge.net
Subject: Re: [f2fs-dev] [PATCH 2/3] f2fs: run GCs synchronously given user
 requests

On 06/28, Chao Yu wrote:
> On 2022/6/23 0:58, Jaegeuk Kim wrote:
> > On 06/22, Chao Yu wrote:
> > > On 2022/6/20 6:34, Jaegeuk Kim wrote:
> > > > On 06/19, Chao Yu wrote:
> > > > > On 2022/6/18 6:31, Jaegeuk Kim wrote:
> > > > > > When users set GC_URGENT or GC_MID, they expected to do GCs right away.
> > > > > > But, there's a condition to bypass it. Let's indicate we need to do now
> > > > > > in the thread.
> > > > > 
> > > > > .should_migrate_blocks is used to force migrating blocks in full
> > > > > section, so what is the condition here? GC should not never select
> > > > > a full section, right?
> > > > 
> > > > I think it'll move a full section given .victim_segno is not NULL_SEGNO,
> > > > as __get_victim will give a dirty segment all the time. wdyt?
> > > 
> > > However, in gc_thread_fun() victim_segno is NULL_SEGNO all the time.
> > 
> > What do you mean? The gc_thread thread sets NULL_SEGNO, which prevents
> > from selecting the full section. But, f2fs_ioc_flush_device will set the
> > segno to move, and f2fs_resize_fs calls do_garbage_collect directly.
> 
> Yes, but I didn't get why this patch updates .should_migrate_blocks for
> gc_thread_func() case? If this is added to avoid breaking from below condition,
> I guess it's not necessary, since victim selected from gc_thread_func() will
> always be dirty, so get_valid_blocks(sbi, segno, true) == BLKS_PER_SEC(sbi)
> will be false anyway? or am I missing something?

I lost the previous test. :( IIRC, once I set GC_URGENT_HIGH, I've seen that
f2fs_gc couldn't migrate blocks quickly, even or never succeeded. And, I
also suspected the below condition tho, if I give force_migrate, it
simply worked. I just realized that that may be caused by large sections
having non-2MB-aligned zone capacity. Will check it again, as I found
some buggy flows in that case.

> 
> 		/*
> 		 * stop BG_GC if there is not enough free sections.
> 		 * Or, stop GC if the segment becomes fully valid caused by
> 		 * race condition along with SSR block allocation.
> 		 */
> 		if ((gc_type == BG_GC && has_not_enough_free_secs(sbi, 0, 0)) ||
> 			(!force_migrate && get_valid_blocks(sbi, segno, true) ==
> 							BLKS_PER_SEC(sbi)))
> 			return submitted;
> 
> Thanks,
> 
> > 
> > > 
> > > I guess .should_migrate_blocks should only be set to true for
> > > F2FS_IOC_FLUSH_DEVICE/F2FS_IOC_RESIZE_FS case? rather than GC_URGENT or GC_MID
> > > case? See commit 7dede88659df ("f2fs: fix to allow migrating fully valid segment").
> > > 
> > > Thanks,
> > > 
> > > > 
> > > > > 
> > > > > Thanks,
> > > > > 
> > > > > > 
> > > > > > Signed-off-by: Jaegeuk Kim <jaegeuk@...nel.org>
> > > > > > ---
> > > > > >     fs/f2fs/gc.c | 8 ++++++--
> > > > > >     1 file changed, 6 insertions(+), 2 deletions(-)
> > > > > > 
> > > > > > diff --git a/fs/f2fs/gc.c b/fs/f2fs/gc.c
> > > > > > index d5fb426e0747..f4aa3c88118b 100644
> > > > > > --- a/fs/f2fs/gc.c
> > > > > > +++ b/fs/f2fs/gc.c
> > > > > > @@ -37,7 +37,6 @@ static int gc_thread_func(void *data)
> > > > > >     	unsigned int wait_ms;
> > > > > >     	struct f2fs_gc_control gc_control = {
> > > > > >     		.victim_segno = NULL_SEGNO,
> > > > > > -		.should_migrate_blocks = false,
> > > > > >     		.err_gc_skipped = false };
> > > > > >     	wait_ms = gc_th->min_sleep_time;
> > > > > > @@ -113,7 +112,10 @@ static int gc_thread_func(void *data)
> > > > > >     				sbi->gc_mode == GC_URGENT_MID) {
> > > > > >     			wait_ms = gc_th->urgent_sleep_time;
> > > > > >     			f2fs_down_write(&sbi->gc_lock);
> > > > > > +			gc_control.should_migrate_blocks = true;
> > > > > >     			goto do_gc;
> > > > > > +		} else {
> > > > > > +			gc_control.should_migrate_blocks = false;
> > > > > >     		}
> > > > > >     		if (foreground) {
> > > > > > @@ -139,7 +141,9 @@ static int gc_thread_func(void *data)
> > > > > >     		if (!foreground)
> > > > > >     			stat_inc_bggc_count(sbi->stat_info);
> > > > > > -		sync_mode = F2FS_OPTION(sbi).bggc_mode == BGGC_MODE_SYNC;
> > > > > > +		sync_mode = F2FS_OPTION(sbi).bggc_mode == BGGC_MODE_SYNC ||
> > > > > > +				sbi->gc_mode == GC_URGENT_HIGH ||
> > > > > > +				sbi->gc_mode == GC_URGENT_MID;
> > > > > >     		/* foreground GC was been triggered via f2fs_balance_fs() */
> > > > > >     		if (foreground)