linux-kernel - Re: [f2fs-dev] [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to allow more data to be written with the CP disable

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CAHJ8P3KhfuCSNwyc16QYQQYMNzjVKc4ey5CTZ2XK4893aW4mCQ@mail.gmail.com>
Date: Fri, 18 Oct 2024 09:52:58 +0800
From: Zhiguo Niu <niuzhiguo84@...il.com>
To: Chao Yu <chao@...nel.org>
Cc: Qi Han <hanqi@...o.com>, jaegeuk@...nel.org, linux-kernel@...r.kernel.org, 
	linux-f2fs-devel@...ts.sourceforge.net
Subject: Re: [f2fs-dev] [PATCH] f2fs: modify f2fs_is_checkpoint_ready logic to
 allow more data to be written with the CP disable

Chao Yu via Linux-f2fs-devel <linux-f2fs-devel@...ts.sourceforge.net>
于2024年10月17日周四 17:57写道：
>
> On 2024/10/9 18:27, Qi Han wrote:
> > When the free segment is used up during CP disable, many write or
> > ioctl operations will get ENOSPC error codes, even if there are
> > still many blocks available. We can reproduce it in the following
> > steps:
> >
> > dd if=/dev/zero of=f2fs.img bs=1M count=55
> > mkfs.f2fs -f f2fs.img
> > mount f2fs.img f2fs_dir -o checkpoint=disable:10%
> > cd f2fs_dir
> > dd if=/dev/zero of=bigfile bs=1M count=50
> > sync
> > rm bigfile
> > i=1; while [[ $i -lt 10000000 ]]; do (file_name=./file$i; dd \
> > if=/dev/random of=$file_name bs=1M count=0); i=$((i+1)); done
> > stat -f ./
> >
> > In f2fs_need_SSR() function, it is allowed to use SSR to allocate
> > blocks when CP is disabled, so in f2fs_is_checkpoint_ready function,
Hi Chao and Qi,
based on this comment  and codes, I have some doubts:
unusable blocks are calculated  from  invalid blocks of Dirty segment
in f2fs_get_unsuable_blocks then minus ovp_hole,
but SSR is allowed when the checkpoint is disabled.
So actually SSR can not use most invalid blocks of Dirty segments?
Is this a contradiction?
Thanks!

> > can we judge the number of invalid blocks when free segment is not
> > enough, and return ENOSPC only if the number of invalid blocks is
> > also not enough?
>
> Can you please try below patch?
>
>  From 38b7c97dcc55ba83de4220c3dc54c2eb66148dd5 Mon Sep 17 00:00:00 2001
> From: Chao Yu <chao@...nel.org>
> Date: Thu, 17 Oct 2024 17:07:05 +0800
> Subject: [PATCH] f2fs: revalidate empty segment when checkpoint is disabled
>
> If checkpoint is off, let's set segment as free once all newly
> written datas were removed.
>
> Signed-off-by: Chao Yu <chao@...nel.org>
> ---
>   fs/f2fs/segment.c | 13 ++++++++++++-
>   1 file changed, 12 insertions(+), 1 deletion(-)
>
> diff --git a/fs/f2fs/segment.c b/fs/f2fs/segment.c
> index f8d6e601e084..9bad13d70afb 100644
> --- a/fs/f2fs/segment.c
> +++ b/fs/f2fs/segment.c
> @@ -853,6 +853,17 @@ static void locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno)
>         valid_blocks = get_valid_blocks(sbi, segno, false);
>         ckpt_valid_blocks = get_ckpt_valid_blocks(sbi, segno, false);
>
> +       /*
> +        * If checkpoint is off, let's set segment as free once all newly
> +        * written datas were removed.
> +        */
> +       if (is_sbi_flag_set(sbi, SBI_CP_DISABLED) &&
> +               valid_blocks == 0 && ckpt_valid_blocks == 0) {
> +               __remove_dirty_segment(sbi, segno, DIRTY);
> +               __set_test_and_free(sbi, segno, false);
> +               goto out_lock;
> +       }
> +
>         if (valid_blocks == 0 && (!is_sbi_flag_set(sbi, SBI_CP_DISABLED) ||
>                 ckpt_valid_blocks == usable_blocks)) {
>                 __locate_dirty_segment(sbi, segno, PRE);
> @@ -863,7 +874,7 @@ static void locate_dirty_segment(struct f2fs_sb_info *sbi, unsigned int segno)
>                 /* Recovery routine with SSR needs this */
>                 __remove_dirty_segment(sbi, segno, DIRTY);
>         }
> -
> +out_lock:
>         mutex_unlock(&dirty_i->seglist_lock);
>   }
>
> --
> 2.40.1
>
> >
> > Signed-off-by: Qi Han <hanqi@...o.com>
> > ---
> >   fs/f2fs/segment.h | 21 +++++++++++++++++++++
> >   1 file changed, 21 insertions(+)
> >
> > diff --git a/fs/f2fs/segment.h b/fs/f2fs/segment.h
> > index 71adb4a43bec..9bf0cf3a6a31 100644
> > --- a/fs/f2fs/segment.h
> > +++ b/fs/f2fs/segment.h
> > @@ -637,12 +637,33 @@ static inline bool has_enough_free_secs(struct f2fs_sb_info *sbi,
> >       return !has_not_enough_free_secs(sbi, freed, needed);
> >   }
> >
> > +static inline bool has_enough_available_blocks(struct f2fs_sb_info *sbi)
> > +{
> > +     unsigned int total_free_blocks = sbi->user_block_count -
> > +                                     valid_user_blocks(sbi) -
> > +                                     sbi->current_reserved_blocks;
> > +
> > +     if (total_free_blocks <= sbi->unusable_block_count)
> > +             total_free_blocks = 0;
> > +     else
> > +             total_free_blocks -= sbi->unusable_block_count;
> > +
> > +     if (total_free_blocks > F2FS_OPTION(sbi).root_reserved_blocks)
> > +             total_free_blocks -= F2FS_OPTION(sbi).root_reserved_blocks;
> > +     else
> > +             total_free_blocks = 0;
> > +
> > +     return (total_free_blocks > 0) ? true : false;
> > +}
> > +
> >   static inline bool f2fs_is_checkpoint_ready(struct f2fs_sb_info *sbi)
> >   {
> >       if (likely(!is_sbi_flag_set(sbi, SBI_CP_DISABLED)))
> >               return true;
> >       if (likely(has_enough_free_secs(sbi, 0, 0)))
> >               return true;
> > +     if (likely(has_enough_available_blocks(sbi)))
> > +             return true;
> >       return false;
> >   }
> >
>
>
>
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@...ts.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel