linux-ext4 - Re: [PATCHv3 1/1] ext4: Optimize file overwrites

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CA+icZUXvhDR2TFEemfGyt=twxHBcu0_9b0H1j5cwGPG5kGA+TA@mail.gmail.com>
Date:   Fri, 18 Sep 2020 09:52:18 +0200
From:   Sedat Dilek <sedat.dilek@...il.com>
To:     Ritesh Harjani <riteshh@...ux.ibm.com>
Cc:     linux-ext4@...r.kernel.org, tytso@....edu, jack@...e.cz,
        dan.j.williams@...el.com, anju@...ux.vnet.ibm.com,
        linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: Re: [PATCHv3 1/1] ext4: Optimize file overwrites

On Fri, Sep 18, 2020 at 7:09 AM Ritesh Harjani <riteshh@...ux.ibm.com> wrote:
>
> In case if the file already has underlying blocks/extents allocated
> then we don't need to start a journal txn and can directly return
> the underlying mapping. Currently ext4_iomap_begin() is used by
> both DAX & DIO path. We can check if the write request is an
> overwrite & then directly return the mapping information.
>
> This could give a significant perf boost for multi-threaded writes
> specially random overwrites.
> On PPC64 VM with simulated pmem(DAX) device, ~10x perf improvement
> could be seen in random writes (overwrite). Also bcoz this optimizes
> away the spinlock contention during jbd2 slab cache allocation
> (jbd2_journal_handle). On x86 VM, ~2x perf improvement was observed.
>
> Reported-by: Dan Williams <dan.j.williams@...el.com>
> Suggested-by: Jan Kara <jack@...e.cz>
> Signed-off-by: Ritesh Harjani <riteshh@...ux.ibm.com>

I have applied your patch on top of recent Linus Git and boot-tested on x86-64.

Here I have LTP installed.
If you have a LTP filesystem test-/use-case you know for testing,
please let me know.

Yes, I have seen the FIO config in the cover-letter.
Maybe you have a different FIO config - 16G AFAIK is too big here.

Feel free to add...

Tested-by: Sedat Dilek <sedat.dilek@...il.com> # Compile and boot on
x86-64 Debian/unstable

Thanks.

- Sedat -

> ---
>  fs/ext4/inode.c | 18 +++++++++++++++---
>  1 file changed, 15 insertions(+), 3 deletions(-)
>
> diff --git a/fs/ext4/inode.c b/fs/ext4/inode.c
> index 10dd470876b3..6eae17758ece 100644
> --- a/fs/ext4/inode.c
> +++ b/fs/ext4/inode.c
> @@ -3437,14 +3437,26 @@ static int ext4_iomap_begin(struct inode *inode, loff_t offset, loff_t length,
>         map.m_len = min_t(loff_t, (offset + length - 1) >> blkbits,
>                           EXT4_MAX_LOGICAL_BLOCK) - map.m_lblk + 1;
>
> -       if (flags & IOMAP_WRITE)
> +       if (flags & IOMAP_WRITE) {
> +               /*
> +                * We check here if the blocks are already allocated, then we
> +                * don't need to start a journal txn and we can directly return
> +                * the mapping information. This could boost performance
> +                * especially in multi-threaded overwrite requests.
> +                */
> +               if (offset + length <= i_size_read(inode)) {
> +                       ret = ext4_map_blocks(NULL, inode, &map, 0);
> +                       if (ret > 0 && (map.m_flags & EXT4_MAP_MAPPED))
> +                               goto out;
> +               }
>                 ret = ext4_iomap_alloc(inode, &map, flags);
> -       else
> +       } else {
>                 ret = ext4_map_blocks(NULL, inode, &map, 0);
> +       }
>
>         if (ret < 0)
>                 return ret;
> -
> +out:
>         ext4_set_iomap(inode, iomap, &map, offset, length);
>
>         return 0;
> --
> 2.26.2
>