linux-kernel - Re: [f2fs-dev] [PATCH v2 00/13] f2fs: introduce inline tail

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-Id: <20240914024112.1069698-1-bo.wu@vivo.com>
Date: Fri, 13 Sep 2024 20:41:12 -0600
From: Wu Bo <bo.wu@...o.com>
To: Chao Yu <chao@...nel.org>
Cc: Wu Bo <bo.wu@...o.com>,
	linux-kernel@...r.kernel.org,
	Wu Bo <wubo.oduw@...il.com>,
	Jaegeuk Kim <jaegeuk@...nel.org>,
	linux-f2fs-devel@...ts.sourceforge.net
Subject: Re: [f2fs-dev] [PATCH v2 00/13] f2fs: introduce inline tail

On Thu, Sep 12, 2024 at 03:14:24PM +0800, Chao Yu via Linux-f2fs-devel wrote:
> On 2024/9/11 11:57, Wu Bo wrote:
> > The inode in F2FS occupies an entire 4k block. For many small files, this means
> > they consume much more space than their actual size. Therefore, there is
> > significant potential to better utilize the inode block space.
> > 
> > Currently, F2FS has two features to make use of the inode block space: inline
> > data and inline xattr.
> > 
> > Inline data stores file which size is smaller then 3.5k in inode block. However,
> > for slightly larger small files, there still have much waste.
> > For example, a 5k file requires 3 blocks, totaling 12k of space, which is
> > more than twice the size of the file itself!
> > 
> > Additionally, the end of a file often does not occupy an entire block. If we can
> > store the end of the file data within the inode block, we can save an entire
> > block for the file. This is particularly important for small files.
> > 
> > In fact, the current inline data is a special case of inline tail, and
> > inline tail is an extension of inline data.
> > 
> > To make it simple, inline tail only on small files(<64k). And for larger files,
> > inline tails don't provide any significant benefits.
> > 
> > The layout of an inline tail inode block is following:
> > 
> > | inode block     | 4096 |     inline tail enable    |
> > | --------------- | ---- | --------------------------|
> > | inode info      | 360  |                           |
> > | --------------- | ---- | --------------------------|
> > |                 |      | extra info         | 0~36 |
> > |                 |      | **compact_addr[16] | 64   |
> > | addr table[923] | 3692 | reserved           | 4    |
> > |                 |      | **tail data        |      |
> > |                 |      | inline_xattr       | 200  |
> > | --------------- | ---- | --------------------------|
> > | nid table[5]    | 20   |
> > | node footer     | 24   |
> > 
> > F2fs-tools to support inline tail:
> > https://lore.kernel.org/linux-f2fs-devel/20240903075931.3339584-1-bo.wu@vivo.com
> > 
> > I tested inline tail by copying the source code of Linux 6.9.7. The storage
> > space was reduced by approximately 8%. Additionally, due to the reduced IO, the
> > copy time also reduced by around 10%.
> > 
> > This patch series has been tested with xfstests by running 'kvm-xfstests -c f2fs
> > -g quick' both with and without the patch; no regressions were observed.
> > The test result is:
> > f2fs/default: 583 tests, 6 failures, 213 skipped, 650 seconds
> >    Failures: generic/050 generic/064 generic/250 generic/252 generic/563
> >        generic/735
> >        Totals: 607 tests, 213 skipped, 30 failures, 0 errors, 579s
> 
> MKFS_OPTIONS  -- -O extra_attr,encrypt,inode_checksum,flexible_inline_xattr,inode_crtime,verity,compression -f /dev/vdc
> MOUNT_OPTIONS -- -o acl,user_xattr -o discard,inline_tail /dev/vdc /mnt/scratch_f2fs

Hi Chao,

I used the default cfg to run xfstest-bld and didn't encounter the failed
failures. This suggests the issue might be related to these additional options.
However, I'm not sure how to include these options when running xfstest-bld.
Could you let me know how to add them?

Thanks

> 
> Before:
> Failures: generic/042 generic/050 generic/250 generic/252 generic/270 generic/389 generic/563 generic/700 generic/735
> Failed 9 of 746 tests
> 
> After:
> Failures: generic/042 generic/050 generic/125 generic/250 generic/252 generic/270 generic/389 generic/418 generic/551 generic/563 generic/700 generic/735
> Failed 12 of 746 tests
> 
> Failures: f2fs/004
> 
> Can you please check failed testcases?
> 
> Thanks,
> 
> > 
> > ---
> > v2:
> > - fix ARCH=arc build warning
> > 
> > ---
> > Wu Bo (13):
> >    f2fs: add inline tail mount option
> >    f2fs: add inline tail disk layout definition
> >    f2fs: implement inline tail write & truncate
> >    f2fs: implement inline tail read & fiemap
> >    f2fs: set inline tail flag when create inode
> >    f2fs: fix address info has been truncated
> >    f2fs: support seek for inline tail
> >    f2fs: convert inline tail when inode expand
> >    f2fs: fix data loss during inline tail writing
> >    f2fs: avoid inlining quota files
> >    f2fs: fix inline tail data lost
> >    f2fs: convert inline tails to avoid potential issues
> >    f2fs: implement inline tail forward recovery
> > 
> >   fs/f2fs/data.c     |  93 +++++++++++++++++++++++++-
> >   fs/f2fs/f2fs.h     |  46 ++++++++++++-
> >   fs/f2fs/file.c     |  85 +++++++++++++++++++++++-
> >   fs/f2fs/inline.c   | 159 +++++++++++++++++++++++++++++++++++++++------
> >   fs/f2fs/inode.c    |   6 ++
> >   fs/f2fs/namei.c    |   3 +
> >   fs/f2fs/node.c     |   6 +-
> >   fs/f2fs/recovery.c |   9 ++-
> >   fs/f2fs/super.c    |  25 +++++++
> >   fs/f2fs/verity.c   |   4 ++
> >   10 files changed, 409 insertions(+), 27 deletions(-)
> > 
> > 
> > base-commit: 67784a74e258a467225f0e68335df77acd67b7ab
> 
> 
> 
> _______________________________________________
> Linux-f2fs-devel mailing list
> Linux-f2fs-devel@...ts.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel