| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-ID: <CAD14+f349Fsx9syBzEtgbOXEts0FRrHWU8vUgbpRO-SJ8DMmJw@mail.gmail.com> Date: Fri, 6 Sep 2024 23:52:49 +0900 From: Juhyung Park <qkrwngud825@...il.com> To: Wu Bo <bo.wu@...o.com> Cc: Wu Bo <wubo.oduw@...il.com>, Jaegeuk Kim <jaegeuk@...nel.org>, Chao Yu <chao@...nel.org>, linux-kernel@...r.kernel.org, linux-f2fs-devel@...ts.sourceforge.net Subject: Re: [f2fs-dev] [PATCH 00/13] f2fs: introduce inline tail Hi Wu, On Fri, Sep 6, 2024 at 7:42 PM Wu Bo <bo.wu@...o.com> wrote: > > On Fri, Sep 06, 2024 at 06:02:42PM +0900, Juhyung Park wrote: > > Hi Wu, > > > > Interesting patch-set. > > > > A quick test here on my daily-driver phone's data (785558 inodes) with > > both compression and encryption disabled, copied via rsync to 2 fresh > > f2fs partitions with and without inline tail: > > Before: 170064928KiB > > After: 169249780KiB > > > > So about 0.48% saved. > > > Hi Juhyung, > > Thanks for your test. If it's not too much trouble, please help check the f2fs > statistics: > cat /sys/kernel/debug/f2fs/status | grep Utilization # w/o inline tail Utilization: 70% (42153224 valid blocks, 0 discard blocks) # w/ inline tail Utilization: 69% (41949437 valid blocks, 0 discard blocks) > > However, it’s more likely that the benefits of inline tail are indeed limited on > mobile devices. I previously evaluated the gains on my own phone, and they were > less than 1% too. > > From the data on your phone, the average size is 170064928KiB/785558≈200KiB. > Storage space on phones is primarily consumed by multimedia files, so the > proportion of files smaller than 64KB is quite small. > > Therefore, compared to phone storage, scenarios like storing Linux source code, > which involve many small files, are more likely to yield noticeable benefits. Mmm. I do have my own Linux setups using f2fs as well with extended node bitmap: # Workstation, 2.5T/3.5T, 55602404 inodes Utilization: 70% (657421720 valid blocks, 902 discard blocks) # Laptop, 736G/1.9T, 12229380 inodes Utilization: 39% (190980868 valid blocks, 2887 discard blocks) I do have a lot of small files here, it'll be interesting to test this out but I can't afford to run an unstable kernel for those atm. (Not to mention finding a new SSD for migration.) > > However, don't be too disappointed with it, as it can still double the benefits > based on the existing gains from inline data. > Inline data: > https://lore.kernel.org/linux-f2fs-devel/1384096401-25169-1-git-send-email-huajun.li.lee@gmail.com/T/#u > > Thanks, > Wu Bo > > > Let me know if this is an unexpected result. > > > > Thanks, > > > > On Tue, Sep 3, 2024 at 5:42 PM Wu Bo via Linux-f2fs-devel > > <linux-f2fs-devel@...ts.sourceforge.net> wrote: > > > > > > The inode in F2FS occupies an entire 4k block. For many small files, this means > > > they consume much more space than their actual size. Therefore, there is > > > significant potential to better utilize the inode block space. > > > > > > Currently, F2FS has two features to make use of the inode block space: inline > > > data and inline xattr. > > > > > > Inline data stores file which size is smaller then 3.5k in inode block. However, > > > for slightly larger small files, there still have much waste. > > > For example, a 5k file requires 3 blocks, totaling 12k of space, which is > > > more than twice the size of the file itself! > > > > > > Additionally, the end of a file often does not occupy an entire block. If we can > > > store the end of the file data within the inode block, we can save an entire > > > block for the file. This is particularly important for small files. > > > > > > In fact, the current inline data is a special case of inline tail, and > > > inline tail is an extension of inline data. > > > > > > To make it simple, inline tail only on small files(<64k). And for larger files, > > > inline tails don't provide any significant benefits. > > > > > > The layout of an inline tail inode block is following: > > > > > > | inode block | 4096 | inline tail enable | > > > | --------------- | ---- | --------------------------| > > > | inode info | 360 | | > > > | --------------- | ---- | --------------------------| > > > | | | extra info | 0~36 | > > > | | | **compact_addr[16] | 64 | > > > | addr table[923] | 3692 | reserved | 4 | > > > | | | **tail data | | > > > | | | inline_xattr | 200 | > > > | --------------- | ---- | --------------------------| > > > | nid table[5] | 20 | > > > | node footer | 24 | > > > > > > F2fs-tools to support inline tail: > > > https://lore.kernel.org/linux-f2fs-devel/20240903075931.3339584-1-bo.wu@vivo.com > > > > > > I tested inline tail by copying the source code of Linux 6.9.7. The storage > > > space was reduced by approximately 8%. Additionally, due to the reduced IO, the > > > copy time also reduced by around 10%. > > > > > > This patch series has been tested with xfstests by running 'kvm-xfstests -c f2fs > > > -g quick' both with and without the patch; no regressions were observed. > > > The test result is: > > > f2fs/default: 583 tests, 6 failures, 213 skipped, 650 seconds > > > Failures: generic/050 generic/064 generic/250 generic/252 generic/563 > > > generic/735 > > > Totals: 607 tests, 213 skipped, 30 failures, 0 errors, 579s > > > > > > Wu Bo (13): > > > f2fs: add inline tail mount option > > > f2fs: add inline tail disk layout definition > > > f2fs: implement inline tail write & truncate > > > f2fs: implement inline tail read & fiemap > > > f2fs: set inline tail flag when create inode > > > f2fs: fix address info has been truncated > > > f2fs: support seek for inline tail > > > f2fs: convert inline tail when inode expand > > > f2fs: fix data loss during inline tail writing > > > f2fs: avoid inlining quota files > > > f2fs: fix inline tail data lost > > > f2fs: convert inline tails to avoid potential issues > > > f2fs: implement inline tail forward recovery > > > > > > fs/f2fs/data.c | 93 +++++++++++++++++++++++++- > > > fs/f2fs/f2fs.h | 46 ++++++++++++- > > > fs/f2fs/file.c | 85 +++++++++++++++++++++++- > > > fs/f2fs/inline.c | 159 +++++++++++++++++++++++++++++++++++++++------ > > > fs/f2fs/inode.c | 6 ++ > > > fs/f2fs/namei.c | 3 + > > > fs/f2fs/node.c | 6 +- > > > fs/f2fs/recovery.c | 9 ++- > > > fs/f2fs/super.c | 25 +++++++ > > > fs/f2fs/verity.c | 4 ++ > > > 10 files changed, 409 insertions(+), 27 deletions(-) > > > > > > -- > > > 2.35.3 > > > > > > > > > > > > _______________________________________________ > > > Linux-f2fs-devel mailing list > > > Linux-f2fs-devel@...ts.sourceforge.net > > > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel > > > > > > _______________________________________________ > > Linux-f2fs-devel mailing list > > Linux-f2fs-devel@...ts.sourceforge.net > > https://lists.sourceforge.net/lists/listinfo/linux-f2fs-devel
Powered by blists - more mailing lists