linux-kernel - Re: [PATCH v3 00/13] fiemap extension for more physical information

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20240403182022.GB6375@frogsfrogsfrogs>
Date: Wed, 3 Apr 2024 11:20:22 -0700
From: "Darrick J. Wong" <djwong@...nel.org>
To: Kent Overstreet <kent.overstreet@...ux.dev>
Cc: Sweet Tea Dorminy <sweettea-kernel@...miny.me>,
	Jonathan Corbet <corbet@....net>, Brian Foster <bfoster@...hat.com>,
	Chris Mason <clm@...com>, Josef Bacik <josef@...icpanda.com>,
	David Sterba <dsterba@...e.com>, Jaegeuk Kim <jaegeuk@...nel.org>,
	Chao Yu <chao@...nel.org>, Alexander Viro <viro@...iv.linux.org.uk>,
	Christian Brauner <brauner@...nel.org>, Jan Kara <jack@...e.cz>,
	Mickaël Salaün <mic@...ikod.net>,
	linux-doc@...r.kernel.org, linux-kernel@...r.kernel.org,
	linux-bcachefs@...r.kernel.org, linux-btrfs@...r.kernel.org,
	linux-f2fs-devel@...ts.sourceforge.net,
	linux-fsdevel@...r.kernel.org, kernel-team@...a.com
Subject: Re: [PATCH v3 00/13] fiemap extension for more physical information

On Wed, Apr 03, 2024 at 02:17:26PM -0400, Kent Overstreet wrote:
> On Wed, Apr 03, 2024 at 03:22:41AM -0400, Sweet Tea Dorminy wrote:
> > For many years, various btrfs users have written programs to discover
> > the actual disk space used by files, using root-only interfaces.
> > However, this information is a great fit for fiemap: it is inherently
> > tied to extent information, all filesystems can use it, and the
> > capabilities required for FIEMAP make sense for this additional
> > information also.
> > 
> > Hence, this patchset adds various additional information to fiemap,
> > and extends filesystems (but not iomap) to return it.  This uses some of
> > the reserved padding in the fiemap extent structure, so programs unaware
> > of the changes will be unaffected.
> > 
> > This is based on next-20240403. I've tested the btrfs part of this with
> > the standard btrfs testing matrix locally and manually, and done minimal
> > testing of the non-btrfs parts.
> > 
> > I'm unsure whether btrfs should be returning the entire physical extent
> > referenced by a particular logical range, or just the part of the
> > physical extent referenced by that range. The v2 thread has a discussion
> > of this.
> 
> I believe there was some talk of using the padding for a device ID, so
> that fiemap could properly support multi device filesystems. Are we sure
> this is the best use of those bytes?

We still have 5x u32 of empty space in struct fiemap after this series,
so I don't think adding the physical length is going to prohibit future
expansion.

--D

> > 
> > Changelog:
> > 
> > v3: 
> >  - Adapted all the direct users of fiemap, except iomap, to emit
> >    the new fiemap information, as far as I understand the other
> >    filesystems.
> > 
> > v2:
> >  - Adopted PHYS_LEN flag and COMPRESSED flag from the previous version,
> >    as per Andreas Dilger' comment.
> >    https://patchwork.ozlabs.org/project/linux-ext4/patch/4f8d5dc5b51a43efaf16c39398c23a6276e40a30.1386778303.git.dsterba@suse.cz/
> >  - https://lore.kernel.org/linux-fsdevel/cover.1711588701.git.sweettea-kernel@dorminy.me/T/#t
> > 
> > v1: https://lore.kernel.org/linux-fsdevel/20240315030334.GQ6184@frogsfrogsfrogs/T/#t
> > 
> > Sweet Tea Dorminy (13):
> >   fs: fiemap: add physical_length field to extents
> >   fs: fiemap: update fiemap_fill_next_extent() signature
> >   fs: fiemap: add new COMPRESSED extent state
> >   btrfs: fiemap: emit new COMPRESSED state.
> >   btrfs: fiemap: return extent physical size
> >   nilfs2: fiemap: return correct extent physical length
> >   ext4: fiemap: return correct extent physical length
> >   f2fs: fiemap: add physical length to trace_f2fs_fiemap
> >   f2fs: fiemap: return correct extent physical length
> >   ocfs2: fiemap: return correct extent physical length
> >   bcachefs: fiemap: return correct extent physical length
> >   f2fs: fiemap: emit new COMPRESSED state
> >   bcachefs: fiemap: emit new COMPRESSED state
> > 
> >  Documentation/filesystems/fiemap.rst | 35 ++++++++++----
> >  fs/bcachefs/fs.c                     | 17 +++++--
> >  fs/btrfs/extent_io.c                 | 72 ++++++++++++++++++----------
> >  fs/ext4/extents.c                    |  3 +-
> >  fs/f2fs/data.c                       | 36 +++++++++-----
> >  fs/f2fs/inline.c                     |  7 +--
> >  fs/ioctl.c                           | 11 +++--
> >  fs/iomap/fiemap.c                    |  2 +-
> >  fs/nilfs2/inode.c                    | 18 ++++---
> >  fs/ntfs3/frecord.c                   |  7 +--
> >  fs/ocfs2/extent_map.c                | 10 ++--
> >  fs/smb/client/smb2ops.c              |  1 +
> >  include/linux/fiemap.h               |  2 +-
> >  include/trace/events/f2fs.h          | 10 ++--
> >  include/uapi/linux/fiemap.h          | 34 ++++++++++---
> >  15 files changed, 178 insertions(+), 87 deletions(-)
> > 
> > 
> > base-commit: 75e31f66adc4c8d049e8aac1f079c1639294cd65
> > -- 
> > 2.43.0
> > 
>