| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Message-Id: <20230824-sowie-april-a3f262c64848@brauner>
Date: Thu, 24 Aug 2023 16:19:41 +0200
From: Christian Brauner <brauner@...nel.org>
To: Linus Torvalds <torvalds@...ux-foundation.org>
Cc: Christian Brauner <brauner@...nel.org>,
linux-fsdevel@...r.kernel.org, linux-kernel@...r.kernel.org
Subject: [GIT PULL] multi-grained timestamps
Hey Linus,
/* Summary */
This adds VFS support for multi-grain timestamps and converts tmpfs,
xfs, ext4, and btrfs to use them. This carries acks from all relevant
filesystems.
The VFS always uses coarse-grained timestamps when updating the ctime
and mtime after a change. This has the benefit of allowing filesystems
to optimize away a lot of metadata updates, down to around 1 per jiffy,
even when a file is under heavy writes.
Unfortunately, this has always been an issue when we're exporting via
NFSv3, which relies on timestamps to validate caches. A lot of changes
can happen in a jiffy, so timestamps aren't sufficient to help the
client decide to invalidate the cache.
Even with NFSv4, a lot of exported filesystems don't properly support a
change attribute and are subject to the same problems with timestamp
granularity. Other applications have similar issues with timestamps
(e.g., backup applications).
If we were to always use fine-grained timestamps, that would improve the
situation, but that becomes rather expensive, as the underlying
filesystem would have to log a lot more metadata updates.
This introduces fine-grained timestamps that are used when they are
actively queried.
This uses the 31st bit of the ctime tv_nsec field to indicate that
something has queried the inode for the mtime or ctime. When this flag
is set, on the next mtime or ctime update, the kernel will fetch a
fine-grained timestamp instead of the usual coarse-grained one.
As POSIX generally mandates that when the mtime changes, the ctime must
also change the kernel always stores normalized ctime values, so only
the first 30 bits of the tv_nsec field are ever used.
Filesytems can opt into this behavior by setting the FS_MGTIME flag in
the fstype. Filesystems that don't set this flag will continue to use
coarse-grained timestamps.
Various preparatory changes, fixes and cleanups are included:
* Fixup all relevant places where POSIX requires updating ctime together
with mtime. This is a wide-range of places and all maintainers
provided necessary Acks.
* Add new accessors for inode->i_ctime directly and change all callers
to rely on them. Plain accesses to inode->i_ctime are now gone and it
is accordingly rename to inode->__i_ctime and commented as requiring
accessors.
* Extend generic_fillattr() to pass in a request mask mirroring in a
sense the statx() uapi. This allows callers to pass in a request mask
to only get a subset of attributes filled in.
* Rework timestamp updates so it's possible to drop the @now parameter
the update_time() inode operation and associated helpers.
* Add inode_update_timestamps() and convert all filesystems to it
removing a bunch of open-coding.
/* Testing */
clang: Ubuntu clang version 15.0.7
gcc: (Ubuntu 12.2.0-3ubuntu1) 12.2.0
All patches are based on v6.5-rc1 and have been sitting in linux-next.
No build failures or warnings were observed. All old and new tests in
selftests, and LTP pass without regressions.
/* Conflicts */
This will have a merge conflict with the v6.6-vfs.tmpfs work. I'm
providing a conflict resolution in the v6.6-vfs.tmpfs pull request. So I
would suggest to merge this pull request here first.
It will also have conflicts with the following trees:
(1) linux-next: manual merge of the vfs-brauner tree with the btrfs tree
https://lore.kernel.org/lkml/20230815112023.5903355c@canb.auug.org.au
(2) linux-next: manual merge of the vfs-brauner tree with the f2fs tree
https://lore.kernel.org/lkml/20230815113013.40cbf98a@canb.auug.org.au
(3) linux-next: manual merge of the vfs-brauner tree with the f2fs tree
https://lore.kernel.org/lkml/20230815113357.11919e63@canb.auug.org.au
The following changes since commit 06c2afb862f9da8dc5efa4b6076a0e48c3fbaaa5:
Linux 6.5-rc1 (2023-07-09 13:53:13 -0700)
are available in the Git repository at:
git@...olite.kernel.org:pub/scm/linux/kernel/git/vfs/vfs tags/v6.6-vfs.ctime
for you to fetch changes up to 50e9ceef1d4f644ee0049e82e360058a64ec284c:
btrfs: convert to multigrain timestamps (2023-08-11 09:04:58 +0200)
Please consider pulling these changes from the signed v6.6-vfs.ctime tag.
Thanks!
Christian
----------------------------------------------------------------
v6.6-vfs.ctime
----------------------------------------------------------------
Jeff Layton (107):
ibmvmc: update ctime in conjunction with mtime on write
bfs: update ctime in addition to mtime when adding entries
efivarfs: update ctime when mtime changes on a write
exfat: ensure that ctime is updated whenever the mtime is
apparmor: update ctime whenever the mtime changes on an inode
cifs: update the ctime on a partial page write
fs: add ctime accessors infrastructure
fs: new helper: simple_rename_timestamp
btrfs: convert to simple_rename_timestamp
ubifs: convert to simple_rename_timestamp
shmem: convert to simple_rename_timestamp
exfat: convert to simple_rename_timestamp
ntfs3: convert to simple_rename_timestamp
reiserfs: convert to simple_rename_timestamp
spufs: convert to ctime accessor functions
s390: convert to ctime accessor functions
binderfs: convert to ctime accessor functions
infiniband: convert to ctime accessor functions
ibm: convert to ctime accessor functions
usb: convert to ctime accessor functions
9p: convert to ctime accessor functions
adfs: convert to ctime accessor functions
affs: convert to ctime accessor functions
afs: convert to ctime accessor functions
fs: convert to ctime accessor functions
autofs: convert to ctime accessor functions
befs: convert to ctime accessor functions
bfs: convert to ctime accessor functions
btrfs: convert to ctime accessor functions
ceph: convert to ctime accessor functions
coda: convert to ctime accessor functions
configfs: convert to ctime accessor functions
cramfs: convert to ctime accessor functions
debugfs: convert to ctime accessor functions
devpts: convert to ctime accessor functions
ecryptfs: convert to ctime accessor functions
efivarfs: convert to ctime accessor functions
efs: convert to ctime accessor functions
erofs: convert to ctime accessor functions
exfat: convert to ctime accessor functions
ext2: convert to ctime accessor functions
ext4: convert to ctime accessor functions
f2fs: convert to ctime accessor functions
fat: convert to ctime accessor functions
freevxfs: convert to ctime accessor functions
fuse: convert to ctime accessor functions
gfs2: convert to ctime accessor functions
hfs: convert to ctime accessor functions
hfsplus: convert to ctime accessor functions
hostfs: convert to ctime accessor functions
hpfs: convert to ctime accessor functions
hugetlbfs: convert to ctime accessor functions
isofs: convert to ctime accessor functions
jffs2: convert to ctime accessor functions
jfs: convert to ctime accessor functions
kernfs: convert to ctime accessor functions
nfs: convert to ctime accessor functions
nfsd: convert to ctime accessor functions
nilfs2: convert to ctime accessor functions
ntfs: convert to ctime accessor functions
ntfs3: convert to ctime accessor functions
ocfs2: convert to ctime accessor functions
omfs: convert to ctime accessor functions
openpromfs: convert to ctime accessor functions
orangefs: convert to ctime accessor functions
overlayfs: convert to ctime accessor functions
procfs: convert to ctime accessor functions
pstore: convert to ctime accessor functions
qnx4: convert to ctime accessor functions
qnx6: convert to ctime accessor functions
ramfs: convert to ctime accessor functions
reiserfs: convert to ctime accessor functions
romfs: convert to ctime accessor functions
smb: convert to ctime accessor functions
squashfs: convert to ctime accessor functions
sysv: convert to ctime accessor functions
tracefs: convert to ctime accessor functions
ubifs: convert to ctime accessor functions
udf: convert to ctime accessor functions
ufs: convert to ctime accessor functions
vboxsf: convert to ctime accessor functions
xfs: convert to ctime accessor functions
zonefs: convert to ctime accessor functions
linux: convert to ctime accessor functions
mqueue: convert to ctime accessor functions
bpf: convert to ctime accessor functions
shmem: convert to ctime accessor functions
sunrpc: convert to ctime accessor functions
apparmor: convert to ctime accessor functions
security: convert to ctime accessor functions
selinux: convert to ctime accessor functions
fs: rename i_ctime field to __i_ctime
gfs2: fix timestamp handling on quota inodes
fs: remove silly warning from current_time
fs: pass the request_mask to generic_fillattr
fs: drop the timespec64 arg from generic_update_time
btrfs: have it use inode_update_timestamps
ubifs: have ubifs_update_time use inode_update_timestamps
fat: remove i_version handling from fat_update_time
fat: make fat_update_time get its own timestamp
xfs: have xfs_vn_update_time gets its own timestamp
fs: drop the timespec64 argument from update_time
fs: add infrastructure for multigrain timestamps
tmpfs: add support for multigrain timestamps
xfs: switch to multigrain timestamps
ext4: switch to multigrain timestamps
btrfs: convert to multigrain timestamps
arch/powerpc/platforms/cell/spufs/inode.c | 2 +-
arch/s390/hypfs/inode.c | 4 +-
drivers/android/binderfs.c | 8 +-
drivers/infiniband/hw/qib/qib_fs.c | 3 +-
drivers/misc/ibmasm/ibmasmfs.c | 2 +-
drivers/misc/ibmvmc.c | 2 +-
drivers/usb/core/devio.c | 16 +--
drivers/usb/gadget/function/f_fs.c | 3 +-
drivers/usb/gadget/legacy/inode.c | 3 +-
fs/9p/vfs_inode.c | 8 +-
fs/9p/vfs_inode_dotl.c | 12 +-
fs/adfs/inode.c | 4 +-
fs/affs/amigaffs.c | 6 +-
fs/affs/inode.c | 16 +--
fs/afs/dynroot.c | 2 +-
fs/afs/inode.c | 8 +-
fs/attr.c | 2 +-
fs/autofs/inode.c | 2 +-
fs/autofs/root.c | 6 +-
fs/bad_inode.c | 6 +-
fs/befs/linuxvfs.c | 2 +-
fs/bfs/dir.c | 16 +--
fs/bfs/inode.c | 5 +-
fs/binfmt_misc.c | 3 +-
fs/btrfs/delayed-inode.c | 8 +-
fs/btrfs/file.c | 37 ++---
fs/btrfs/inode.c | 66 +++------
fs/btrfs/ioctl.c | 2 +-
fs/btrfs/reflink.c | 3 +-
fs/btrfs/super.c | 5 +-
fs/btrfs/transaction.c | 3 +-
fs/btrfs/tree-log.c | 4 +-
fs/btrfs/volumes.c | 4 +-
fs/btrfs/xattr.c | 4 +-
fs/ceph/acl.c | 2 +-
fs/ceph/caps.c | 2 +-
fs/ceph/inode.c | 18 +--
fs/ceph/snap.c | 2 +-
fs/ceph/xattr.c | 2 +-
fs/coda/coda_linux.c | 3 +-
fs/coda/dir.c | 2 +-
fs/coda/file.c | 2 +-
fs/coda/inode.c | 5 +-
fs/configfs/inode.c | 7 +-
fs/cramfs/inode.c | 3 +-
fs/debugfs/inode.c | 3 +-
fs/devpts/inode.c | 6 +-
fs/ecryptfs/inode.c | 7 +-
fs/efivarfs/file.c | 2 +-
fs/efivarfs/inode.c | 2 +-
fs/efs/inode.c | 4 +-
fs/erofs/inode.c | 14 +-
fs/exfat/file.c | 6 +-
fs/exfat/inode.c | 6 +-
fs/exfat/namei.c | 26 ++--
fs/exfat/super.c | 3 +-
fs/ext2/acl.c | 2 +-
fs/ext2/dir.c | 6 +-
fs/ext2/ialloc.c | 2 +-
fs/ext2/inode.c | 12 +-
fs/ext2/ioctl.c | 4 +-
fs/ext2/namei.c | 8 +-
fs/ext2/super.c | 2 +-
fs/ext2/xattr.c | 2 +-
fs/ext4/acl.c | 2 +-
fs/ext4/ext4.h | 90 +++++++------
fs/ext4/extents.c | 12 +-
fs/ext4/ialloc.c | 2 +-
fs/ext4/inline.c | 4 +-
fs/ext4/inode-test.c | 6 +-
fs/ext4/inode.c | 18 ++-
fs/ext4/ioctl.c | 9 +-
fs/ext4/namei.c | 26 ++--
fs/ext4/super.c | 4 +-
fs/ext4/xattr.c | 6 +-
fs/f2fs/dir.c | 8 +-
fs/f2fs/f2fs.h | 4 +-
fs/f2fs/file.c | 22 +--
fs/f2fs/inline.c | 2 +-
fs/f2fs/inode.c | 10 +-
fs/f2fs/namei.c | 12 +-
fs/f2fs/recovery.c | 4 +-
fs/f2fs/super.c | 2 +-
fs/f2fs/xattr.c | 2 +-
fs/fat/fat.h | 3 +-
fs/fat/file.c | 2 +-
fs/fat/inode.c | 5 +-
fs/fat/misc.c | 10 +-
fs/freevxfs/vxfs_inode.c | 3 +-
fs/fuse/control.c | 2 +-
fs/fuse/dir.c | 10 +-
fs/fuse/inode.c | 16 +--
fs/gfs2/acl.c | 2 +-
fs/gfs2/bmap.c | 11 +-
fs/gfs2/dir.c | 15 ++-
fs/gfs2/file.c | 2 +-
fs/gfs2/glops.c | 4 +-
fs/gfs2/inode.c | 16 +--
fs/gfs2/quota.c | 2 +-
fs/gfs2/super.c | 4 +-
fs/gfs2/xattr.c | 8 +-
fs/hfs/catalog.c | 8 +-
fs/hfs/dir.c | 2 +-
fs/hfs/inode.c | 13 +-
fs/hfs/sysdep.c | 4 +-
fs/hfsplus/catalog.c | 8 +-
fs/hfsplus/dir.c | 6 +-
fs/hfsplus/inode.c | 18 +--
fs/hostfs/hostfs_kern.c | 3 +-
fs/hpfs/dir.c | 8 +-
fs/hpfs/inode.c | 6 +-
fs/hpfs/namei.c | 29 ++--
fs/hpfs/super.c | 5 +-
fs/hugetlbfs/inode.c | 12 +-
fs/inode.c | 216 ++++++++++++++++++++++++------
fs/isofs/inode.c | 9 +-
fs/isofs/rock.c | 16 +--
fs/jffs2/dir.c | 24 ++--
fs/jffs2/file.c | 3 +-
fs/jffs2/fs.c | 10 +-
fs/jffs2/os-linux.h | 2 +-
fs/jfs/acl.c | 2 +-
fs/jfs/inode.c | 2 +-
fs/jfs/ioctl.c | 2 +-
fs/jfs/jfs_imap.c | 8 +-
fs/jfs/jfs_inode.c | 4 +-
fs/jfs/namei.c | 24 ++--
fs/jfs/super.c | 2 +-
fs/jfs/xattr.c | 2 +-
fs/kernfs/inode.c | 7 +-
fs/libfs.c | 59 +++++---
fs/minix/bitmap.c | 2 +-
fs/minix/dir.c | 6 +-
fs/minix/inode.c | 12 +-
fs/minix/itree_common.c | 4 +-
fs/minix/namei.c | 6 +-
fs/nfs/callback_proc.c | 2 +-
fs/nfs/fscache.h | 4 +-
fs/nfs/inode.c | 22 +--
fs/nfs/namespace.c | 3 +-
fs/nfsd/nfsctl.c | 2 +-
fs/nfsd/vfs.c | 2 +-
fs/nilfs2/dir.c | 6 +-
fs/nilfs2/inode.c | 12 +-
fs/nilfs2/ioctl.c | 2 +-
fs/nilfs2/namei.c | 8 +-
fs/nsfs.c | 2 +-
fs/ntfs/inode.c | 15 ++-
fs/ntfs/mft.c | 3 +-
fs/ntfs3/file.c | 8 +-
fs/ntfs3/frecord.c | 3 +-
fs/ntfs3/inode.c | 14 +-
fs/ntfs3/namei.c | 11 +-
fs/ntfs3/xattr.c | 4 +-
fs/ocfs2/acl.c | 6 +-
fs/ocfs2/alloc.c | 6 +-
fs/ocfs2/aops.c | 2 +-
fs/ocfs2/dir.c | 8 +-
fs/ocfs2/dlmfs/dlmfs.c | 4 +-
fs/ocfs2/dlmglue.c | 7 +-
fs/ocfs2/file.c | 18 +--
fs/ocfs2/inode.c | 12 +-
fs/ocfs2/move_extents.c | 6 +-
fs/ocfs2/namei.c | 21 +--
fs/ocfs2/refcounttree.c | 14 +-
fs/ocfs2/xattr.c | 6 +-
fs/omfs/dir.c | 4 +-
fs/omfs/inode.c | 9 +-
fs/openpromfs/inode.c | 5 +-
fs/orangefs/inode.c | 7 +-
fs/orangefs/namei.c | 2 +-
fs/orangefs/orangefs-kernel.h | 2 +-
fs/orangefs/orangefs-utils.c | 6 +-
fs/overlayfs/file.c | 7 +-
fs/overlayfs/inode.c | 2 +-
fs/overlayfs/overlayfs.h | 2 +-
fs/overlayfs/util.c | 2 +-
fs/pipe.c | 2 +-
fs/posix_acl.c | 2 +-
fs/proc/base.c | 6 +-
fs/proc/fd.c | 2 +-
fs/proc/generic.c | 2 +-
fs/proc/inode.c | 2 +-
fs/proc/proc_net.c | 2 +-
fs/proc/proc_sysctl.c | 4 +-
fs/proc/root.c | 3 +-
fs/proc/self.c | 2 +-
fs/proc/thread_self.c | 2 +-
fs/pstore/inode.c | 4 +-
fs/qnx4/inode.c | 3 +-
fs/qnx6/inode.c | 3 +-
fs/ramfs/inode.c | 6 +-
fs/reiserfs/inode.c | 12 +-
fs/reiserfs/ioctl.c | 4 +-
fs/reiserfs/namei.c | 18 +--
fs/reiserfs/stree.c | 4 +-
fs/reiserfs/super.c | 2 +-
fs/reiserfs/xattr.c | 5 +-
fs/reiserfs/xattr_acl.c | 2 +-
fs/romfs/super.c | 3 +-
fs/smb/client/file.c | 4 +-
fs/smb/client/fscache.h | 5 +-
fs/smb/client/inode.c | 16 +--
fs/smb/client/smb2ops.c | 3 +-
fs/smb/server/smb2pdu.c | 30 ++---
fs/smb/server/vfs.c | 3 +-
fs/squashfs/inode.c | 2 +-
fs/stack.c | 2 +-
fs/stat.c | 65 +++++++--
fs/sysv/dir.c | 6 +-
fs/sysv/ialloc.c | 2 +-
fs/sysv/inode.c | 5 +-
fs/sysv/itree.c | 7 +-
fs/sysv/namei.c | 6 +-
fs/tracefs/inode.c | 2 +-
fs/ubifs/debug.c | 4 +-
fs/ubifs/dir.c | 41 +++---
fs/ubifs/file.c | 31 ++---
fs/ubifs/ioctl.c | 2 +-
fs/ubifs/journal.c | 4 +-
fs/ubifs/super.c | 4 +-
fs/ubifs/ubifs.h | 2 +-
fs/ubifs/xattr.c | 6 +-
fs/udf/ialloc.c | 2 +-
fs/udf/inode.c | 17 ++-
fs/udf/namei.c | 24 ++--
fs/udf/symlink.c | 2 +-
fs/ufs/dir.c | 6 +-
fs/ufs/ialloc.c | 2 +-
fs/ufs/inode.c | 23 ++--
fs/ufs/namei.c | 8 +-
fs/vboxsf/utils.c | 6 +-
fs/xfs/libxfs/xfs_inode_buf.c | 5 +-
fs/xfs/libxfs/xfs_trans_inode.c | 6 +-
fs/xfs/xfs_acl.c | 2 +-
fs/xfs/xfs_bmap_util.c | 6 +-
fs/xfs/xfs_inode.c | 3 +-
fs/xfs/xfs_inode_item.c | 2 +-
fs/xfs/xfs_iops.c | 25 ++--
fs/xfs/xfs_itable.c | 4 +-
fs/xfs/xfs_super.c | 2 +-
fs/zonefs/super.c | 8 +-
include/linux/fs.h | 100 +++++++++++++-
include/linux/fs_stack.h | 2 +-
ipc/mqueue.c | 23 ++--
kernel/bpf/inode.c | 6 +-
mm/shmem.c | 30 ++---
net/sunrpc/rpc_pipe.c | 2 +-
security/apparmor/apparmorfs.c | 11 +-
security/apparmor/policy_unpack.c | 11 +-
security/inode.c | 2 +-
security/selinux/selinuxfs.c | 2 +-
252 files changed, 1267 insertions(+), 1049 deletions(-)
Powered by blists - more mailing lists