Message-ID: <CAMw=ZnRtmhi8aaO+xsT=kgXYhB8u3sgBdtevrxDWctTLteWYoA@mail.gmail.com>
Date: Tue, 19 Aug 2025 23:38:11 +0100
From: Luca Boccassi <bluca@...ian.org>
To: Christian Heusel <christian@...sel.eu>
Cc: Zhang Yi <yi.zhang@...wei.com>, Sasha Levin <sashal@...nel.org>, 
	"Theodore Ts'o" <tytso@....edu>, linux-ext4@...r.kernel.org, linux-kernel@...r.kernel.org, 
	regressions@...ts.linux.dev, stable@...r.kernel.org, heftig@...hlinux.org
Subject: Re: [REGRESSION][STABLE] ext4: too many credits wanted / file system
 issue in v6.16.1

On Tue, 19 Aug 2025 at 21:53, Christian Heusel <christian@...sel.eu> wrote:
>
> Hello everyone,
>
> the systemd CI has [recently noticed][0] an issue within the ext4 file
> system after the Arch Linux kernel was upgraded to 6.16.1. The issue is
> exclusive to the stable tree and does not occur on 6.16 and not on
> 6.17-rc2. I have also tested 6.16.2-rc1 and it still contains the bug.
>
> I was able to bisect the issue between 6.16 and 6.16.1 to the following
> commit:
>
>     b9c561f3f29c2 ("ext4: fix insufficient credits calculation in ext4_meta_trans_blocks()")
>
> The issue can be reproduced by running the tests from
> [TEST-58-REPART.sh][1] by running the [systemd integration tests][2].
> But if there are any suggestions I can also test myself as the initial
> setup for the integration tests is a bit involved.
>
> It is not yet clear to me whether this has real-world impact besides the
> test, but the systemd devs said that it's not a particularily demanding
> workflow, so I guess it is expected to work and could cause issues on
> other systems too.
>
> Also does anybody have an idea which backport could be missing?
>
> Cheers,
> Chris
>
> [0]: https://github.com/systemd/systemd/actions/runs/17053272497/job/48345703316#step:14:233
> [1]: https://github.com/systemd/systemd/blob/main/test/units/TEST-58-REPART.sh
> [2]: https://github.com/systemd/systemd/blob/main/test/integration-tests/README.md
>
> ---
>
> #regzbot introduced: b9c561f3f29c2
> #regzbot title: [STABLE] ext4: too many credits wanted / file system issue in v6.16.1
> #regzbot link: https://github.com/systemd/systemd/actions/runs/17053272497/job/48345703316#step:14:233
>
> ---
>
> git bisect start
> # status: waiting for both good and bad commits
> # good: [038d61fd642278bab63ee8ef722c50d10ab01e8f] Linux 6.16
> git bisect good 038d61fd642278bab63ee8ef722c50d10ab01e8f
> # status: waiting for bad commit, 1 good commit known
> # bad: [3e0969c9a8c57ff3c6139c084673ebedfc1cf14f] Linux 6.16.1
> git bisect bad 3e0969c9a8c57ff3c6139c084673ebedfc1cf14f
> # good: [288f1562e3f6af6d9b461eba49e75c84afa1b92c] media: v4l2-ctrls: Fix H264 SEPARATE_COLOUR_PLANE check
> git bisect good 288f1562e3f6af6d9b461eba49e75c84afa1b92c
> # bad: [f427460a1586c2e0865f9326b71ed6e5a0f404f2] f2fs: turn off one_time when forcibly set to foreground GC
> git bisect bad f427460a1586c2e0865f9326b71ed6e5a0f404f2
> # bad: [5f57327f41a5bbb85ea382bc389126dd7b8f2d7b] scsi: elx: efct: Fix dma_unmap_sg() nents value
> git bisect bad 5f57327f41a5bbb85ea382bc389126dd7b8f2d7b
> # good: [9143c604415328d5dcd4d37b8adab8417afcdd21] leds: pca955x: Avoid potential overflow when filling default_label (take 2)
> git bisect good 9143c604415328d5dcd4d37b8adab8417afcdd21
> # good: [9c4f20b7ac700e4b4377f85e36165a4f6ca85995] RDMA/hns: Fix accessing uninitialized resources
> git bisect good 9c4f20b7ac700e4b4377f85e36165a4f6ca85995
> # good: [0b21d1962bec2e660c22c4c4231430f97163dcf8] perf tests bp_account: Fix leaked file descriptor
> git bisect good 0b21d1962bec2e660c22c4c4231430f97163dcf8
> # good: [3dbe96d5481acd40d6090f174d2be8433d88716d] clk: thead: th1520-ap: Correctly refer the parent of osc_12m
> git bisect good 3dbe96d5481acd40d6090f174d2be8433d88716d
> # bad: [c6714f30ef88096a8da9fcafb6034dc4e9aa467d] clk: sunxi-ng: v3s: Fix de clock definition
> git bisect bad c6714f30ef88096a8da9fcafb6034dc4e9aa467d
> # bad: [b9c561f3f29c2d6e1c1d3ffc202910bef250b7d8] ext4: fix insufficient credits calculation in ext4_meta_trans_blocks()
> git bisect bad b9c561f3f29c2d6e1c1d3ffc202910bef250b7d8
> # first bad commit: [b9c561f3f29c2d6e1c1d3ffc202910bef250b7d8] ext4: fix insufficient credits calculation in ext4_meta_trans_blocks()

The full kernel warning (immediately after the ext4 fs stops working):

[ 156.796652] JBD2: (sd-copy) wants too many credits credits:554
rsv_credits:0 max:338
[ 156.802468] ------------[ cut here ]------------
[ 156.802929] WARNING: CPU: 1 PID: 2904 at fs/jbd2/transaction.c:334
start_this_handle.cold+0x21/0x2e
[ 156.803222] Modules linked in: dm_crypt encrypted_keys trusted
asn1_encoder tee joydev mousedev iTCO_wdt intel_pmc_bxt
iTCO_vendor_support kvm_amd i2c_i801 psmouse pcspkr vfat fat intel_agp
mac_hid serio_raw lpc_ich intel_rapl_msr cfg80211 rfkill tun nfnetlink
ip_tables intel_rapl_common ccp kvm irqbypass polyval_clmulni
ghash_clmulni_intel sha512_ssse3 sha1_ssse3 aesni_intel virtio_balloon
virtio_scsi i2c_smbus i2c_mux intel_gtt loop dm_mod qemu_fw_cfg
vmw_vsock_virtio_transport vmw_vsock_virtio_transport_common vsock
virtio_rng x_tables
[ 156.804702] CPU: 1 UID: 0 PID: 2904 Comm: (sd-copy) Not tainted
6.16.1-arch1-1 #1 PREEMPT(full)
83823f140bb4fc8c507f38d1610ad9b642cd4b9a
[ 156.805292] H[ 156.792161] TEST-70-TPM2.sh[2894]: Successfully
forked off '(sd-copy)' as PID 2904.ardware name: QEMU Standard PC (Q35
+ ICH9, 2009), BIOS unknown 02/02/2022
[ 156.806972] RIP: 0010:start_this_handle.cold+0x21/0x2e
[ 156.807210] Code: 4c 8b 14 24 e9 46 d7 64 00 65 48 8b 05 63 40 0d 03
44 89 f1 44 89 fa 48 c7 c7 30 84 3e a2 48 8d b0 f0 0c 00 00 e8 32 8b
fe ff <0f> 0b 41 b9 e4 ff ff ff e9 dc d9 64 00 49 8b 54 24 18 49 8b 74
24
[ 156.807896] RSP: 0018:ffffd110c351f990 EFLAGS: 00010246
[ 156.808182] RAX: 0000000000000048 RBX: ffff8a8ed19b3000 RCX: 0000000000000000
[ 156.808367] RDX: 0000000000000000 RSI: 0000000000000001 RDI: ffff8a8f3bd1cfc0
[ 156.808483] RBP: 0000000000000000 R08: 0000000000000000 R09: 00000000fffff35f
[ 156.808591] R10: ffffffffa2e60de0 R11: ffffd110c351f830 R12: ffff8a8ec0775508
[ 156.808761] R13: 0000000000000005 R14: 0000000000000000 R15: 000000000000022a
[ 156.808977] FS: 00007fa54aec3440(0000) GS:ffff8a8f9821b000(0000)
knlGS:0000000000000000
[ 156.810623] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 156.811062] CR2: 00005624ae24ce30 CR3: 00000001071d4002 CR4: 0000000000370ef0
[ 156.811288] Call Trace:
[ 156.811402] <TASK>
[ 156.811460] ? kmem_cache_alloc_noprof+0x12d/0x410
[ 156.811989] ? jbd2__journal_start+0x82/0x210
[ 156.812501] jbd2__journal_start+0xfc/0x210
[ 156.812507] ext4_truncate+0x168/0x440
[ 156.812512] ? unmap_mapping_range+0x80/0x130
[ 156.812517] ext4_setattr+0x706/0xa90
[ 156.812521] notify_change+0x340/0x4e0
[ 156.812526] ? do_truncate+0xc2/0xf0
[ 156.812529] do_truncate+0xc2/0xf0
[ 156.812533] [ 156.803411] TEST-70-TPM2.sh[2904]: Mounting
/dev/mapper/luks-repart-2d18cd9c567b253b (ext4) on
/run/systemd/mount-root (MS_NOSUID|MS_NODEV|MS_NOEXEC|MS_NOATIME
"")...do_ftruncate+0x12d/0x1f0
[ 156.812536]
do_sys_ftruncate+0x40/0x80
[ 156.812539] __x64_sys_ftruncate+0x1a/0x30
[ 156.812541] do_syscall_64+0x81/0x970
[ 156.812546] ? shmem_file_llseek+0x6b/0xc0
[ 156.812551] ? syscall_exit_work+0x143/0x1b0
[ 156.812555] ? do_syscall_64+0x22f/0x970
[ 156.812557] ? syscall_exit_work+0x143/0x1b0
[ 156.812559] ? do_syscall_64+0x22f/0x970
[ 156.812563] ? syscall_exit_work+0x143/0x1b0
[ 156.812566] ? do_syscall_64+0x22f/0x970
[ 156.812568] ? shmem_file_llseek+0x6b/0xc0
[ 156.812571] ? syscall_exit_work+0x143/0x1b0
[ 156.812573] ? do_syscall_64+0x22f/0x970
[ 156.812575] ? do_syscall_64+0x22f/0x970
[ 156.812578] ? syscall_exit_work+0x143/0x1b0
[ 156.812580] ? do_syscall_64+0x22f/0x970
[ 156.812582] ? exc_page_fault+0x7e/0x1a0
[ 156.812585] entry_SYSCALL_64_after_hwframe+0x76/0x7e
[ 156.812588] RIP: 0033:0x7fa54a110ceb
[ 156.812595] Code: d2 31 f6 48 89 e5 48 83 ec 08 6a 4a e8 0e 25 f8 ff
c9 c3 66 2e 0f 1f 84 00 00 00 00 00 66 90 f3 0f 1e fa b8 4d 00 00 00
0f 05 <48> 3d 00 f0 ff ff 77 05 c3 0f 1f 40 00 48 8b 15 f1 7f 0f 00 f7
d8
[ 156.812597] RSP: 002b:00007ffd70e12d48 EFLAGS: 00000206 ORIG_RAX:
000000000000004d
[ 156.812599] RAX: ffffffffffffffda RBX: ffffffffffffffff RCX: 00007fa54a110ceb
[ 156.812601] RDX: 0000000000000001 RSI: 0000000000000048 RDI: 0000000000000012
[ 156.812602] RBP: 00007ffd70e12e00 R08: 0000000000000000 R09: 00007fa54aec32d0
[ 156.812603] R10: 0000000000000008 R11: 0000000000000206 R12: 0000000000000013
[ 156.812604] R13: 7fffffffffffffb7 R14: 0000000000000048 R15: 0000000000010000
[ 156.812609] </TASK>
[ 156.819965] ---[ end trace 0000000000000000 ]---
[ 156.819988] EXT4-fs error (device dm-0) in ext4_setattr:5987: error 28

Powered by blists - more mailing lists