[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <20250822192437.GA458494@ax162>
Date: Fri, 22 Aug 2025 12:24:37 -0700
From: Nathan Chancellor <nathan@...nel.org>
To: Steven Rostedt <rostedt@...dmis.org>
Cc: Linus Torvalds <torvalds@...ux-foundation.org>,
LKML <linux-kernel@...r.kernel.org>,
Masami Hiramatsu <mhiramat@...nel.org>,
Mathieu Desnoyers <mathieu.desnoyers@...icios.com>,
Mark Rutland <mark.rutland@....com>,
Andrew Morton <akpm@...ux-foundation.org>,
Liao Yuanhong <liaoyuanhong@...o.com>,
Pu Lehui <pulehui@...wei.com>, Tao Chen <chen.dylane@...ux.dev>,
Tengda Wu <wutengda@...weicloud.com>,
Ye Weihua <yeweihua4@...wei.com>
Subject: Re: [GIT PULL] tracing: Fixes for v6.17
Hi Steve,
On Fri, Aug 22, 2025 at 12:49:33PM -0400, Steven Rostedt wrote:
> - Allocate and copy ftrace hash for reader of ftrace filter files
>
> When the set_ftrace_filter or set_ftrace_notrace files are open for read,
> an iterator is created and sets its hash pointer to the associated hash that
> represents filtering or notrace filtering to it. The issue is that the hash
> it points to can change while the iteration is happening. All the locking
> used to access the tracer's hashes are released which means those hashes can
> change or even be freed. Using the hash pointed to by the iterator can cause
> UAF bugs or similar.
>
> Have the read of these files allocate and copy the corresponding hashes and
> use that as that will keep them the same while the iterator is open. This
> also simplifies the code as opening it for write already does an allocate
> and copy, and now that the read is doing the same, there's no need to check
> which way it was opened on the release of the file, and the iterator hash
> can always be freed.
...
> Steven Rostedt (2):
> ftrace: Also allocate and copy hash for reading of filter files
I just bisected a crash that I see when running LTP's read_all test
(which I have statically compiled at [1]) on /sys:
# bad: [0f4c93f7eb861acab537dbe94441817a270537bf] Add linux-next specific files for 20250822
# good: [3957a5720157264dcc41415fbec7c51c4000fc2d] Merge tag 'cgroup-for-6.17-rc2-fixes' of git://git.kernel.org/pub/scm/linux/kernel/git/tj/cgroup
git bisect start '0f4c93f7eb861acab537dbe94441817a270537bf' '3957a5720157264dcc41415fbec7c51c4000fc2d'
# bad: [1eca822fd0fc88c51825a929dee4a82aa37de102] Merge branch 'cpufreq/arm/linux-next' of https://git.kernel.org/pub/scm/linux/kernel/git/vireshk/pm.git
git bisect bad 1eca822fd0fc88c51825a929dee4a82aa37de102
# bad: [6fdae20d32f045dad3f9d89a7bc53a201ae6061c] Merge branch 'for-next' of https://git.kernel.org/pub/scm/linux/kernel/git/s390/linux.git
git bisect bad 6fdae20d32f045dad3f9d89a7bc53a201ae6061c
# bad: [b16cd43ecfee91682ed0f6c7e6686252812a1d53] Merge branch 'mm-unstable' of https://git.kernel.org/pub/scm/linux/kernel/git/akpm/mm
git bisect bad b16cd43ecfee91682ed0f6c7e6686252812a1d53
# good: [c1a5408bb0df483c9a6e1b0bb585aa120304b869] Merge branch 'i2c/i2c-host-fixes' of https://git.kernel.org/pub/scm/linux/kernel/git/andi.shyti/linux.git
git bisect good c1a5408bb0df483c9a6e1b0bb585aa120304b869
# good: [ad0cb3a198c18e9ea623415998371967625b7a6f] mm/huge_memory: convert "tva_flags" to "enum tva_type"
git bisect good ad0cb3a198c18e9ea623415998371967625b7a6f
# bad: [219d594f4ae85b505c8900149eeae48de58714ef] Merge branch 'for-linux-next-fixes' of https://gitlab.freedesktop.org/drm/misc/kernel.git
git bisect bad 219d594f4ae85b505c8900149eeae48de58714ef
# bad: [117c87380b3a3f9fbc925d39f20fec65cfc998f7] Merge branch 'msm-fixes' of https://gitlab.freedesktop.org/drm/msm.git
git bisect bad 117c87380b3a3f9fbc925d39f20fec65cfc998f7
# good: [553666f839b86545300773954df7426a45c169c4] drm/msm/kms: move snapshot init earlier in KMS init
git bisect good 553666f839b86545300773954df7426a45c169c4
# bad: [d1bd269dc6608aef35e150ec60644545f2084584] Merge branch 'trace/fixes' of https://git.kernel.org/pub/scm/linux/kernel/git/trace/linux-trace.git
git bisect bad d1bd269dc6608aef35e150ec60644545f2084584
# good: [cd6e4faba96fe41d6b686e144b96dad5e6f2e771] ring-buffer: Remove redundant semicolons
git bisect good cd6e4faba96fe41d6b686e144b96dad5e6f2e771
# bad: [48d06e78b7cba941e991da71ca351f5104ea927e] ftrace: Also allocate and copy hash for reading of filter files
git bisect bad 48d06e78b7cba941e991da71ca351f5104ea927e
# good: [edede7a6dcd7435395cf757d053974aaab6ab1c2] trace/fgraph: Fix the warning caused by missing unregister notifier
git bisect good edede7a6dcd7435395cf757d053974aaab6ab1c2
# first bad commit: [48d06e78b7cba941e991da71ca351f5104ea927e] ftrace: Also allocate and copy hash for reading of filter files
$ sudo ./read_all -d /sys
tst_test.c:1459: TINFO: Timeout per run is 0h 05m 00s
read_all.c:216: TINFO: read(/sys/kernel/mm/page_idle/bitmap): EINVAL (22)
read_all.c:204: TINFO: open(/sys/kernel/mm/hugepages/hugepages-1048576kB/demote): EACCES (13)
read_all.c:204: TINFO: open(/sys/kernel/tracing/osnoise/per_cpu/cpu7/timerlat_fd): EINVAL (22)
read_all.c:204: TINFO: open(/sys/kernel/tracing/osnoise/per_cpu/cpu6/timerlat_fd): EINVAL (22)
read_all.c:204: TINFO: open(/sys/kernel/tracing/osnoise/per_cpu/cpu5/timerlat_fd): EINVAL (22)
read_all.c:204: TINFO: open(/sys/kernel/tracing/osnoise/per_cpu/cpu4/timerlat_fd): EINVAL (22)
read_all.c:204: TINFO: open(/sys/kernel/tracing/osnoise/per_cpu/cpu3/timerlat_fd): EINVAL (22)
read_all.c:204: TINFO: open(/sys/kernel/tracing/osnoise/per_cpu/cpu2/timerlat_fd): EINVAL (22)
read_all.c:204: TINFO: open(/sys/kernel/tracing/osnoise/per_cpu/cpu1/timerlat_fd): EINVAL (22)
read_all.c:204: TINFO: open(/sys/kernel/tracing/osnoise/per_cpu/cpu0/timerlat_fd): EINVAL (22)
^CSending SIGKILL to test process...
tst_test.c:1503: TINFO: Killed the leftover descendant processes
tst_test.c:1509: TINFO: If you are running on slow machine, try exporting LTP_TIMEOUT_MUL > 1
tst_test.c:1511: TBROK: Test killed! (timeout?)
Summary:
passed 0
failed 0
broken 1
skipped 0
warnings 0
$ dmesg
[ 62.221518] BUG: kernel NULL pointer dereference, address: 0000000000000000
[ 62.222457] #PF: supervisor read access in kernel mode
[ 62.223068] #PF: error_code(0x0000) - not-present page
[ 62.223720] PGD 1076a2067 P4D 10fe33067 PUD 112688067 PMD 0
[ 62.224436] Oops: Oops: 0000 [#1] SMP NOPTI
[ 62.224939] CPU: 4 UID: 0 PID: 1145 Comm: read_all Not tainted 6.17.0-rc2-00006-g48d06e78b7cb #1 PREEMPT(full) ab6dff6fe4772c3d341055188b1594d9637c1b0d
[ 62.226579] Hardware name: QEMU Standard PC (Q35 + ICH9, 2009), BIOS unknown 02/02/2022
[ 62.227561] RIP: 0010:ftrace_regex_open+0x153/0x280
[ 62.228177] Code: 48 89 04 24 e8 4e af ff ff 48 8b 04 24 48 89 c7 48 8b 00 49 39 fe 75 e8 48 c7 c7 80 b6 55 ba e8 93 7e 10 01 48 8b 45 50 eb 0b <8b> 3e e8 d6 bc ff ff 48 89 45 50 48 85 c0 0f 84 fd 00 00 00 41 f6
[ 62.230434] RSP: 0018:ff4bded7c4e5bba0 EFLAGS: 00010246
[ 62.231052] RAX: 0000000000000000 RBX: ffffffffba728660 RCX: 0000000000000000
[ 62.231983] RDX: ff172e52cc1b2180 RSI: 0000000000000000 RDI: ffffffffba728698
[ 62.232852] RBP: ff172e52c44f3500 R08: ff172e52c3db6c00 R09: ff172e52c3db6c00
[ 62.233725] R10: ff4bded7c4e5bb88 R11: 00000000ffffffff R12: 0000000000000000
[ 62.234594] R13: 0000000000000000 R14: 0000000000000000 R15: ff172e52d45d1240
[ 62.235465] FS: 0000000000449778(0000) GS:ff172e5674a92000(0000) knlGS:0000000000000000
[ 62.236433] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 62.237110] CR2: 0000000000000000 CR3: 00000001144fd006 CR4: 0000000000771ef0
[ 62.237968] PKRU: 55555554
[ 62.238321] Call Trace:
[ 62.238632] <TASK>
[ 62.238898] ? __pfx_stack_trace_filter_open+0x10/0x10
[ 62.239565] do_dentry_open+0x23d/0x480
[ 62.240035] vfs_open+0x30/0x100
[ 62.240448] path_openat+0x7ea/0x12e0
[ 62.240900] ? srso_alias_return_thunk+0x5/0xfbef5
[ 62.241511] ? __memcg_slab_free_hook+0xf4/0x140
[ 62.242065] do_filp_open+0xd8/0x180
[ 62.242518] ? alloc_fd+0x12e/0x190
[ 62.242944] do_sys_openat2+0x88/0xe0
[ 62.243409] __x64_sys_open+0x5f/0xa0
[ 62.243852] do_syscall_64+0x81/0x970
[ 62.244331] ? srso_alias_return_thunk+0x5/0xfbef5
[ 62.244899] ? do_syscall_64+0x81/0x970
[ 62.245398] ? srso_alias_return_thunk+0x5/0xfbef5
[ 62.245972] ? __x64_sys_open+0x5f/0xa0
[ 62.246462] ? srso_alias_return_thunk+0x5/0xfbef5
[ 62.247036] ? do_syscall_64+0x81/0x970
[ 62.247528] ? srso_alias_return_thunk+0x5/0xfbef5
[ 62.248093] ? __irq_exit_rcu+0x4c/0xf0
[ 62.248588] entry_SYSCALL_64_after_hwframe+0x76/0x7e
[ 62.249196] RIP: 0033:0x4243b8
[ 62.249590] Code: 0f 05 48 83 f8 da 75 08 4c 89 c0 48 89 d6 0f 05 c3 48 89 f8 4d 89 c2 48 89 f7 4d 89 c8 48 89 d6 4c 8b 4c 24 08 48 89 ca 0f 05 <c3> e9 e1 ff ff ff 48 8d 3d 9b 52 02 00 e9 8a 06 00 00 48 8d 3d 8f
[ 62.251882] RSP: 002b:00007ffedeaeff18 EFLAGS: 00000286 ORIG_RAX: 0000000000000002
[ 62.252805] RAX: ffffffffffffffda RBX: 0000000000000800 RCX: 00000000004243b8
[ 62.253687] RDX: 0000000000000000 RSI: 0000000000008800 RDI: 00007ffedeaf0040
[ 62.254558] RBP: 000000002150ffc1 R08: 0000000000000000 R09: 0000000000000000
[ 62.255421] R10: 0000000000000000 R11: 0000000000000286 R12: 00007febbedb9000
[ 62.256280] R13: 000000000042b00c R14: 00007ffedeaf0040 R15: 000000000043f130
[ 62.257129] </TASK>
[ 62.257428] Modules linked in:
[ 62.257808] CR2: 0000000000000000
[ 62.258213] ---[ end trace 0000000000000000 ]---
[ 62.258795] RIP: 0010:ftrace_regex_open+0x153/0x280
[ 62.259400] Code: 48 89 04 24 e8 4e af ff ff 48 8b 04 24 48 89 c7 48 8b 00 49 39 fe 75 e8 48 c7 c7 80 b6 55 ba e8 93 7e 10 01 48 8b 45 50 eb 0b <8b> 3e e8 d6 bc ff ff 48 89 45 50 48 85 c0 0f 84 fd 00 00 00 41 f6
[ 62.261614] RSP: 0018:ff4bded7c4e5bba0 EFLAGS: 00010246
[ 62.262231] RAX: 0000000000000000 RBX: ffffffffba728660 RCX: 0000000000000000
[ 62.263084] RDX: ff172e52cc1b2180 RSI: 0000000000000000 RDI: ffffffffba728698
[ 62.263938] RBP: ff172e52c44f3500 R08: ff172e52c3db6c00 R09: ff172e52c3db6c00
[ 62.264796] R10: ff4bded7c4e5bb88 R11: 00000000ffffffff R12: 0000000000000000
[ 62.265659] R13: 0000000000000000 R14: 0000000000000000 R15: ff172e52d45d1240
[ 62.266526] FS: 0000000000449778(0000) GS:ff172e5674a92000(0000) knlGS:0000000000000000
[ 62.267488] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 62.268171] CR2: 0000000000000000 CR3: 00000001144fd006 CR4: 0000000000771ef0
[ 62.269026] PKRU: 55555554
[ 62.269386] note: read_all[1145] exited with irqs disabled
[1]: https://github.com/nathanchance/env/raw/a98b8aa3a7017f6b1d94ee26dd217a968da81dd1/bin/x86_64/read_all
If there is any other information I can provide or patches I can test, I
am happy to do so.
Cheers,
Nathan
Powered by blists - more mailing lists