| lists.openwall.net | lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC | |
|
Open Source and information security mailing list archives
| ||
|
Date: Wed, 20 Jul 2022 18:22:25 +0300
From: Alexey Dobriyan <adobriyan@...il.com>
To: rostedt@...dmis.org, mingo@...hat.com
Cc: linux-kernel@...r.kernel.org
Subject: ftrace_kill() leads to kmalloc-512 UAF
I'm debugging crash of our product which does live kernel patching together
with ISV security scanner which uses ftrace kprobes to do whatever it does.
What happens is that is ftrace ever detects .text change, refuses to patch
and prints a warning with FTRACE_WARN_ON_ONCE() then there is reliable way
to cause UAF on kmalloc-512 cache by trying to register kprobe with
perf_event_open() and then unregistering it by exiting the process.
1) live kernel patching happens, first instruction of some function changes
2) kprobe on that function is registered with perf_event_open()
WARNING: CPU: 5 PID: 2109 at kernel/trace/ftrace.c:1853 ftrace_bug+0x25d/0x270
[<ffffffff811638ed>] ftrace_bug+0x25d/0x270
[<ffffffff81065571>] ftrace_replace_code+0x2b1/0x420
[<ffffffff81163f9a>] ftrace_modify_all_code+0x6a/0xb0
[<ffffffff810656f0>] arch_ftrace_update_code+0x10/0x20
[<ffffffff81164077>] ftrace_run_update_code+0x17/0x70
[<ffffffff81165512>] ftrace_set_hash+0x1c2/0x1f0
[<ffffffff8126fee0>] ? SyS_dup2+0x60/0x60
[<ffffffff8126fee0>] ? SyS_dup2+0x60/0x60
[<ffffffff811655a0>] ftrace_set_filter_ip+0x60/0x70
[<ffffffff8179624c>] arm_kprobe+0x9c/0x140
[<ffffffff81796368>] enable_kprobe+0x78/0xa0
[<ffffffff81187bab>] enable_trace_kprobe+0x7b/0x120
[<ffffffff81797e5f>] kprobe_register+0x2f/0x60
[<ffffffff8118348a>] perf_trace_event_init+0x1aa/0x230
[<ffffffff811836b7>] perf_kprobe_init+0xa7/0xf0
[<ffffffff811a8919>] perf_kprobe_event_init+0x49/0x70
[<ffffffff811aa569>] perf_try_init_event+0x99/0xc0
[<ffffffff811b29f2>] perf_init_event+0x92/0x150
[<ffffffff811b2fa1>] perf_event_alloc+0x4f1/0x910
[<ffffffff811b3789>] SYSC_perf_event_open+0x3c9/0xe50
[<ffffffff811b4679>] SyS_perf_event_open+0x9/0x10
[<ffffffff81799f92>] system_call_fastpath+0x25/0x2a
ftrace failed to modify [<ffffffff8126fee0>] SyS_dup+0x0/0x120
actual: e9:4b:50:2e:3f
3) FTRACE_WARN_ON_ONCE() calls ftrace_kill() which sets ftrace_disabled:
ftrace_disabled = 1;
4) process exits, all kprobes are unregistered but "ftrace_disabled" is 1 now
so disarming fails with -ENODEV:
[ 253.042821] WARNING: CPU: 2 PID: 1682 at kernel/kprobes.c:1006 disarm_kprobe+0x102/0x180
[ 253.044424] Failed to disarm kprobe-ftrace at do_exit+0x0/0xa30 (-19)
[ 253.086658] [<ffffffff811655a0>] ? ftrace_set_filter_ip+0x60/0x70
[ 253.089599] [<ffffffff810a1810>] ? mm_update_next_owner+0x230/0x230
[ 253.090878] [<ffffffff81796492>] disarm_kprobe+0x102/0x180
[ 253.092084] [<ffffffff817965dd>] __disable_kprobe+0xcd/0xf0
[ 253.093242] [<ffffffff81796c13>] disable_kprobe+0x23/0x40
[ 253.094390] [<ffffffff81187d14>] disable_trace_kprobe+0xc4/0x100
[ 253.095751] [<ffffffff81797e53>] kprobe_register+0x23/0x60
[ 253.097420] [<ffffffff8118327c>] perf_trace_event_unreg.isra.1+0x3c/0xa0
[ 253.098866] [<ffffffff81183730>] perf_kprobe_destroy+0x30/0x40
[ 253.100085] [<ffffffff811af50a>] _free_event+0xfa/0x2f0
[ 253.101147] [<ffffffff811af769>] put_event+0x19/0x20
[ 253.102154] [<ffffffff811af993>] perf_event_release_kernel+0x223/0x300
[ 253.103457] [<ffffffff811afa80>] perf_release+0x10/0x20
[ 253.106188] [<ffffffff8125063c>] __fput+0xec/0x230
and more importantly underlying kprobe is not removed from "kprobe_table" hashtable:
unregister_kprobe
unregister_kprobes
__unregister_kprobe_top
__disable_kprobe
ret = disarm_kprobe(orig_p, true);
disarm_kprobe_ftrace
__disarm_kprobe_ftrace
ftrace_set_filter_ip
ftrace_set_addr
ftrace_set_hash
if (unlikely(ftrace_disabled))
return -ENODEV;
// BOOM, function does nothing!!!
if (ret) {
p->flags &= ~KPROBE_FLAG_DISABLED;
return ERR_PTR(ret);
}
perf_kprobe_destroy() will free the containing kprobe with inner kprobe
still in hashtable manifesting the bug as regular oopses, mystical oopses
in unrelated processes and doublefaults even.
void perf_kprobe_destroy(struct perf_event *p_event)
{
perf_trace_event_close(p_event);
// does more or less nothing
perf_trace_event_unreg(p_event);
// does kfree
destroy_local_trace_kprobe(p_event->tp_event);
}
crash> p kprobe_table
kprobe_table = $1 =
{{
}, {
first = 0xffff880135e9ddd8
}, {
...
crash> struct kprobe 0xffff880135e9ddd8
struct kprobe {
hlist = {
next = 0x6b6b6b6b6b6b6b6b,
pprev = 0x6b6b6b6b6b6b6b6b
},
list = {
next = 0x6b6b6b6b6b6b6b6b,
prev = 0x6b6b6b6b6b6b6b6b
},
nmissed = 7740398493674204011,
addr = 0x6b6b6b6b6b6b6b6b <Address 0x6b6b6b6b6b6b6b6b out of bounds>,
....
This is much easier to reproduce by adding the following BUG_ON:
--- a/kernel/trace/trace_kprobe.c.orig 2022-07-20 14:35:06.760511285 +0300
+++ b/kernel/trace/trace_kprobe.c 2022-07-20 14:41:14.699877744 +0300
@@ -512,8 +512,10 @@ static void __unregister_trace_kprobe(st
if (trace_probe_is_registered(&tk->tp)) {
if (trace_kprobe_is_return(tk))
unregister_kretprobe(&tk->rp);
- else
+ else {
unregister_kprobe(&tk->rp.kp);
+ BUG_ON(!hlist_unhashed(&tk->rp.kp.hlist));
+ }
tk->tp.flags &= ~TP_FLAG_REGISTERED;
/* Cleanup kprobe for reuse */
if (tk->rp.kp.symbol_name)
Basically, if ftrace_kill() is ever called ever there is a ticking UAFbomb.
Alexey (CloudLinux)
Powered by blists - more mailing lists