linux-kernel - Re: [PATCH] kprobes, x86/alternatives: use text_mutex to protect smp_alt

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <59F31BB5.90905@huawei.com>
Date:   Fri, 27 Oct 2017 19:42:45 +0800
From:   zhouchengming <zhouchengming1@...wei.com>
To:     Borislav Petkov <bp@...e.de>
CC:     <mhiramat@...nel.org>, <ananth@...ux.vnet.ibm.com>,
        <anil.s.keshavamurthy@...el.com>, <davem@...emloft.net>,
        <hpa@...or.com>, <tglx@...utronix.de>, <peterz@...radead.org>,
        <jkosina@...e.cz>, <rostedt@...dmis.org>, <mjurczyk@...gle.com>,
        <x86@...nel.org>, <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] kprobes, x86/alternatives: use text_mutex to protect
 smp_alt_modules

On 2017/10/27 19:15, Borislav Petkov wrote:
> On Fri, Oct 27, 2017 at 05:34:44PM +0800, Zhou Chengming wrote:
>> Fixes: 2cfa197 "ftrace/alternatives: Introducing *_text_reserved
>> functions"
>>
>> We use alternatives_text_reserved() to check if the address is in
>> the fixed pieces of alternative reserved, but the problem is that
>> we don't hold the smp_alt mutex when call this function. So the list
>> traversal may encounter a deleted list_head if another path is doing
>> alternatives_smp_module_del().
> Is this something you've triggered on a real machine or is this just
> from code staring?
>

Hi, thanks for your reply.
This is a real bug happened on one of our machines, below is the calltrace.
We can see the trigger is at alternatives_text_reserved+0x20/0x80, and
encounter a deleted (poisoned) list_head.

[   14.016190] general protection fault: 0000 [#1] SMP
[   14.016988] CPU 0
[   14.017287] Modules linked in: mlx4_ib(O+) mlx4_en(O+) ib_sa(O) ib_mad(O) ib_core(O) ib_addr(O) ipv6 mlx4_core(O) compat(O) igb(O) rtc_cmos dca button ata_piix ahci libahci libata ext3 jbd mbcache usbhid hid uhci_hcd ehci_hcd processor thermal_sys hwmon usbcore usb_common sd_mod crc_t10dif virtio_console(O) virtio_pci(O) virtio_net(O) kvm_ivshmem(O) virtio_scsi(O) scsi_mod virtio_blk(O) virtio(O) virtio_ring(O) pv_channel(O)
[   14.020005]
[   14.020005] Pid: 1979, comm: modprobe Tainted: G           O 3.4.24.19-0.11-default #1 QEMU Standard PC (i440FX + PIIX, 1996)
[   14.020005] RIP: 0010:[<ffffffff81007eb0>]  [<ffffffff81007eb0>] alternatives_text_reserved+0x20/0x80
[   14.020005] RSP: 0018:ffff880ea355bcb8  EFLAGS: 00010283
[   14.020005] RAX: dead000000000100 RBX: ffffffffa02af720 RCX: dead0000000000d0
[   14.020005] RDX: ffffffffa02f0588 RSI: ffffffffa02d2fc0 RDI: ffffffffa02d2fc0
[   14.020005] RBP: ffff880ea355bcb8 R08: ffffffffa02f3b68 R09: 00017f4ae12d2fc0
[   14.020005] R10: 00000000000000e8 R11: ffffffffa02bb9d7 R12: 0000000000000000
[   14.020005] R13: ffffffffa02af720 R14: ffffffffa0307140 R15: ffffffffa02af730
[   14.020005] FS:  00007f26c6acc700(0000) GS:ffff880fff200000(0000) knlGS:0000000000000000
[   14.020005] CS:  0010 DS: 0000 ES: 0000 CR0: 000000008005003b
[   14.020005] CR2: 00007fd4adc3b000 CR3: 0000000ea40ea000 CR4: 00000000001407f0
[   14.020005] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[   14.020005] DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
[   14.020005] Process modprobe (pid: 1979, threadinfo ffff880ea355a000, task ffff880e9eb9e600)
[   14.020005] Stack:
[   14.020005]  ffff880ea355bcd8 ffffffff8145b148 ffffffffa02af720 ffffffffa02af720
[   14.020005]  ffff880ea355bd18 ffffffff8145ed38 0000000000000000 0000000000000000
[   14.041015]  ffff880ea355bd90 ffffffffa02af720 0000000000000001 ffff880ea355bd90
[   14.041015] Call Trace:
[   14.041015]  [<ffffffff8145b148>] arch_prepare_kprobe+0x18/0x80
[   14.042982]  [<ffffffff8145ed38>] register_kprobe+0x338/0x4c0
[   14.042982]  [<ffffffff8145f658>] register_jprobes+0x98/0xc0
[   14.042982]  [<ffffffff8145f69a>] register_jprobe+0x1a/0x20
[   14.042982]  [<ffffffffa02a5f5d>] mlx4_stats_sysfs_create+0x2d/0x150 [mlx4_en]
[   14.042982]  [<ffffffffa02a4087>] mlx4_en_init_netdev+0xc77/0xe50 [mlx4_en]
[   14.042982]  [<ffffffffa029263a>] mlx4_en_add+0x44a/0x550 [mlx4_en]
[   14.042982]  [<ffffffffa02d2d1c>] mlx4_add_device+0x4c/0xe0 [mlx4_core]
[   14.042982]  [<ffffffffa02b6000>] ? 0xffffffffa02b5fff
[   14.042982]  [<ffffffffa02d2e23>] mlx4_register_interface+0x73/0xb0 [mlx4_core]
[   14.042982]  [<ffffffffa02b6000>] ? 0xffffffffa02b5fff
[   14.042982]  [<ffffffffa02b6031>] __init_backport+0x31/0x1000 [mlx4_en]
[   14.042982]  [<ffffffff810002b2>] do_one_initcall+0x122/0x180
[   14.042982]  [<ffffffff8109ed2d>] sys_init_module+0xbd/0x220
[   14.042982]  [<ffffffff81460c99>] system_call_fastpath+0x16/0x1b
[   14.042982] Code: 66 66 2e 0f 1f 84 00 00 00 00 00 48 8b 05 49 e9 80 00 55 48 89 e5 48 3d e0 67 81 81 48 8d 48 d0 74 59 66 0f 1f 84 00 00 00 00 00 <48> 3b 71 20 72 3a 48 3b 79 28 77 34 48 8b 41 10 4c 8b 41 18 4c
[   14.042982] RIP  [<ffffffff81007eb0>] alternatives_text_reserved+0x20/0x80
[   14.042982]  RSP <ffff880ea355bcb8>