netdev - Re: general protection fault in process_one

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20191031162521.GA31303@splinter>
Date:   Thu, 31 Oct 2019 18:25:21 +0200
From:   Ido Schimmel <idosch@...sch.org>
To:     Dmitry Vyukov <dvyukov@...gle.com>
Cc:     syzbot <syzbot+9ed8f68ab30761f3678e@...kaller.appspotmail.com>,
        Jiri Pirko <jiri@...lanox.com>,
        Alexei Starovoitov <ast@...nel.org>, bpf <bpf@...r.kernel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        David Miller <davem@...emloft.net>,
        Jakub Kicinski <jakub.kicinski@...ronome.com>,
        Martin KaFai Lau <kafai@...com>,
        LKML <linux-kernel@...r.kernel.org>,
        netdev <netdev@...r.kernel.org>,
        Song Liu <songliubraving@...com>,
        syzkaller-bugs <syzkaller-bugs@...glegroups.com>,
        Yonghong Song <yhs@...com>
Subject: Re: general protection fault in process_one_work

On Tue, Oct 29, 2019 at 06:54:07PM +0200, Ido Schimmel wrote:
> On Tue, Oct 29, 2019 at 10:45:19AM +0200, Ido Schimmel wrote:
> > On Tue, Oct 29, 2019 at 09:43:27AM +0100, Dmitry Vyukov wrote:
> > > On Tue, Oct 29, 2019 at 9:38 AM syzbot
> > > <syzbot+9ed8f68ab30761f3678e@...kaller.appspotmail.com> wrote:
> > > >
> > > > Hello,
> > > >
> > > > syzbot found the following crash on:
> > > >
> > > > HEAD commit:    38207291 bpf: Prepare btf_ctx_access for non raw_tp use case
> > > > git tree:       bpf-next
> > > > console output: https://syzkaller.appspot.com/x/log.txt?x=14173c0f600000
> > > > kernel config:  https://syzkaller.appspot.com/x/.config?x=41648156aa09be10
> > > > dashboard link: https://syzkaller.appspot.com/bug?extid=9ed8f68ab30761f3678e
> > > > compiler:       gcc (GCC) 9.0.0 20181231 (experimental)
> > > >
> > > > Unfortunately, I don't have any reproducer for this crash yet.
> > > >
> > > > IMPORTANT: if you fix the bug, please add the following tag to the commit:
> > > > Reported-by: syzbot+9ed8f68ab30761f3678e@...kaller.appspotmail.com
> > > 
> > > +Jiří Pírko, this seems to be related to netdevsim.
> > 
> > Will check it.
> 
> Didn't have a lot of time today, but I think the issue is a race
> condition (note that syzbot only triggered it twice so far).
> 
> Upon reload nsim_dev_port_del_all() is called and starts deleting ports
> from the ports list without holding the ports list mutex. It is possible
> that during this time nsim_dev_trap_report_work() is executing from a
> workqueue and accessing freed memory despite holding the ports list
> mutex.
> 
> I'll try to reproduce and send a fix later this week.

Sent a fix:
https://patchwork.ozlabs.org/patch/1187587/

It was quite difficult to reproduce, so I used the below patch to
increase the time window in which the race could occur. Then it became
trivial to trigger :)

diff --git a/drivers/net/netdevsim/dev.c b/drivers/net/netdevsim/dev.c
index 54ca6681ba31..d12abd84c218 100644
--- a/drivers/net/netdevsim/dev.c
+++ b/drivers/net/netdevsim/dev.c
@@ -457,6 +457,7 @@ static void nsim_dev_trap_report_work(struct work_struct *work)
         */
        mutex_lock(&nsim_dev->port_list_lock);
        list_for_each_entry(nsim_dev_port, &nsim_dev->port_list, list) {
+               msleep(100);
                if (!netif_running(nsim_dev_port->ns->netdev))
                        continue;