linux-kernel - Re: [PATCH] sched/core: Fix forceidle balancing

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <CAEXW_YSHVUgqc_Uvx6dyiC2ynYTi2YLTuX87KkrAJDpm6D0EzQ@mail.gmail.com>
Date:   Tue, 5 Apr 2022 15:25:36 -0400
From:   Joel Fernandes <joel@...lfernandes.org>
To:     Peter Zijlstra <peterz@...radead.org>
Cc:     Ingo Molnar <mingo@...nel.org>,
        Steven Rostedt <rostedt@...dmis.org>,
        Vincent Guittot <vincent.guittot@...aro.org>,
        LKML <linux-kernel@...r.kernel.org>,
        Thomas Gleixner <tglx@...utronix.de>,
        Sebastian Andrzej Siewior <bigeasy@...utronix.de>,
        Dietmar Eggemann <dietmar.eggemann@....com>,
        Ben Segall <bsegall@...gle.com>, Mel Gorman <mgorman@...e.de>,
        Daniel Bristot de Oliveira <bristot@...hat.com>,
        Guenter Roeck <groeck@...omium.org>
Subject: Re: [PATCH] sched/core: Fix forceidle balancing

On Fri, Apr 1, 2022 at 7:46 AM Peter Zijlstra <peterz@...radead.org> wrote:
>
> On Thu, Mar 31, 2022 at 03:00:40PM -0400, Joel Fernandes wrote:
> > Hi,
> >
> > By the way, might be slightly related - we still see crashes with
> > pick_task_fair() in our kernel even with this change:
> > https://lkml.org/lkml/2020/11/17/2137
>
> Please as to not use lkml.org. Please use something with a MsgID in like
> lore.

Yep, will do.

> > Is it possible that when doing pick_task_fair() especially on a remote
> > CPU, both the "cfs_rq->curr" and the rbtree's "left" be NULL with core
> > scheduling? In this case, se will be NULL and can cause crashes right?
> > I think the code assumes this can never happen.
> >
> > +Guenter Roeck  kindly debugged pick_task_fair() in a crash as
> > follows. Copying some details he mentioned in a bug report:
> >
> > Assembler/source:
> >
> >   25:   e8 4f 11 00 00          call   0x1179             ; se =
> > pick_next_entity(cfs_rq, curr);
> >   2a:*  48 8b 98 60 01 00 00    mov    0x160(%rax),%rbx   ; trapping
> > instruction [cfs_rq = group_cfs_rq(se);]
> >   31:   48 85 db                test   %rbx,%rbx
> >   34:   75 d1                   jne    0x7
> >   36:   48 89 c7                mov    %rax,%rdi
> >
> > At 2a: RAX = se == NULL after pick_next_entity(). Looking closely into
> > pick_next_entity(), it can indeed return NULL if curr is NULL and if
> > left in pick_next_entity() is NULL. Per line 7:, curr is in %r14 and
> > indeed 0.
> >
> > Thoughts?
>
> It is possible for ->curr and ->leftmost to be NULL, but then we should
> also be having ->nr_running == 0 and not call pick in the first place.
> Because picking a task from no tasks doesn't make much sense.

Indeed the code checks for nr_running so it is really bizarre. My
guess is this is kernel memory corruption due to an unrelated bug or
something, it is also not easy to trigger.

Thanks,

 - Joel