linux-kernel - Re: KASAN: use-after-free Read in percpu_ref_switch_to_atomic

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20200306164443.GU2935@paulmck-ThinkPad-P72>
Date:   Fri, 6 Mar 2020 08:44:43 -0800
From:   "Paul E. McKenney" <paulmck@...nel.org>
To:     Jann Horn <jannh@...gle.com>
Cc:     Jens Axboe <axboe@...nel.dk>, Dmitry Vyukov <dvyukov@...gle.com>,
        syzbot <syzbot+e017e49c39ab484ac87a@...kaller.appspotmail.com>,
        Al Viro <viro@...iv.linux.org.uk>,
        io-uring <io-uring@...r.kernel.org>,
        linux-fsdevel <linux-fsdevel@...r.kernel.org>,
        Borislav Petkov <bp@...en8.de>,
        "H. Peter Anvin" <hpa@...or.com>,
        LKML <linux-kernel@...r.kernel.org>,
        Ingo Molnar <mingo@...hat.com>,
        Peter Zijlstra <peterz@...radead.org>,
        syzkaller-bugs <syzkaller-bugs@...glegroups.com>,
        Thomas Gleixner <tglx@...utronix.de>, tony.luck@...el.com,
        the arch/x86 maintainers <x86@...nel.org>,
        Dan Carpenter <dan.carpenter@...cle.com>
Subject: Re: KASAN: use-after-free Read in percpu_ref_switch_to_atomic_rcu

On Fri, Mar 06, 2020 at 04:36:20PM +0100, Jann Horn wrote:
> On Fri, Mar 6, 2020 at 4:34 PM Jens Axboe <axboe@...nel.dk> wrote:
> > On 3/6/20 7:57 AM, Jann Horn wrote:
> > > +paulmck
> > >
> > > On Wed, Mar 4, 2020 at 3:40 PM Jens Axboe <axboe@...nel.dk> wrote:
> > >> On 3/4/20 12:59 AM, Dmitry Vyukov wrote:
> > >>> On Fri, Feb 7, 2020 at 9:14 AM syzbot
> > >>> <syzbot+e017e49c39ab484ac87a@...kaller.appspotmail.com> wrote:
> > >>>>
> > >>>> Hello,
> > >>>>
> > >>>> syzbot found the following crash on:
> > >>>>
> > >>>> HEAD commit:    4c7d00cc Merge tag 'pwm/for-5.6-rc1' of git://git.kernel.o..
> > >>>> git tree:       upstream
> > >>>> console output: https://syzkaller.appspot.com/x/log.txt?x=12fec785e00000
> > >>>> kernel config:  https://syzkaller.appspot.com/x/.config?x=e162021ddededa72
> > >>>> dashboard link: https://syzkaller.appspot.com/bug?extid=e017e49c39ab484ac87a
> > >>>> compiler:       clang version 10.0.0 (https://github.com/llvm/llvm-project/ c2443155a0fb245c8f17f2c1c72b6ea391e86e81)
> > >>>>
> > >>>> Unfortunately, I don't have any reproducer for this crash yet.
> > >>>>
> > >>>> IMPORTANT: if you fix the bug, please add the following tag to the commit:
> > >>>> Reported-by: syzbot+e017e49c39ab484ac87a@...kaller.appspotmail.com
> > >>>
> > >>> +io_uring maintainers
> > >>>
> > >>> Here is a repro:
> > >>> https://gist.githubusercontent.com/dvyukov/6b340beab6483a036f4186e7378882ce/raw/cd1922185516453c201df8eded1d4b006a6d6a3a/gistfile1.txt
> > >>
> > >> I've queued up a fix for this:
> > >>
> > >> https://git.kernel.dk/cgit/linux-block/commit/?h=io_uring-5.6&id=9875fe3dc4b8cff1f1b440fb925054a5124403c3
> > >
> > > I believe that this fix relies on call_rcu() having FIFO ordering; but
> > > <https://www.kernel.org/doc/Documentation/RCU/Design/Memory-Ordering/Tree-RCU-Memory-Ordering.html#Callback%20Registry>
> > > says:
> > >
> > > | call_rcu() normally acts only on CPU-local state[...] It simply
> > > enqueues the rcu_head structure on a per-CPU list,

Indeed.  For but one example, if there was a CPU-to-CPU migration between
the two call_rcu() invocations, it would not be at all surprising for
the two callbacks to execute out of order.

> > > Is this fix really correct?
> >
> > That's a good point, there's a potentially stronger guarantee we need
> > here that isn't "nobody is inside an RCU critical section", but rather
> > that we're depending on a previous call_rcu() to have happened. Hence I
> > think you are right - it'll shrink the window drastically, since the
> > previous callback is already queued up, but it's not a full close.
> >
> > Hmm...
> 
> You could potentially hack up the semantics you want by doing a
> call_rcu() whose callback does another call_rcu(), or something like
> that - but I'd like to hear paulmck's opinion on this first.

That would work!

Or, alternatively, do an rcu_barrier() between the two calls to
call_rcu(), assuming that the use case can tolerate rcu_barrier()
overhead and latency.

							Thanx, Paul