Date:   Fri, 17 Apr 2020 03:39:49 +0200
From:   Jann Horn <jannh@...gle.com>
To:     Alexei Starovoitov <alexei.starovoitov@...il.com>
Cc:     bpf <bpf@...r.kernel.org>, Alexei Starovoitov <ast@...nel.org>,
        Daniel Borkmann <daniel@...earbox.net>,
        Martin KaFai Lau <kafai@...com>,
        Song Liu <songliubraving@...com>, Yonghong Song <yhs@...com>,
        Andrii Nakryiko <andriin@...com>,
        John Fastabend <john.fastabend@...il.com>,
        KP Singh <kpsingh@...omium.org>,
        Network Development <netdev@...r.kernel.org>
Subject: Re: [PATCH bpf] bpf: Use pointer type whitelist for XADD

On Fri, Apr 17, 2020 at 2:41 AM Alexei Starovoitov
<alexei.starovoitov@...il.com> wrote:
> On Fri, Apr 17, 2020 at 12:34:42AM +0200, Jann Horn wrote:
> > On Thu, Apr 16, 2020 at 11:11 PM Alexei Starovoitov
> > <alexei.starovoitov@...il.com> wrote:
> > > On Wed, Apr 15, 2020 at 10:47:43PM +0200, Jann Horn wrote:
> > > > At the moment, check_xadd() uses a blacklist to decide whether a given
> > > > pointer type should be usable with the XADD instruction. Out of all the
> > > > pointer types that check_mem_access() accepts, only four are currently let
> > > > through by check_xadd():
> > > >
> > > > PTR_TO_MAP_VALUE
> > > > PTR_TO_CTX           rejected
> > > > PTR_TO_STACK
> > > > PTR_TO_PACKET        rejected
> > > > PTR_TO_PACKET_META   rejected
> > > > PTR_TO_FLOW_KEYS     rejected
> > > > PTR_TO_SOCKET        rejected
> > > > PTR_TO_SOCK_COMMON   rejected
> > > > PTR_TO_TCP_SOCK      rejected
> > > > PTR_TO_XDP_SOCK      rejected
> > > > PTR_TO_TP_BUFFER
> > > > PTR_TO_BTF_ID
> > > >
> > > > Looking at the currently permitted ones:
> > > >
> > > >  - PTR_TO_MAP_VALUE: This makes sense and is the primary usecase for XADD.
> > > >  - PTR_TO_STACK: This doesn't make much sense, there is no concurrency on
> > > >    the BPF stack. It also causes confusion further down, because the first
> > > >    check_mem_access() won't check whether the stack slot being read from is
> > > >    STACK_SPILL and the second check_mem_access() assumes in
> > > >    check_stack_write() that the value being written is a normal scalar.
> > > >    This means that unprivileged users can leak kernel pointers.
> > > >  - PTR_TO_TP_BUFFER: This is a local output buffer without concurrency.
> > > >  - PTR_TO_BTF_ID: This is read-only, XADD can't work. When the verifier
> > > >    tries to verify XADD on such memory, the first check_ptr_to_btf_access()
> > > >    invocation gets confused by value_regno not being a valid array index
> > > >    and writes to out-of-bounds memory.
> > >
> > > > Limit XADD to PTR_TO_MAP_VALUE, since everything else at least doesn't make
> > > > sense, and is sometimes broken on top of that.
> > > >
> > > > Fixes: 17a5267067f3 ("bpf: verifier (add verifier core)")
> > > > Signed-off-by: Jann Horn <jannh@...gle.com>
> > > > ---
> > > > I'm just sending this on the public list, since the worst-case impact for
> > > > non-root users is leaking kernel pointers to userspace. In a context where
> > > > you can reach BPF (no sandboxing), I don't think that kernel ASLR is very
> > > > effective at the moment anyway.
> > > >
> > > > This breaks ten unit tests that assume that XADD is possible on the stack,
> > > > and I'm not sure how all of them should be fixed up; I'd appreciate it if
> > > > someone else could figure out how to fix them. I think some of them might
> > > > be using XADD to cast pointers to numbers, or something like that? But I'm
> > > > not sure.
> > > >
> > > > Or is XADD on the stack actually something you want to support for some
> > > > reason, meaning that that part would have to be fixed differently?
> > >
> > > yeah. 'doesnt make sense' is relative.
> > > I prefer to fix the issues instead of disabling them.
> > > xadd to PTR_TO_STACK, PTR_TO_TP_BUFFER, PTR_TO_BTF_ID should all work
> > > because they are direct pointers to objects.
> >
> > PTR_TO_STACK and PTR_TO_TP_BUFFER I can sort of understand. But
> > PTR_TO_BTF_ID is always readonly, so XADD on PTR_TO_BTF_ID really
> > doesn't make any sense AFAICS.
>
> Not quite. See bpf_tcp_ca_btf_struct_access(). Few fields of one specific
> 'struct tcp_sock' are whitelisted for write.

Oh... but that kind of thing is not really safe, right? While there
aren't really any pointers to struct tcp_sock in the kernel, I've
noticed that there are also some helpers that take ARG_PTR_TO_BTF_ID
arguments, which is kind of similar; and those look like it wouldn't
be hard for root to abuse them to corrupt kernel memory. E.g.
bpf_skb_output_proto is reachable from tracing programs, so I expect
that it'd be pretty easy to corrupt kernel memory with that.

As far as I can tell, fundamentally, BPF must not write through BTF
pointers because the BPF verifier can't guarantee that BTF pointers
actually point to the type they're supposed to point to.

> > > Unlike pointer to ctx and flow_key that will be rewritten and are not
> > > direct pointers.
> > >
> > > Short term I think it's fine to disable PTR_TO_TP_BUFFER because
> > > prog breakage is unlikely (if it's actually broken which I'm not sure yet).
> > > But PTR_TO_BTF_ID and PTR_TO_STACK should be fixed.
> > > The former could be used in bpf-tcp-cc progs. I don't think it is now,
> > > but it's certainly conceivable.
> > > PTR_TO_STACK should continue to work because tests are using it.
> > > 'but stack has no concurrency' is not an excuse to break tests.
> >
> > Meh, if you insist, I guess I can patch it differently. Although I
> > really think that "tests abuse it as a hack" shouldn't be a reason to
> > keep around functionality that doesn't make sense for production use.
>
> The pointer could have reached __sync_fetch_and_add() via two different paths
> just to simplify the C code:
> if (..)
>  my_value = lookup();
> else
>  my_value = &my_init_value;
> __sync_fetch_and_add(&my_init_value->counter, 1);

Yeah, okay, I guess that could happen.

Powered by blists - more mailing lists