netdev - Re: [PATCH net] net: netem: fix use after free and double free with packet corruption

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20190614171644.766547d1@cakuba.netronome.com>
Date:   Fri, 14 Jun 2019 17:16:44 -0700
From:   Jakub Kicinski <jakub.kicinski@...ronome.com>
To:     Cong Wang <xiyou.wangcong@...il.com>
Cc:     David Miller <davem@...emloft.net>,
        Stephen Hemminger <stephen@...workplumber.org>,
        Jamal Hadi Salim <jhs@...atatu.com>,
        Jiri Pirko <jiri@...nulli.us>,
        netem@...ts.linux-foundation.org,
        Linux Kernel Network Developers <netdev@...r.kernel.org>,
        oss-drivers@...ronome.com, Eric Dumazet <edumazet@...gle.com>,
        posk@...gle.com
Subject: Re: [PATCH net] net: netem: fix use after free and double free with
 packet corruption

On Fri, 14 Jun 2019 09:40:18 -0700, Cong Wang wrote:
> On Wed, Jun 12, 2019 at 11:52 AM Jakub Kicinski wrote:
> >
> > Brendan reports that the use of netem's packet corruption capability
> > leads to strange crashes.  This seems to be caused by
> > commit d66280b12bd7 ("net: netem: use a list in addition to rbtree")
> > which uses skb->next pointer to construct a fast-path queue of
> > in-order skbs.
> >
> > Packet corruption code has to invoke skb_gso_segment() in case
> > of skbs in need of GSO.  skb_gso_segment() returns a list of
> > skbs.  If next pointers of the skbs on that list do not get cleared
> > fast path list goes into the weeds and tries to access the next
> > segment skb multiple times.  
> 
> Mind to be more specific? How could it be accessed multiple times?

You're right, the commit message is not great :S

So we segment an skb and get a list:

A -> B -> C

And then we hook in A to the t_head t_tail list:

h t
|/
A -> B -> C

Now if B and C get also get enqueued successfully all is fine, because
we will overwrite the list in order.  IOW:

h    t
|    |
A -> B -> C 

h         t
|         |
A -> B -> C 

But if B and C get reordered we may end up with

h t               RB
|/                |
A -> B -> C       B
                    \
                      C

Or if they get dropped (overlimits) just:

h t
|/
A -> B -> C

while A and B are already freed.

> > Reported-by: Brendan Galloway <brendan.galloway@...ronome.com>
> > Fixes: d66280b12bd7 ("net: netem: use a list in addition to rbtree")
> > Signed-off-by: Jakub Kicinski <jakub.kicinski@...ronome.com>
> > Reviewed-by: Dirk van der Merwe <dirk.vandermerwe@...ronome.com>
> > ---
> >  net/sched/sch_netem.c | 11 ++++-------
> >  1 file changed, 4 insertions(+), 7 deletions(-)
> >
> > diff --git a/net/sched/sch_netem.c b/net/sched/sch_netem.c
> > index 956ff3da81f4..1fd4405611e5 100644
> > --- a/net/sched/sch_netem.c
> > +++ b/net/sched/sch_netem.c
> > @@ -494,16 +494,13 @@ static int netem_enqueue(struct sk_buff *skb, struct Qdisc *sch,
> >          */
> >         if (q->corrupt && q->corrupt >= get_crandom(&q->corrupt_cor)) {
> >                 if (skb_is_gso(skb)) {
> > -                       segs = netem_segment(skb, sch, to_free);
> > -                       if (!segs)
> > +                       skb = netem_segment(skb, sch, to_free);
> > +                       if (!skb)
> >                                 return rc_drop;
> > -               } else {
> > -                       segs = skb;
> > +                       segs = skb->next;
> > +                       skb_mark_not_on_list(skb);
> >                 }
> >
> > -               skb = segs;
> > -               segs = segs->next;
> > -  
> 
> I don't see how this works when we hit goto finish_segs?
> Either goto finish_segs can be removed or needs to be fixed?

Note that I'm removing the else branch.  So for non-GSO we end up with:

	skb = original
	segs = NULL

for GSO we end up with:

	skb = first seg (->next == NULL)
	segs = second seg (->next == third, etc.)

So should work all as is?