[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-ID: <87mwij4nih.fsf@natisbad.org>
Date: Sun, 26 Jan 2014 01:04:06 +0100
From: arno@...isbad.org (Arnaud Ebalard)
To: Willy Tarreau <w@....eu>
Cc: Eric Dumazet <eric.dumazet@...il.com>,
David Miller <davem@...emloft.net>,
Eric Dumazet <edumazet@...gle.com>,
Daniel Borkmann <dborkman@...hat.com>,
Herbert Xu <herbert@...dor.apana.org.au>,
netdev@...r.kernel.org
Subject: Re: [BUG] null pointer dereference in tcp_gso_segment()
Hi,
Willy Tarreau <w@....eu> writes:
> On Wed, Jan 22, 2014 at 02:18:45PM -0800, Eric Dumazet wrote:
>> On Wed, 2014-01-22 at 23:02 +0100, Arnaud Ebalard wrote:
>> > Hi Eric,
>> >
>> > Eric Dumazet <eric.dumazet@...il.com> writes:
>> >
>> > >> Unless there is an assumption I missed somewhere in the function, the
>> > >> problem may occur during the first round of the loop, because (unlike
>> > >> the 'while' condition does at line 21) skb->next is not checked against
>> > >> null at lines 17 above before it is passed to tcp_hdr() at line 18.
>> > >>
>> > >> To be honest, I am asking because I am not familiar w/ the code and it
>> > >> is somewhat old so I wonder why noone got hit before. AFAICT,
>> > >> f4c50d990dcf ([NET]: Add software TSOv4) added TSOv4 support in 2006 via
>> > >> introduction of tcp_tso_segmen() (with the same kind of deref but
>> > >> possibly different assumptions) which was more recently modified via
>> > >> 28850dc7c7 (net: tcp: move GRO/GSO functions to tcp_offload) to become
>> > >> tcp_gso_segment().
>> > >>
>> > >> David, can you confirm the analysis and possibly comment on the
>> > >> conditions needed for the bug to manifest?
>> > >
>> > > A gso packet contains at least 2 segments.
>> >
>> > By whom / where is it enforced?
>>
>> For example, tcp_gso_segment() does the following check :
>>
>> if (unlikely(skb->len <= mss))
>> goto out;
>>
>> If there was one segment, then skb->len should also be smaller than mss
>>
>> Since TCP stack seemed to be the provider of the packet in your stack
>> trace, check tcp_set_skb_tso_segs()
>
> Thanks Eric for the explanation. From Arnaud's trace, I suspect that he's
> received an ACK which has released some pending data, so it's very likely
> indeed that at least two segments were released at once given that the
> receiver is likely to ACK every two segments.
>
> Also we can expect that the received ACK was copy-breaked. I don't know
> if some sort of skb recycling may happen at this stage and reveal some
> bad corner cases (eg: improperly initialized skb during the rx path that
> causes everything to break when it's recycled for the tx path), but Arnaud
> you can easily disable the rx_copybreak feature by setting the rx_copybreak
> module argument to zero. You can change it at run time in /sys/module. At
> least it will tell us if it could be related or not.
The problem is that I cannot simply use that trick to test your hypothesis
as the bug is not easily reproducible. I was lucky to trigger it twice but
never got it then (when I tested with an additonal BUG_ON(skb->next == NULL)
before the main loop in tcp_gso_segment()).
Cheers,
a+
--
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Powered by blists - more mailing lists