linux-kernel - Re: Steam is broken on new kernels

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <b72599d1-b5d5-1c23-15fc-8e2f9454af05@valvesoftware.com>
Date:   Fri, 21 Jun 2019 18:01:44 -0700
From:   "Pierre-Loup A. Griffais" <pgriffais@...vesoftware.com>
To:     Eric Dumazet <edumazet@...gle.com>,
        Linus Torvalds <torvalds@...ux-foundation.org>
CC:     Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        lkml <linux-kernel@...r.kernel.org>
Subject: Re: Steam is broken on new kernels



On 6/21/19 5:19 PM, Eric Dumazet wrote:
> On Fri, Jun 21, 2019 at 7:54 PM Linus Torvalds
> <torvalds@...ux-foundation.org> wrote:
>>
>> Eric is talking about this patch, I think:
>>
>>     https://patchwork.ozlabs.org/patch/1120222/
>>
> 
> That  is correct.
> 
> I am about to take a flight from Boston to Paris, so I can not really
> follow discussions/tests for the following hours.

I built the tip of linux-5.1.y and reproduced the issue while trying to 
log out and back into Steam; it exhibited this symptom as well:

pgriffais@...iffais:~$ nstat -az | grep -i wqueue
TcpExtTCPWqueueTooBig           31                 0.0

I applied Eric's path to the tip of the branch and ran that kernel and 
the bug didn't occur through several logout / login cycles, so things 
look good at first glance. I'll keep running that kernel and report back 
if anything crops up in the future, but I believe we're good, beyond 
getting distros to ship this additional fix.

Thanks,
  - Pierre-Loup

> 
> Thanks.
> 
>> I guess I'll ask people on the github thread to test that too.
>>
>>                    Linus
>>
>> On Fri, Jun 21, 2019 at 3:38 PM Eric Dumazet <edumazet@...gle.com> wrote:
>>>
>>> Please look at my recent patch.
>>>   Sorry I am travelling....
>>>
>>> On Fri, Jun 21, 2019, 6:19 PM Linus Torvalds <torvalds@...ux-foundation.org> wrote:
>>>>
>>>> On Fri, Jun 21, 2019 at 2:41 PM Greg Kroah-Hartman
>>>> <gregkh@...uxfoundation.org> wrote:
>>>>>
>>>>> What specific commit caused the breakage?
>>>>
>>>> Both on reddit and on github there seems to be confusion about whether
>>>> it's a problem or not. Some people have it working with the exact same
>>>> kernel that breaks for others.
>>>>
>>>> And then some people seem to say it works intermittently for them,
>>>> which seems to indicate a timing issue.
>>>>
>>>> Looking at the SACK patches (assuming it's one of them), I'd suspect
>>>> the "tcp: tcp_fragment() should apply sane memory limits".
>>>>
>>>> Eric, that one does
>>>>
>>>>         if (unlikely((sk->sk_wmem_queued >> 1) > sk->sk_sndbuf)) {
>>>>                 NET_INC_STATS(sock_net(sk), LINUX_MIB_TCPWQUEUETOOBIG);
>>>>                 return -ENOMEM;
>>>>         }
>>>>
>>>> but I think it's *normal* for "sk_wmem_queued >> 1" to be around the
>>>> same size as sk_sndbuf. So if there is some fragmentation, and we add
>>>> more skb's to it, that would seem to trigger fairly easily.
>>>> Particularly since this is all in "truesize" units, which can be a lot
>>>> bigger than the packets themselves.
>>>>
>>>> I don't know the code, so I may be out to lunch and barking up
>>>> completely the wrong tree, but that particular check does seem like it
>>>> might trigger much more easily than I think the code _intended_ it to
>>>> trigger?
>>>>
>>>> Pierre-Loup - do you guys have a test-case inside of valve? Or is this
>>>> purely "we see some people with problems"?
>>>>
>>>>                 Linus
>