netdev - Re: selftests/net: udpgso: LTS kernels supportability ?

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <fd4c527a-daeb-1c73-c423-95d02e1b8e86@kernel.org>
Date:   Tue, 18 Dec 2018 07:53:36 -0700
From:   shuah <shuah@...nel.org>
To:     Rafael David Tinoco <rafael.tinoco@...aro.org>
Cc:     "David S. Miller" <davem@...emloft.net>, netdev@...r.kernel.org,
        linux-kselftest@...r.kernel.org, linux-kernel@...r.kernel.org,
        Willem de Bruijn <willemb@...gle.com>,
        Dan Rue <dan.rue@...aro.org>,
        Anders Roxell <anders.roxell@...aro.org>,
        shuah <shuah@...nel.org>
Subject: Re: selftests/net: udpgso: LTS kernels supportability ?

On 12/18/18 4:37 AM, Rafael David Tinoco wrote:
> On 12/17/18 4:42 PM, shuah wrote:
>> Hi Rafael,
>>
>> On 12/17/18 10:53 AM, Rafael David Tinoco wrote:
>>> Shuah,
>>>
>>> I was recently investigating some errors coming out of our functional
>>> tests and we, Dan and I, came up with a discussion that might not be new
>>> for you, but, interests us, in defining how to better use kselftests as
>>> a regression mechanism/tool in our LKFT (https://lkft.linaro.org).
>>>
>>> David / Willem,
>>>
>>> I'm only using udpgso as an example for what I'd like to ask Shuah. Feel
>>> free to jump in in the discussion if you think its worth.
>>>
>>> All,
>>>
>>> Regarding: udpgso AND https://bugs.linaro.org/show_bug.cgi?id=3980
>>>
>>> udpgso tests are failing in kernels bellow 4.18 because of 2 main
>>> reasons:
>>>
>>> 1) udp4_ufo_fragment does not seem to demand the GSO SKB to be > than
>>> the MTU for older kernels (4th test case in udpgso.c).
>>>
>>> 2) setsockopt(...UDP_SEGMENT) support is not present for older kernels.
>>> (commits "udp: generate gso with UDP_SEGMENT" and its fixes seem to be
>>> needed).
>>
>> This case is easy right? Based on the test output below , I can see that
>> the failure is due to
>>
>> ./udpgso: setsockopt udp segment: Protocol not available. setsockopt()
>> is returning an error to clearly indicate that this options isn't
>> supported. This will be a test change to say test is a skip as opposed
>> to fail.
> 
> You referred to (2). (1) isn't that straightforward.
> 
>> We have a solution for this - test should SKIP as opposed to FAIL.
>>
>>> With that explained, finally the question/discussion:
>>>
>>> Shouldn't we enforce a versioning mechanism for tests that are testing
>>> recently added features ? I mean, some of the tests inside udpgso
>>> selftest are good enough for older kernels...
>>
>> Right - we do have generic way to handle that by detecting if feature is
>> supported and skip instead of using Kernel version which is going to be
>> hard to maintain.
> 
> You can't distinguish case (1) failures between real failures OR older
> kernel behaving differently then testcase expects.
> 
>>>
>>> But, because we have no control over "kernel features" and "supported
>>> test cases", we, Linaro, have to end up blacklisting all selftests that
>>> have new feature oriented tests, because one or two test cases only.
>>>

Can you share the blacklisted tests?

thanks,
-- Shuah