linux-kernel - Re: [LTP] Towards 4.14 LTS

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Mon, 20 Nov 2017 10:48:58 -0600
From:   Tom Gall <tom.gall@...aro.org>
To:     Cyril Hrubis <chrubis@...e.cz>
Cc:     linux-kernel@...r.kernel.org,
        linux- stable <stable@...r.kernel.org>,
        torvalds@...ux-foundation.org,
        Greg Kroah-Hartman <gregkh@...uxfoundation.org>,
        linux-kselftest@...r.kernel.org, ltp@...ts.linux.it,
        shuahkh@....samsung.com, Guenter Roeck <linux@...ck-us.net>
Subject: Re: [LTP] Towards 4.14 LTS



> On Nov 20, 2017, at 10:10 AM, Cyril Hrubis <chrubis@...e.cz> wrote:
> 
> Hi!
>> So why didn???t we report these? As mentioned we???ve been tossing out dodgy
>> test cases to get to a clean baseline. We don???t need or want noise. 
>> 
>> For LTS, I want the system when it detects a failure to enable a quick 
>> bisect involving the affected test bucket. Given the nature of kernel 
>> bugs tho, there is that class of bug which only happens occasionally.
> 
> From my experience debugging kernel bugs requires an actuall human
> interaction and there is only certain level of automation that can be
> achieved. Don't take me wrong, automatic bisection and other bells and
> whistles are a nice to have, but at the end of the day you usually need
> someone to reproduce/look at the problem, possibly check the source
> code, report a bug, etc. Hence it does not make much sense to have an
> automated system without dedicated engineers assigned to review the test
> results.

You are entirely right automation only gets so far. We have a few lines
of defense that probably are worth a mention.

1) infra - sometimes results/runs need to be re-run for whatever reason.
2) triage - Crappy test case or something that is real?
3) kernel - bisecting etc

We don’t have huge dedicated teams for each category but likewise each 
has a team.

> -- 
> Cyril Hrubis
> chrubis@...e.cz