linux-kernel - Re: [PATCH 1/2] selftests: ftrace: Do not failure if there is unsupported tests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite for Android: free password hash cracker in your pocket

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <20170703091543.66f6198a@gandalf.local.home>
Date:   Mon, 3 Jul 2017 09:15:43 -0400
From:   Steven Rostedt <rostedt@...dmis.org>
To:     Masami Hiramatsu <mhiramat@...nel.org>
Cc:     linux-kselftest@...r.kernel.org, shuah@...nel.org,
        Ingo Molnar <mingo@...hat.com>, linux-kernel@...r.kernel.org,
        naresh.kamboju@...aro.org
Subject: Re: [PATCH 1/2] selftests: ftrace: Do not failure if there is
 unsupported tests

On Mon,  3 Jul 2017 14:25:36 +0900
Masami Hiramatsu <mhiramat@...nel.org> wrote:

> Do not return failure exit code (1) for unsupported testcases,
> since it is expected for stable kernels.
> 
> Previously, ftracetest is expected to run only on current
> release for avoiding regressions. However, nowadays we run
> it on stable kernels. This means some test cases must return
> unsupported result. In such case, we should NOT exit
> ftracetest with error status for unsupported results so that
> kselftest (upper tests wrapper) shows it passed correctly.

I wonder if we should change kselftest instead. There are case where we
want to report "unsupported" as a failure. For instance, I have tests
where I enable everything, and if a test returns "unsupported" then it
is a failure for me.

Can we add an option in kselftest, or to ftracetest that decides if
unsupported is a failure or not? Otherwise I can not ack this patch.

-- Steve


> 
> Note that we continue to treat unresolved results as failure,
> if test writers would like to notice user that the test result
> should be reviewed, they can use exit_unresolved.
> 
> Signed-off-by: Masami Hiramatsu <mhiramat@...nel.org>
> ---
>  tools/testing/selftests/ftrace/ftracetest |    2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
> index 14a03ea..290cd42 100755
> --- a/tools/testing/selftests/ftrace/ftracetest
> +++ b/tools/testing/selftests/ftrace/ftracetest
> @@ -187,7 +187,7 @@ eval_result() { # sigval
>      $UNSUPPORTED)
>        prlog "	[UNSUPPORTED]"
>        UNSUPPORTED_CASES="$UNSUPPORTED_CASES $CASENO"
> -      return 1 # this is not a bug, but the result should be reported.
> +      return 0 # this is not a bug.
>      ;;
>      $XFAIL)
>        prlog "	[XFAIL]"