linux-kernel - Re: [RFC PATCH v3 4/4] ftracetest: Add XFAIL/XPASS and POSIX.3 std. result codes

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <87mwag9kyp.fsf@sejong.aot.lge.com>
Date:	Thu, 04 Sep 2014 11:29:50 +0900
From:	Namhyung Kim <namhyung@...nel.org>
To:	Masami Hiramatsu <masami.hiramatsu.pt@...achi.com>
Cc:	Shuah Khan <shuah.kh@...sung.com>,
	Tom Zanussi <tom.zanussi@...ux.intel.com>,
	Yoshihiro YUNOMAE <yoshihiro.yunomae.ez@...achi.com>,
	Oleg Nesterov <oleg@...hat.com>,
	Steven Rostedt <rostedt@...dmis.org>,
	Ingo Molnar <mingo@...nel.org>,
	Linux Kernel Mailing List <linux-kernel@...r.kernel.org>
Subject: Re: [RFC PATCH v3 4/4] ftracetest: Add XFAIL/XPASS and POSIX.3 std. result codes

On Tue, 02 Sep 2014 11:06:59 +0000, Masami Hiramatsu wrote:
> Add XFAIL, XPASS and POSIX 1003.3 std. codes (UNRESOLVED/
> UNTESTED/UNSUPPORTED) as a result code. These are
> used for the results that test case is expected to fail or
> unsupported feature (by config).
> This also introduces PASS/FAIL/XFAIL/XPASS/UNSUP/UNTST/UNRES
> result codes for each testcase. Since the results are not
> binary, each testcase must use these codes to return the
> test result.
>
> Changes in v3:
>  - Add UNRESOLVED/UNTESTED codes.
>  - Fix to handle undefined codes.
>  - Add a document about return codes.
>  - Fix to show failure logs.
>  - Don't use -e option for echo since dash doesn't support it.
>
> Signed-off-by: Masami Hiramatsu <masami.hiramatsu.pt@...achi.com>
> ---

[SNIP]
> diff --git a/tools/testing/ftrace/test.d/basic3.tc b/tools/testing/ftrace/test.d/basic3.tc
> index 0c1a3a2..7bc5a53 100644
> --- a/tools/testing/ftrace/test.d/basic3.tc
> +++ b/tools/testing/ftrace/test.d/basic3.tc
> @@ -1,8 +1,9 @@
>  #!/bin/sh
>  # description: Basic trace clock test
> -[ -f trace_clock ] || exit 1
> +[ -f trace_clock ] || exit $FAIL # this is basic feature, must be there
>  for c in `cat trace_clock | tr  -d \[\]`; do
> -  echo $c > trace_clock || exit 1
> -  grep '\['$c'\]' trace_clock || exit 1
> +  echo $c > trace_clock || exit $FAIL
> +  grep '\['$c'\]' trace_clock || exit $FAIL
>  done
> -echo local > trace_clock
> +echo local > trace_clock || exit $FAIL
> +exit $PASS

IMHO it's a bit sad that we need to write every test in this way -
adding '|| exit $FAIL' for every command.  It'd be better if only
exceptional cases require that explicitly and normal cases live in
simple.

Can we do better - like using trap or something?

Thanks,
Namhyung
--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@...r.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/