linux-kernel - Re: [PATCHv5 kselftest next] selftests/run_kselftest.sh: make each test individually selectable

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-ID: <CA+G9fYvT6Mw2BamoiVyw=wLUqD-3LB2oaDqcuabOyWfFxEN1qg@mail.gmail.com>
Date:   Fri, 25 Sep 2020 13:51:53 +0530
From:   Naresh Kamboju <naresh.kamboju@...aro.org>
To:     Hangbin Liu <liuhangbin@...il.com>
Cc:     "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>, Shuah Khan <shuah@...nel.org>,
        Jonathan Corbet <corbet@....net>, linux-doc@...r.kernel.org,
        open list <linux-kernel@...r.kernel.org>, Tim.Bird@...y.com,
        lkft-triage@...ts.linaro.org, Kees Cook <keescook@...omium.org>,
        Anders Roxell <anders.roxell@...aro.org>,
        Justin Cook <justin.cook@...aro.org>,
        Linux-Next Mailing List <linux-next@...r.kernel.org>
Subject: Re: [PATCHv5 kselftest next] selftests/run_kselftest.sh: make each
 test individually selectable

On Mon, 14 Sep 2020 at 07:53, Hangbin Liu <liuhangbin@...il.com> wrote:
>
> Currently, after generating run_kselftest.sh, there is no way to choose
> which test we could run. All the tests are listed together and we have
> to run all every time. This patch enhanced the run_kselftest.sh to make
> the tests individually selectable. e.g.
>
>   $ ./run_kselftest.sh -t "bpf size timers"

My test run break on linux next

./run_kselftest.sh: line 1331: syntax error near unexpected token `)'
./run_kselftest.sh: line 1331: `-e -s | --summary )
logfile=$BASE_DIR/output.log; cat /dev/null > $logfile; shift ;;'

steps do run:
# run_kselftest.sh file generated by kselftest Makefile and included in tarball
./run_kselftest.sh 2>&1 | tee "${LOGFILE}"

ref:
https://github.com/nareshkamboju/test-definitions/blob/master/automated/linux/kselftest/kselftest.sh#L222

full test run log:
https://lkft.validation.linaro.org/scheduler/job/1786826#L1391

>
> Before the patch:
> ================
>
> $ cat run_kselftest.sh
> \#!/bin/sh
> BASE_DIR=$(realpath $(dirname $0))
> cd $BASE_DIR
> . ./kselftest/runner.sh
> ROOT=$PWD
> if [ "$1" = "--summary" ]; then
>   logfile=$BASE_DIR/output.log
>   cat /dev/null > $logfile
> fi
> [ -w /dev/kmsg ] && echo "kselftest: Running tests in android" >> /dev/kmsg
> cd android
> run_many        \
>         "run.sh"
> cd $ROOT
> ...<snip>...
> [ -w /dev/kmsg ] && echo "kselftest: Running tests in zram" >> /dev/kmsg
> cd zram
> run_many        \
>         "zram.sh"
> cd $ROOT
>
> After the patch:
> ===============
>
> $ cat run_kselftest.sh
> \#!/bin/sh
> BASE_DIR=$(realpath $(dirname $0))
> . ./kselftest/runner.sh
> TESTS="android ...<snip>... filesystems/binderfs ...<snip>... zram"
>
> run_android()
> {
>         [ -w /dev/kmsg ] && echo "kselftest: Running tests in android" >> /dev/kmsg
>         cd android
>         run_many        \
>                 "run.sh"
>         cd $ROOT
> }
>
> ...<snip>...
>
> run_filesystems_binderfs()
> {
>         [ -w /dev/kmsg ] && echo "kselftest: Running tests in filesystems/binderfs" >> /dev/kmsg
>         cd filesystems/binderfs
>         run_many        \
>                 "binderfs_test"
>         cd $ROOT
> }
>
> ...<snip>...
>
> run_zram()
> {
>         [ -w /dev/kmsg ] && echo "kselftest: Running tests in zram" >> /dev/kmsg
>         cd zram
>         run_many        \
>                 "zram.sh"
>         cd $ROOT
> }
>
> usage()
> {
>         cat <<EOF
> usage: ${0##*/} OPTS
>         -s | --summary          Only print summary info and put detailed log in output.log
>         -t | --tests            Test name you want to run specifically
>         -h | --help             Show this usage info
> EOF
> }
>
> while true; do
>         case "$1" in
>         -s | --summary ) logfile=$BASE_DIR/output.log; cat /dev/null > $logfile; shift ;;
>         -t | --tests ) TESTS=$2; shift 2 ;;
>         -l | --list ) echo $TESTS; exit 0 ;;
>         -h | --help ) usage; exit 0 ;;
>         "" ) break;;
>         * ) usage; exit 1;;
>         esac
> done
>
> cd $BASE_DIR
> ROOT=$PWD
> for folder in $TESTS; do
>         folder=$(echo $folder | tr -s '/-' '_')
>         run_$folder
> done
>
> Signed-off-by: Hangbin Liu <liuhangbin@...il.com>

Reported-by: Naresh Kamboju <naresh.kamboju@...aro.org>

-- 
Linaro LKFT
https://lkft.linaro.org