linux-kernel - Re: [PATCH v2] selftests: ftrace: Allow some tests to be run in a tracing instance

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Message-Id: <20170422080036.606e14853537f2933f8c09e5@kernel.org>
Date:   Sat, 22 Apr 2017 08:00:36 +0900
From:   Masami Hiramatsu <mhiramat@...nel.org>
To:     Steven Rostedt <rostedt@...dmis.org>
Cc:     LKML <linux-kernel@...r.kernel.org>, Shuah Khan <shuah@...nel.org>,
        Masami Hiramatsu <mhiramat@...nel.org>,
        Namhyung Kim <namhyung@...nel.org>
Subject: Re: [PATCH v2] selftests: ftrace: Allow some tests to be run in a
 tracing instance

On Fri, 21 Apr 2017 10:41:44 -0400
Steven Rostedt <rostedt@...dmis.org> wrote:

> 
> An tracing instance has several of the same capabilities as the top level
> instance, but may be implemented slightly different. Instead of just writing
> tests that duplicat the same test cases of the top level instance, allow a
> test to be written for both the top level as well as for an instance.
> 
> If a test case can be run in both the top level as well as in an tracing
> instance directory, then it should add a tag "# flags: instance" in the
> header of the test file. Then after all tests have run, any test that has an
> instance flag set, will run again within a tracing instance.
> 
> Cc: Shuah Khan <shuah@...nel.org>
> Cc: Masami Hiramatsu <mhiramat@...nel.org>
> Cc: Namhyung Kim <namhyung@...nel.org>
> Signed-off-by: Steven Rostedt (VMware) <rostedt@...dmis.org>
> ---
> 
> Changes since v1: (comments from Masami Hiramatsu)
> 
>  Use test flag instead of file name change.
>  Remove -i option to run_test function
>  Use global variable to denote instance running
>  Update TRACING_DIR instead of cd'ing to another directory.
> 
>  tools/testing/selftests/ftrace/ftracetest | 40 +++++++++++++++++++++++++++++++
>  1 file changed, 40 insertions(+)
> 
> diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests/ftrace/ftracetest
> index a8631d9..9992ff6 100755
> --- a/tools/testing/selftests/ftrace/ftracetest
> +++ b/tools/testing/selftests/ftrace/ftracetest
> @@ -157,6 +157,15 @@ testcase() { # testfile
>    prlog -n "[$CASENO]$desc"
>  }
>  
> +INSTANCE=0
> +testinstance() { # testfile
> +    if grep -q '^#[ \t]*flags:.*instance' $1 ; then
> +	INSTANCE=1
> +    else
> +	INSTANCE=0
> +    fi
> +}

In this case we can just run grep.

BTW, this seems too complecated (with many similar variables).
I think we just need following patch, if we run the tests which
have "instance" flag twice on top-level and an instance.
(If you'd like to run those tests only on instance, we need
 just one more line in the main loop ;-) )

-----------
diff --git a/tools/testing/selftests/ftrace/ftracetest b/tools/testing/selftests
index 52e3c4d..bdb10e6 100755
--- a/tools/testing/selftests/ftrace/ftracetest
+++ b/tools/testing/selftests/ftrace/ftracetest
@@ -152,6 +152,10 @@ testcase() { # testfile
   prlog -n "[$CASENO]$desc"
 }
 
+test_on_instance() { # testfile
+  grep "^#[ \t]*flag:.*instance" $1
+}
+
 eval_result() { # sigval
   case $1 in
     $PASS)
@@ -266,6 +270,16 @@ for t in $TEST_CASES; do
   run_test $t
 done
 
+# Test on instance loop
+for t in $TEST_CASES; do
+  test_on_instance $t || continue
+  SAVED_TRACING_DIR=$TRACING_DIR
+  export TRACING_DIR=`mktemp -d $TRACING_DIR/instances/ftracetest.XXXXXX`
+  run_test $t
+  rmdir $TRACING_DIR
+  TRACING_DIR=$SAVED_TRACING_DIR
+done
+
 prlog ""
 prlog "# of passed: " `echo $PASSED_CASES | wc -w`
 prlog "# of failed: " `echo $FAILED_CASES | wc -w`
-----------

Would this work for you?

Thank you,

> +
>  eval_result() { # sigval
>    case $1 in
>      $PASS)
> @@ -233,18 +242,26 @@ exit_xfail () {
>  }
>  trap 'SIG_RESULT=$XFAIL' $SIG_XFAIL
>  
> +INSTANCE_DIR="."
>  __run_test() { # testfile
>    # setup PID and PPID, $$ is not updated.
>    (cd $TRACING_DIR; read PID _ < /proc/self/stat; set -e; set -x; initialize_ftrace; . $1)
>    [ $? -ne 0 ] && kill -s $SIG_FAIL $SIG_PID
>  }
>  
> +TEST_INSTANCES=0
> +
>  # Run one test case
>  run_test() { # testfile
>    local testname=`basename $1`
>    local testlog=`mktemp $LOG_DIR/${testname}-log.XXXXXX`
>    export TMPDIR=`mktemp -d /tmp/ftracetest-dir.XXXXXX`
>    testcase $1
> +  local SAVE_TRACING_DIR=$TRACING_DIR
> +  if [ $TEST_INSTANCES -eq 1 ]; then
> +    TRACING_DIR=$TRACING_DIR/instances/${testname}_test_$$
> +    mkdir $TRACING_DIR
> +  fi
>    echo "execute: "$1 > $testlog
>    SIG_RESULT=0
>    if [ $VERBOSE -ge 2 ]; then
> @@ -260,17 +277,40 @@ run_test() { # testfile
>      [ $VERBOSE -ge 1 ] && catlog $testlog
>      TOTAL_RESULT=1
>    fi
> +  if [ $TEST_INSTANCES -eq 1 ]; then
> +    rmdir $TRACING_DIR
> +  fi
> +  TRACING_DIR=$SAVE_TRACING_DIR
>    rm -rf $TMPDIR
>  }
>  
>  # load in the helper functions
>  . $TEST_DIR/functions
>  
> +RUN_INSTANCES=0
> +
>  # Main loop
>  for t in $TEST_CASES; do
> +  testinstance $t
> +  if [ $INSTANCE -eq 1 ]; then
> +    RUN_INSTANCES=1
> +  fi
>    run_test $t
>  done
>  
> +TEST_INSTANCES=1
> +
> +if [ $RUN_INSTANCES -eq 1 ]; then
> +    echo "Running tests in a tracing instance:"
> +    for t in $TEST_CASES; do
> +	testinstance $t
> +	if [ $INSTANCE -eq 0 ]; then
> +	    continue
> +	fi
> +	run_test $t
> +    done
> +fi
> +
>  prlog ""
>  prlog "# of passed: " `echo $PASSED_CASES | wc -w`
>  prlog "# of failed: " `echo $FAILED_CASES | wc -w`
> -- 
> 2.9.3
> 


-- 
Masami Hiramatsu <mhiramat@...nel.org>