linux-kernel - Re: [PATCH] selftests: use $SHELL to exec selftests

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Date:   Thu, 3 Aug 2017 22:23:31 +0200
From:   "Luis R. Rodriguez" <mcgrof@...nel.org>
To:     Kees Cook <keescook@...omium.org>
Cc:     "Luis R. Rodriguez" <mcgrof@...nel.org>,
        Shuah Khan <shuah@...nel.org>,
        "open list:KERNEL SELFTEST FRAMEWORK" 
        <linux-kselftest@...r.kernel.org>,
        Andrew Morton <akpm@...ux-foundation.org>,
        Petr Mladek <pmladek@...e.com>,
        Dan Carpenter <dan.carpenter@...cle.com>,
        Colin King <colin.king@...onical.com>, dcb314@...mail.com,
        LKML <linux-kernel@...r.kernel.org>
Subject: Re: [PATCH] selftests: use $SHELL to exec selftests

On Thu, Aug 03, 2017 at 10:27:33AM -0700, Kees Cook wrote:
> On Thu, Aug 3, 2017 at 9:59 AM, Luis R. Rodriguez <mcgrof@...nel.org> wrote:
> > Executing selftests is fragile as if someone forgot to set a secript
> > as executable the test will fail. Setting scripts as executable is
> > desirable to enable folks to execute tests as independent units,
> > however, we can avoid the fragile errors of forgetting to set the
> > script as executable by just invoking the $SHELL for running each
> > script.
> >
> > Suggsted-by: Andrew Morton <akpm@...ux-foundation.org>
> > Signed-off-by: Luis R. Rodriguez <mcgrof@...nel.org>
> > ---
> >
> > Shuah, while the last two patches could be queued in for 4.13-final,
> > this one I think is more appropriate for v4.14-rc1 only.
> >
> >  tools/testing/selftests/lib.mk | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
> > index 959273c3a52e..2d6abb8037be 100644
> > --- a/tools/testing/selftests/lib.mk
> > +++ b/tools/testing/selftests/lib.mk
> > @@ -14,7 +14,7 @@ all: $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES)
> >  define RUN_TESTS
> >         @for TEST in $(TEST_GEN_PROGS) $(TEST_PROGS); do \
> >                 BASENAME_TEST=`basename $$TEST`;        \
> > -               cd `dirname $$TEST`; (./$$BASENAME_TEST && echo "selftests: $$BASENAME_TEST [PASS]") || echo "selftests:  $$BASENAME_TEST [FAIL]"; cd -;\
> > +               cd `dirname $$TEST`; ($$SHELL ./$$BASENAME_TEST && echo "selftests: $$BASENAME_TEST [PASS]") || echo "selftests:  $$BASENAME_TEST [FAIL]"; cd -;\
> >         done;
> >  endef
> 
> Is BASENAME_TEST always a script? Can't it be a built binary too?

True, this should only be attempted then if the file is *not* a shell script.
Furthermore any interpreter could be used, so using a shell would not work
for all anyway. So this would be wrong too:

diff --git a/tools/testing/selftests/lib.mk b/tools/testing/selftests/lib.mk
index 959273c3a52e..e963ee375d77 100644
--- a/tools/testing/selftests/lib.mk
+++ b/tools/testing/selftests/lib.mk
@@ -14,7 +14,12 @@ all: $(TEST_GEN_PROGS) $(TEST_GEN_PROGS_EXTENDED) $(TEST_GEN_FILES)
 define RUN_TESTS
 	@for TEST in $(TEST_GEN_PROGS) $(TEST_PROGS); do \
 		BASENAME_TEST=`basename $$TEST`;	\
-		cd `dirname $$TEST`; (./$$BASENAME_TEST && echo "selftests: $$BASENAME_TEST [PASS]") || echo "selftests:  $$BASENAME_TEST [FAIL]"; cd -;\
+		USE_SHELL="";				\
+		if [ ! -x $$BASENAME_TEST ]; then	\
+			USE_SHELL="$$SHELL";		\
+			echo "Warning: file $$BASENAME_TEST is not executable, correct this. Assuming we can run it as a shell script";\
+		fi;					\
+		cd `dirname $$TEST`; ($$USE_SHELL ./$$BASENAME_TEST && echo "selftests: $$BASENAME_TEST [PASS]") || echo "selftests:  $$BASENAME_TEST [FAIL]"; cd -;\
 	done;
 endef
 
Just distinguishing between an ELF binary and an interpreted script seems
rather hacky here too -- even if say we use head -c 2 and grep for "#!" -- this
all just seems convoluted for a Makefile. As such for now I think its best we
just inform the user that the reason why a test failed was the lack of the
executable bit.

Will send patch follow up as v2.

  Luis