linux-kernel - RE: [PATCH v7 00/13] selftests/resctrl: Add resctrl selftest

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]

Date:   Fri, 10 May 2019 19:20:50 +0000
From:   "Yu, Fenghua" <fenghua.yu@...el.com>
To:     Andre Przywara <andre.przywara@....com>
CC:     Thomas Gleixner <tglx@...utronix.de>,
        Ingo Molnar <mingo@...hat.com>, Borislav Petkov <bp@...en8.de>,
        H Peter Anvin <hpa@...or.com>,
        "Luck, Tony" <tony.luck@...el.com>,
        "Chatre, Reinette" <reinette.chatre@...el.com>,
        "Shankar, Ravi V" <ravi.v.shankar@...el.com>,
        "Shen, Xiaochen" <xiaochen.shen@...el.com>,
        "Pathan, Arshiya Hayatkhan" <arshiya.hayatkhan.pathan@...el.com>,
        "Prakhya, Sai Praneeth" <sai.praneeth.prakhya@...el.com>,
        Babu Moger <babu.moger@....com>,
        linux-kernel <linux-kernel@...r.kernel.org>,
        James Morse <James.Morse@....com>
Subject: RE: [PATCH v7 00/13] selftests/resctrl: Add resctrl selftest

>On Friday, May 10, 2019 10:36 AM
>Andre Przywara [mailto:andre.przywara@....com] wrote:
> On Sat,  9 Feb 2019 18:50:29 -0800
> Fenghua Yu <fenghua.yu@...el.com> wrote:
> 
> Hi Fenghua, Babu,
> 
> > With more and more resctrl features are being added by Intel, AMD and
> > ARM, a test tool is becoming more and more useful to validate that
> > both hardware and software functionalities work as expected.
> 
> That's very much appreciated! We decided to use that tool here to detect
> regressions in James' upcoming resctrl rework series. While doing so we
> spotted some shortcomings:

This is great!

> - There is some unconditional x86 inline assembly which obviously breaks
> the build on ARM.

Will fix this as much as possible.

BTW, does ARM support perf imc_count events which are used in CAT tests?

> - The result output is somewhat confusing, I find it hard to see whether a
> test succeeded or not, also it's not easily parseable by scripts.
> - There are some basic tests missing (does the filesystem exist? Is the
> mount point visible?)

Is it OK to fold the basic tests into bigger tests? E.g. the filesystem and mount point
tests are actually part of CAT test and we just dump the sub-test results during CAT test.
Seems your test results show the sub-test results during CAT test.

> - Some tests create files in the current directory. This is somewhat confusing
> if you happen to be in some sysfs directory, for instance. I don't think we
> really need temporary files (pipes should cover most of the cases), but in
> case we do, we should create them in some /tmp directory, probably by
> using a glibc function for this.

You are right. Will generate the temp file in /tmp.

> - There were some problems that newer GCC and clang complained about.

Which GCC do you use to see the issues? We use 8.2.1 and no compilation issue is reported.

> 
> For now I didn't look too closely at the actual test algorithms, but decided
> to fix those things first. I replied to some of the smaller issues I spotted on
> the way in the individual patch mails.
> 
> For the bigger things (new tests, result formatting) I just went ahead and
> fixed them in my own branch [1] (work in progress!). I went with TAP
> formatting, because this is both readable by humans and can be processed
> by the "prove" tool (or any other simple UNIX tool, for that matter).
> 
> I used this amended version to do some regression testing on James'
> resctrl rework series, which involved boot testing more than 100 patches, so
> automation is key here. The result looks like this:
> ===================
> TAP version 13
> ok kernel supports resctrl filesystem
> ok resctrl mountpoint "/sys/fs/resctrl" exists # resctrl filesystem not
> mounted # Starting MBM BW change ...
> ok MBM: bw change # SKIP memory bandwidth not supported # Starting
> MBA Schemata Change ...
> ok MBA: change-schemata # SKIP memory bandwidth not supported #
> Starting CQM test ...
> ok mounting resctrl to "/sys/fs/resctrl"
> ok CQM: test # SKIP L3 not supported
> # Starting CAT test ...
> ok mounting resctrl to "/sys/fs/resctrl"
> cache size :31457280
> ok writing benchmark parameters to resctrl FS ok Write schema "L3:0=7fff"
> to resctrl FS ok cache difference more than 4 % # Percent diff=16 # Number
> of bits: 15 # Avg_llc_perf_miss: 306994 # Allocated cache lines: 368640 ok
> CAT: test
> 1..11
> =================
> 
> Appreciate any feedback on this!
> 
> Cheers,
> Andre.
> 
> [1] http://linux-arm.org/git?p=linux-ap.git;a=shortlog;h=refs/heads/resctrl-
> selftests-wip
> git://linux-arm.org/linux-ap.git    branch: resctrl-selftests-wip