linux-kernel - Re: [PATCH v2 2/4] llvm-cov: add Clang's MC/DC support

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives
Hash Suite: Windows password security audit tool. GUI, reports in PDF.
[<prev] [next>] [<thread-prev] [thread-next>] [day] [month] [year] [list]
Message-Id: <20241003031429.46276-1-wentaoz5@illinois.edu>
Date: Wed,  2 Oct 2024 22:14:29 -0500
From: Wentao Zhang <wentaoz5@...inois.edu>
To: nathan@...nel.org
Cc: Matt.Kelly2@...ing.com,
	akpm@...ux-foundation.org,
	andrew.j.oppelt@...ing.com,
	anton.ivanov@...bridgegreys.com,
	ardb@...nel.org,
	arnd@...db.de,
	bhelgaas@...gle.com,
	bp@...en8.de,
	chuck.wolber@...ing.com,
	dave.hansen@...ux.intel.com,
	dvyukov@...gle.com,
	hpa@...or.com,
	jinghao7@...inois.edu,
	johannes@...solutions.net,
	jpoimboe@...nel.org,
	justinstitt@...gle.com,
	kees@...nel.org,
	kent.overstreet@...ux.dev,
	linux-arch@...r.kernel.org,
	linux-efi@...r.kernel.org,
	linux-kbuild@...r.kernel.org,
	linux-kernel@...r.kernel.org,
	linux-trace-kernel@...r.kernel.org,
	linux-um@...ts.infradead.org,
	llvm@...ts.linux.dev,
	luto@...nel.org,
	marinov@...inois.edu,
	masahiroy@...nel.org,
	maskray@...gle.com,
	mathieu.desnoyers@...icios.com,
	matthew.l.weber3@...ing.com,
	mhiramat@...nel.org,
	mingo@...hat.com,
	morbo@...gle.com,
	ndesaulniers@...gle.com,
	oberpar@...ux.ibm.com,
	paulmck@...nel.org,
	peterz@...radead.org,
	richard@....at,
	rostedt@...dmis.org,
	samitolvanen@...gle.com,
	samuel.sarkisian@...ing.com,
	steven.h.vanderleest@...ing.com,
	tglx@...utronix.de,
	tingxur@...inois.edu,
	tyxu@...inois.edu,
	wentaoz5@...inois.edu,
	x86@...nel.org
Subject: Re: [PATCH v2 2/4] llvm-cov: add Clang's MC/DC support

Hi Nathan,

Thanks for your review! See some of my responses below inline. Other
comments, including those to [1/4] and [4/4], are acknowledged and will be
updated in v3.

On 2024-10-01 20:10, Nathan Chancellor wrote:
> ...
> > maximum value supported by Clang is 32767. According to local experiments,
> > the working maximum for Linux kernel is 46, with the largest decisions in
> > kernel codebase (with 47 conditions, as of v6.11) excluded, otherwise the
> > kernel image size limit will be exceeded. The largest decisions in kernel
> > are contributed for example by macros checking CPUID.
> > 
> > Code exceeding LLVM_COV_KERNEL_MCDC_MAX_CONDITIONS will produce compiler
> > warnings.
> > 
> > As of LLVM 19, certain expressions are still not covered, and will produce
> > build warnings when they are encountered:
> > 
> > "[...] if a boolean expression is embedded in the nest of another boolean
> >  expression but separated by a non-logical operator, this is also not
> >  supported. For example, in x = (a && b && c && func(d && f)), the d && f
> >  case starts a new boolean expression that is separated from the other
> >  conditions by the operator func(). When this is encountered, a warning
> >  will be generated and the boolean expression will not be
> >  instrumented." [4]
> 
> These two sets of warnings appear to be pretty noisy in my build
> testing... Is there any way to shut them up? Perhaps it is good for

These two warnings are currently implemented as custom diagnostic in
clang/lib/CodeGen/CodeGenPGO.cpp:dataTraverseStmtPost. So I'm afraid there
is no corresponding "-W[no-]xxx" flag at this moment. I agree such switches
would be desirable but we might have to nudge this in LLVM community.

> users to see these limitations but it basically makes the build output
> useless. If there were switches, then they could be disabled in the
> default case with a Kconfig option to turn them on if the user is
> concerned with seeing which parts of their code are not instrumented. I
> could see developers wanting to run this for writing tests and they
> might not care about this as much as someone else might.
> 
> I did leave LLVM_COV_KERNEL_MCDC_MAX_CONDITIONS at its default value.
> Perhaps there is a more reasonable default that would result in less
> noisy build output but not run afoul of potential memory usage concerns?
> I assume that mention means that memory usage may be a concern for the
> type of deployments this technology would commonly be used with?

To my own experiences, enlarging this threshold won't really help with the
issue, because the other type of warning ("nested boolean") is even more
prevalent in kernel codebase. I once built the kernel serially and counted
the number of instances from the gigantic log:

  unsupported number of conditions (>6): 837
  unsupported nested boolean:            8029

So again we should probably improve this on the tool side. I can talk to
developers there separately.

> ...
> > diff --git a/Makefile b/Makefile
> > index 51498134c..1185b38d6 100644
> > --- a/Makefile
> > +++ b/Makefile
> > @@ -740,6 +740,12 @@ all: vmlinux
> >  CFLAGS_LLVM_COV := -fprofile-instr-generate -fcoverage-mapping
> >  export CFLAGS_LLVM_COV
> >  
> > +CFLAGS_LLVM_COV_MCDC := -fcoverage-mcdc
> > +ifdef CONFIG_LLVM_COV_KERNEL_MCDC_MAX_CONDITIONS
> > +CFLAGS_LLVM_COV_MCDC += -Xclang -fmcdc-max-conditions=$(CONFIG_LLVM_COV_KERNEL_MCDC_MAX_CONDITIONS)
> 
> Why is -Xclang needed here? Is this not a full frontend flag?

"-fmcdc-max-conditions" is a cc1 option only, while "-fcoverage-mcdc" is
both a cc1 option and a clang option. See llvm/llvm-project#82448 and their
changes to clang/include/clang/Driver/Options.td.

Thanks,
Wentao

> 
> > +endif
> > +export CFLAGS_LLVM_COV_MCDC
> > +
> >  CFLAGS_GCOV	:= -fprofile-arcs -ftest-coverage
> >  ifdef CONFIG_CC_IS_GCC
> >  CFLAGS_GCOV	+= -fno-tree-loop-im