linux-kernel - Re: [PATCH 09/11] tools/objtool: Copy hashtable.h into tools directory

lists.openwall.net		lists / announce owl-users owl-dev john-users john-dev passwdqc-users yescrypt popa3d-users / oss-security kernel-hardening musl sabotage tlsify passwords / crypt-dev xvendor / Bugtraq Full-Disclosure linux-kernel linux-netdev linux-ext4 linux-hardening linux-cve-announce PHC
Open Source and information security mailing list archives

Hash Suite: Windows password security audit tool. GUI, reports in PDF.

[<prev] [next>] [<thread-prev] [day] [month] [year] [list]

Message-ID: <20160309183945.GA17956@gmail.com>
Date:	Wed, 9 Mar 2016 19:39:45 +0100
From:	Ingo Molnar <mingo@...nel.org>
To:	Josh Poimboeuf <jpoimboe@...hat.com>
Cc:	Arnaldo Carvalho de Melo <acme@...radead.org>,
	Thomas Gleixner <tglx@...utronix.de>,
	"H. Peter Anvin" <hpa@...or.com>, x86@...nel.org,
	linux-kernel@...r.kernel.org, live-patching@...r.kernel.org,
	Michal Marek <mmarek@...e.cz>,
	Peter Zijlstra <peterz@...radead.org>,
	Andy Lutomirski <luto@...nel.org>,
	Borislav Petkov <bp@...en8.de>,
	Linus Torvalds <torvalds@...ux-foundation.org>,
	Andi Kleen <andi@...stfloor.org>,
	Pedro Alves <palves@...hat.com>,
	Namhyung Kim <namhyung@...il.com>,
	Bernd Petrovitsch <bernd@...rovitsch.priv.at>,
	Chris J Arges <chris.j.arges@...onical.com>,
	Andrew Morton <akpm@...ux-foundation.org>,
	Jiri Slaby <jslaby@...e.cz>,
	Arnaldo Carvalho de Melo <acme@...nel.org>
Subject: Re: [PATCH 09/11] tools/objtool: Copy hashtable.h into tools
 directory

* Josh Poimboeuf <jpoimboe@...hat.com> wrote:

> > So it would be nice to also add a build time warning if the 'upstream' copy of 
> > hashtable.h deviates from the tooling file.
> > 
> > Just like you are already doing it for other files:
> > 
> > objtool/Makefile:       diff -I'^#include' arch/x86/insn/insn.c ../../arch/x86/lib/insn.c >/dev/null && \
> > 
> > Btw., eventually we might want to factor out such duplication into a single 
> > place in tools/lib/ or so, to only have a 'master copy' (upstream kernel 
> > source), and the tooling copy.
> 
> Yeah, since we already have at least two instances of this type of diffing in 
> the tools tree, it would be good to generalize these diffs in a common place 
> with a common kernel build target.  I'll add it to my TODOs.
> 
> BTW, do you know why the policy is to copy them instead of just including the 
> kernel versions?

Yeah, so we started with that, but it occasionally resulted in build failures 
being introduced on the kernel side, which wasn't noticed on the tooling side 
immediately. (as you don't normally build tooling if you build the kernel.)

Linus (rightfully) complained about that kind of overly tight coupling, and we 
figured out the diff method you can see on the tooling side: we copy files but 
don't let them stay forked too long: we detect differences and warn about them in 
a soft fashion (just emit a warning), without breaking the tooling side or the 
kernel side.

And you are perfectly right that this should be made an integral, easy to utilize 
part of the tooling build system.

> Or if the goal is only to make it possible to compile kernel headers in user 
> space, then we may only need to copy or recreate some headers and I may have 
> been overzealous with my copying.

So the goal is to have all the benefits of sharing code (kernel libraries and 
headers are cool and well-maintained), without any of the disadvantages of such 
tight code sharing! ;-)

That's why we came up with this method, which is essentially a distributed file 
system with a manual, high latency cache coherency protocol.

Thanks,

	Ingo